Skip to content

Made sortBy and sortByDesc throw when a column is not found in a group - #2179

Open
Allex-Nik wants to merge 4 commits into
masterfrom
sort-by-reporting-errors-fix
Open

Allex-Nik wants to merge 4 commits into
masterfrom
sort-by-reporting-errors-fix

Conversation

@Allex-Nik

Copy link
Copy Markdown
Collaborator

Fixes #780

Currently, if a user tries to sort rows within groups of a GroupBy using the sortBy or sortByDesc functions, and this column is not found in a group, the column is skipped and both functions finish silently without sorting anything and without throwing any exception.
The proposed changes make this functions throw an IllegalStateException with a clear report when a column the user wants to sort by is not found in a group.

Preparatory work was done in #2176

@jetbrains-air jetbrains-air Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this one — two things are worth a fix before merge, both noted inline.

Produced by Air Automations. Name: AI Review / Run: https://air.jetbrains.cloud/org/05cf1a7f-6ab5-713b-abd3-29d0c8a05e2d/automations/8daf2412-0b6c-4433-90c0-b40ab5aebcdb?run=c1b37ce6-75b9-4a47-b73c-be32d1726806

Comment thread core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/impl/api/sort.kt Outdated
Comment thread core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/api/sort.kt Outdated

@jetbrains-air jetbrains-air Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround — one more thing to fix before merge, noted inline.

Produced by Air Automations. Name: AI Review / Run: https://air.jetbrains.cloud/org/05cf1a7f-6ab5-713b-abd3-29d0c8a05e2d/automations/8daf2412-0b6c-4433-90c0-b40ab5aebcdb?run=4d940783-869f-41a8-932d-dc688c139fe3

Comment thread core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/impl/api/sort.kt Outdated

@jetbrains-air jetbrains-air Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good — approving; one non-blocking note inline for whenever you get to it.

Produced by Air Automations. Name: AI Review / Run: https://air.jetbrains.cloud/org/05cf1a7f-6ab5-713b-abd3-29d0c8a05e2d/automations/8daf2412-0b6c-4433-90c0-b40ab5aebcdb?run=9b5f720c-f4ca-4a3a-a086-c99a334771c2

.with { it.sortByImpl(UnresolvedColumnsPolicy.Skip, columns) }
.asGroupBy { it.getFrameColumn(groups.name()).castFrameColumn<G>() }
internal fun <T, G> GroupBy<T, G>.sortByImpl(columns: SortColumnsSelector<G, *>): GroupBy<T, G> {
if (groups.size() == 0 && groups.schema.value.columns.isNotEmpty()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The columns.isNotEmpty() half of this guard silently turns the new validation off whenever the group schema was lost, and nothing here or in the tests says so.

Failure scenario: df.groupBy { … }.filter { false }.sortBy("typo") returns normally (verified on this branch), while df.filter { false }.groupBy { … }.sortBy("typo") throws. GroupByImpl.filter rebuilds the frame column via df[indices].asGroupBy(groups), which drops FrameColumnImpl.columnSchema and recomputes the schema as empty, so the pre-check is skipped. No wrong data results (the GroupBy is empty either way), but the missing-column diagnostic #780 asks for is absent on that path — and the neighbouring grouped sort accepts valid selections after all groups are filtered out test only pins the happy path, so the suite doesn't record the gap.

Suggested fix: carry the original schema through when GroupByImpl.filter re-wraps the frame column; if that's out of scope here, say in a comment on this condition why validation is skipped and add a test pinning the current no-throw behaviour, so the next reader doesn't "simplify" the condition away or assume coverage is complete.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be fixed by #2154 (FrameColumnImpl)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we have groups.isEmpty if you want it :)

@Allex-Nik
Allex-Nik requested a review from Jolanrensen October 23, 2026 10:35
*
* @param cols The [Column Names][String] that defines which columns are used for sorting.
* @return A new [GroupBy] with rows sorted within each group and the original key-group order preserved.
* @throws IllegalStateException if a referenced column cannot be resolved in a group.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know our exceptions are a mess, but what do you think of IlegalArgumentException? Since the provided 'argument', namely the column, doesn't exist. On the other hand, you could say it's a "valid argument" in the function but resolving it breaks and thus we reach an illegal state... something can be said for both

@Allex-Nik Allex-Nik Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Jolanrensen I agree that IlegalArgumentException suits better. But df.sortBy("wrongCol") (and some other functions) also throws IllegalStateException because of the way the check function in DataFrameReceiver.kt is implemented:

UnresolvedColumnsPolicy.Fail -> error(formatMissingColumnMessage(path))

Changing the type of exception here changes it for some other functions as well (from tests that break after this change I can see allColsExcept, colGroup, col, frameCol, valueCol, groupBy.aggregate).

So I was thinking that it might make sense to consider it as a possible separate change in another PR. What do you think?

@Allex-Nik
Allex-Nik force-pushed the sort-by-reporting-errors-fix branch from 32c93e2 to d69f0ef Compare October 23, 2026 16:11

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GroupBy.sort and sortBy vague error reporting

2 participants