Skip to content

Add opt-in normalized value counts - #2169

Draft
Kanish2002 wants to merge 2 commits into
Kotlin:masterfrom
Kanish2002:feat/normalized-value-counts
Draft

Kanish2002 wants to merge 2 commits into
Kotlin:masterfrom
Kanish2002:feat/normalized-value-counts

Conversation

@Kanish2002

Copy link
Copy Markdown
Contributor

Reports often need proportions rather than absolute counts. This adds required normalize overloads to the existing DataColumn and DataFrame selector/string operation families.

For ["a", "a", null, "b"], valueCounts(normalize = true) excludes the missing observation and returns counts [2/3, 1/3] as nonnullable Double. With dropNA = false, the denominator includes it and the result is [0.5, 0.25, 0.25].

Behavior and compatibility:

  • Reuse existing counting, sort/ascending/dropNA rules and collision-safe result names. The default name remains count.
  • Divide by the sum of retained counts. Empty/all-excluded input produces zero rows with a declared Double count column.
  • Preserve every old signature and default. New overloads require normalize; false retains Int counts.
  • Return DataFrame<*> for the new overloads because the existing ValueCount.count: Int schema cannot describe conditional Double output.
  • Per dataframe-compiler-plugin-core/AGENTS.md, new overloads omit both refinement and interpreter annotations until supported Kotlin plugin versions understand the parameter. Name-based access works; generated result accessors are not implemented.
  • Existing updateGroups composition supports normalization per group.

Related to #2165. Depends on #2168 for empty observed-column preservation. The branch contains that prerequisite commit followed by the normalization commit; it will need rebasing after the fix lands.

Validation:

  • 55 isolated JUnit tests pass: 30 existing/empty-result value-count tests, 22 normalization tests (including 200 seeded nullable-input/policy cases), and three documentation examples.
  • Compiles actual changed production sources plus unchanged current annotations/Plugin.kt with Kotlin 2.4.21 against published dataframe-core 1.0.0-rc01, using friend paths. Production targets JVM 8; tests target JVM 11. This is not a full current-repository build.
  • Three exact sample tests also compile and pass with the actual DataFrame compiler plugin 2.4.21. A new-overload client compiles/runs with and without it; a stale Int count accessor is rejected.
  • Legacy calls and bound callable references compile against both baseline/current classes. A client compiled against baseline runs with the changed classes. All 11 existing public ValueCountsKt JVM method descriptors remain; six overload/default methods are added.
  • Standalone ktlint 1.8.0, git diff --check, and all three website sample references pass.

Draft for API-design review under the pre-1.0 freeze. Full debug build, repository tests, generated API dump/check, KoDEx and Korro rendering remain pending. The sample references are registered, but rendered snippet generation is not validated.

AI-assisted.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant