Skip to content

Fix null counts exceeding per-file record counts - #8

Open
keemgdeok wants to merge 1 commit into
mainfrom
fix/null-count-record-consistency
Open

keemgdeok wants to merge 1 commit into
mainfrom
fix/null-count-record-consistency

Conversation

@keemgdeok

Copy link
Copy Markdown
Owner

Summary

Treat a top-level column as having inconsistent metrics when a data file reports more nulls than records, independently of whether its value count is present. The affected null-ratio rule now returns UNKNOWN / INCONSISTENT_METRICS; row-count evidence and unrelated columns remain unaffected.

Regression coverage

  • Missing or inflated value counts with null counts exceeding file rows.
  • Positive null counts in a zero-row file.
  • Valid all-null, zero-null, and zero-row boundaries.
  • Per-file contradictions hidden by the total row count across other files.
  • Row-count and unrelated-column results remain valid.

Validation

  • Confirmed four regression inputs failed before the production fix.
  • Java 17 Docker build and formatting checks passed; all 45 tests passed.
  • Repeated the original 10-row / 20-null reproduction against the rebuilt JAR: null ratio returns UNKNOWN / INCONSISTENT_METRICS, while row count returns PASS.
  • Packaged doctor --demo passed with the no-content-file guard.
  • git diff --check passed.

Production change is limited to a three-line consistency check. No API, dependency, release-version, or E2E-environment changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant