Skip to content

fix(layout): restore column detection on table pages and sparse pages - #430

Merged
abimaelmartell merged 3 commits into
mainfrom
feat/mc-weave-class
Aug 19, 2026
Merged

abimaelmartell merged 3 commits into
mainfrom
feat/mc-weave-class

Conversation

@abimaelmartell

@abimaelmartell abimaelmartell commented Aug 18, 2026

Copy link
Copy Markdown
Member

What

Two gates in relative-valley column detection were starving real multi-column pages of their gutters, dropping them to single-column Y-sorting where the columns interleave line by line:

  1. Table pages: the page_has_table guard predates item claiming — by grouping time the detected table's items have already left the flow, so a table cannot fake a gutter in the histogram the guard protects. The prose gate inside relative-valley acceptance is the real defense, and it stays. Without the fallback, the prose remainder of a table-bearing two-column page wove line by line.
  2. Sparse pages: the 100-item floor guarded against shallow histogram dips, but OCR'd multi-column pages arrive as few long line-runs (a two-column French academic page is ~60 items) and never qualified. Lowered to 30; the same prose gate judges the split.

Results

  • Multi-column reading order: 460 → 462 / 884; table structure tests unchanged (446/1022); all other categories unchanged.
  • Regression corpus: all pass. Five documents change, four are clear recoveries — a French two-column bioethics text that was fully woven now reads perfectly, a refrigerant datasheet's intro column untangles from its sidebar, an IRS publication's three-column region largely un-weaves, an annual report's prose recovers around its tables. One decorative magazine-style page that was already woven is now differently (not worse) woven; it has no semantic baseline to arbitrate, disclosed here for the record.
  • Semantic scorer on baselined movers: reading order up (0.10 → 0.18 on the datasheet), no decliners.
  • Full test suite passes; local review clean.

🤖 Generated with Claude Code


Summary by cubic

Restores relative‑valley column detection on table-bearing and sparse pages to prevent interleaved reading order. Previously it required ≥100 items and no detected tables; now it runs with ≥30 items regardless of tables, and the XY‑cut fallback is re‑guarded on table pages.

  • Remove the table guard from the relative‑valley path: table items are out of flow and the prose gate rejects table‑shaped splits.
  • Lower the item floor from 100 to 30 to handle OCR’d multi‑column pages with few long line‑runs.
  • Re‑guard the XY‑cut fallback on table pages because it has no prose gate, preventing unvalidated splits after a rejected valley.
  • Impact: improved multi‑column reading order; table handling unaffected; benchmarks and tests unchanged.

Written for commit dfe64b8. Summary will update on new commits.

Review in cubic

The page_has_table guard predates item claiming: by grouping time the
detected table's items have already left the flow, so a table cannot
fake a gutter in the histogram this guard protects, and the prose gate
inside relative-valley acceptance rejects any residual table-shaped
split. Without the fallback, the prose remainder of a table-bearing
two-column page fell to single-column Y-sorting and its columns
interleaved line by line.
The 100-item floor guarded against shallow histogram dips on sparse
pages, but OCR'd multi-column pages produce few long line-runs (a
two-column French academic page arrives as ~60 items) and were falling
to single-column Y-sorting, weaving their columns line by line. The
prose gate inside relative-valley acceptance is the real defense
against spurious dips; 30 items is enough for it to judge.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file

Shadow auto-approve: would not auto-approve because issues were found.

Fix all with cubic | Re-trigger cubic

Comment thread src/extractor/layout.rs
Comment thread src/extractor/layout.rs
Comment thread src/extractor/layout.rs
Comment thread src/extractor/layout.rs
Removing page_has_table from the relative-valley condition also
unlocked the XY-cut fallback inside that block, which has no prose
gate — a table page whose valley candidate was just rejected could
take an unvalidated split. The guard is restored on that call
specifically; the relative-valley path keeps its prose-gated access.
Also rewrite the stale dense-page comment for the 30-item floor.
Bench and corpus unchanged (462/884, corpus byte-identical).

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

Shadow auto-approve: would require human review. Expands relative-valley column detection to table-bearing and sparse pages (removes table guard, lowers item floor 100→30). This broader heuristic rollout and threshold change require human sign-off; the claimed improvements rely on external benchmarks not visible in the diff.

Re-trigger cubic

@abimaelmartell
abimaelmartell merged commit bac056e into main Aug 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant