You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f6884fb
Browse filesBrowse the repository at this point in the historyBrowse files
T1.35 fix 3: a held-out ID matched as a plain substring
The subagent's round-3 P2: a held-out ID followed by '.' (sentence-final,
inside a token or a path) was shown. Held-out IDs are now matched as plain
substrings wherever they are matched, in compare's lines, the ratchet's
notes and the register's entries: the orchestrator's fail-safe rule, which
may hide too much and never too little. Seeded by the test author at
58cf7b1; the 9 pending tests pass and leave t135-pending.json.
Codex's P3: as-built's T1.35 fix round 2 bullet, filed under T1.20's fix
history by mistake, moved after T1.35's fix round 1 bullet, unchanged.
The tracker's T1.35 notes record fix round 3.
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Copy file name to clipboardExpand all lines: contract/tests/t135-pending.json
+1-11Lines changed: 1 addition & 11 deletions
Original file line number
Diff line number
Diff line change
@@ -1,17 +1,7 @@
1
1
{
2
2
"about": "T1.35 part 1's acceptance tests (the evidence-pack generator, scripts/rust_core/evidence.py) that failed when the independent test author wrote them, before the implementation, at 8af842b2 (tests/test_t135_evidence.py, its interface in that file's docstring). 'exits' holds the Python test ids, 'py:' and the test's name with its parameters. Each listed test must still fail (strict); the implementer takes an id out of 'pending' once it passes, and changes nothing else: 'pending' stays within 'authored', which is the test author's. The test author added the review's tests (orchestrator's decisions on the subagent review of 39cccb09: a core INTERNAL_ERROR a regression even as the reference, python-old's held only by old_client_crash; the compare output as gate.sh prints it, its one verdict adding up; a gating difference never a correct outcome; every observation of the gate's commit and the cases given; held-out file names caught, unruled lines counted; seams per core runner, a panic named; every per-cell run required) on 39cccb09, failing when written. The test author added the second review's tests (Codex's on 39cccb09, orchestrator's decisions: no held-out case reaches the pack through a register entry; no bytecode written outside --out, run with bytecode writing on) on 89d506d2, failing when written. The test author added the third review's tests (orchestrator's rule: a held-out ID matched as a plain substring, so a sentence-final one, one before any punctuation and one inside a longer token are withheld and counted, in a register entry, a compare.log line and a ratchet.log line) on f6fd89ef, failing when written.",
Copy file name to clipboardExpand all lines: docs/design/rust-core/phase-1/as-built.md
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -476,7 +476,6 @@ The Phase 1a review's evidence (design, the T1.20 row; H11, H12, H13 and H26), e
476
476
477
477
T1.1, T1.5, T1.6a, T1.7, T1.8b, T1.12b, T1.25b and T1.28w activate no check of their own (their checks are in the rows above or in later ones). No activated check is disabled: the skip, ignore, `xfail` and `continue-on-error` ban (`l0_scan.py`) holds in `rust-lint`.
478
478
-**Fix round 1** (Codex's Phase 1a review; E80), each with its mechanism:
479
-
-**Fix round 2** (the subagent approved 89d506d2; Codex's review of 39cccb09 raised two new P2s): a register entry naming a held-out case, by its case lists or its free text, carried that case's ID and text into `comparator.md`; the harness modules' bytecode was written beside them, outside `--out`. Both fixed as above, on the test author's 7d206b99, with the subagent's round-2 P3s: a gating difference naming no case refused, the failing step's header read, a dirty checkout's observation refused.
480
479
- A date literal the host cannot hold: `HostFacts.date_literal_range` (`sdm_contract::facts::InstantRange`, `min_ns` and `max_ns`, serde-optional, contract version 21 (20 being T1.28b's)); `sdm_core::pql` hands it to `sdm-pql`'s `parse_held` as `dates::Held`, and the visitor refuses a `DATE` literal whose reading lies outside it where its node is built, as `BuildError::Literal`, so `PQL_VALUE_INVALID_LITERAL` with no span and the reference's message, and nothing after it runs (resolution, capabilities). The Python host's `_rust/facts.py` declares `Timestamp.min.value` to `Timestamp.max.value` when `pandas.to_datetime('0001-01-01')` raises `OutOfBoundsDatetime` (pandas 2) and nothing on pandas 3; `sdm-testkit`'s host declares nothing. Held by the test author's `sdm-core` `tests/pql_literal_range.rs` (the pandas-2 boundary table in a filter and a target, to the nanosecond at both ends, an array element, the literal winning over resolution and capability refusals, node order between two bad literals, none declared accepting all), `sdm-contract` `tests/host_facts.rs` (the field's round trip, absent when none, unknown fields refused), the client's `test_the_date_literal_range.py` (both routes alike on 14 boundary literals, an array and an unknown column, `INVALID_REQUEST` outside and accepted inside on pandas 2, all accepted on pandas 3; it runs in CI's `test` job on every interpreter, pandas 2 on Python 3.10, and in `engine`), `test_the_rust_host.py` (the fact exactly where pandas bounds it), and `test_the_routing_seam.py`, whose year 0 and 1 rows now expect `INVALID_REQUEST` on both routes on pandas 2 in place of the core's `INTERNAL_ERROR`. Measured on pandas 2.2.3 and 3.0.6: every boundary literal, as a filter and in an array, answers the same on both routes.
481
480
- Landing evidence: `land.py`'s `runs_on` keeps only `gating` runs, a push to the phase branch or a topic branch under it (an archive branch never), so `check_candidate_ci`, the landing-run wait and `verify` read only runs in which CI ran the checkpoint's gate. Held by the test author's 14 seeds in `land_selftest.py` (105 self-tests became 119) (the combined scenario, a green pull-request run as the candidate's only one, refused; a green pull-request run newer than a failed push run; archive and `main` runs; the landing-run wait and `verify` given pull-request or archive runs).
482
481
- The owner's approvals of 2026-10-12, relayed by the orchestrator, recorded here: E77(e) in E77's status and register PQL-3 approved by the owner.
@@ -644,10 +643,12 @@ Part 1: the generator of the evidence pack's items 2 and 3 and part of item 4 (d
644
643
-`approved improvement` where every outcome difference is excused;
645
644
-`regression` otherwise. That includes a runner or the reference not observing the case; an `INTERNAL_ERROR` from python-core or rust-host, a panic, even where the reference's outcome is the same; python-old's `INTERNAL_ERROR` unless `ratchet.py`'s `old_client_crash` holds it as the reference's own crash (E110), under the policy's `ordered`; and any gating `DIFFERS <rule>: <case>:` line, with no excuse path, since the gate compares against expectations the approved entries have already moved. Diagnostic and reported lines class nothing.
646
645
-**`seams.md`, per hard seam.** The stages read from a core runner's `core`: `driver Arrow` (`fetches` not empty), `admission` (a description, request state or prediction past fetched data), `inferred schema` (`described`), `payload` (`materialized`), `result` (`predicted.result`). Each case's deepest over every core runner, and a column per core runner and cell, so a runner stopping earlier shows. Before any stage, an `INTERNAL_ERROR` reads `panic` and another refusal `refusal`. Per seam (every seam the case schema names), the stages none of its cases reached on any core runner are listed as unproven, and beside them those rust-host alone never reached.
647
-
-**`comparator.md`.** The compare step's lines counted by rule and kind (gating, diagnostic rule, reported); a line whose first word is no rule `compare.py` runs counts under `no rule`. Then the register's entries by approver, the owner's first, with the proposed ones listed; then the comparator's and the ratchet's lines as they printed them. `compare.py` prints a field's expected and observed values, so these lines do carry values. A line naming a held-out case, by its ID or its file name (`h11.X.json`), is counted, never shown, and the page says how many were left out ("not shown: N"). A register entry naming a held-out case anywhere, in `cases`, `case_outcomes` or its free text (an `h11.X.json` name included), is left out of every listing and only counted; its ID alone may stand in `cases.md` where it excuses a listed case's difference.
646
+
-**`comparator.md`.** The compare step's lines counted by rule and kind (gating, diagnostic rule, reported); a line whose first word is no rule `compare.py` runs counts under `no rule`. Then the register's entries by approver, the owner's first, with the proposed ones listed; then the comparator's and the ratchet's lines as they printed them. `compare.py` prints a field's expected and observed values, so these lines do carry values. A line naming a held-out case, its ID anywhere in the line as a plain substring (so `h11.X.json`, a path or a sentence-final ID are caught; it may hide too much, never too little), is counted, never shown, and the page says how many were left out ("not shown: N"). A register entry naming a held-out case anywhere, in `cases`, `case_outcomes` or its free text (an `h11.X.json` name included), is left out of every listing and only counted; its ID alone may stand in `cases.md` where it excuses a listed case's difference.
648
647
-**The gate JSON and the held-out bundle.** Each file's header names the gate JSON by its file name, the commit it gated and its SHA-256; the JSON is never copied (design.md:432), and one inside `--out` is refused. `heldout.md` holds the bundle's case count and its BUNDLE and INPUTS digests by as-built's T1.20 recipes, the same as `tail-prep/t1.35/heldout.py`'s; on the real bundle both match T1.20's committed digests.
649
648
-**Where it writes.** Only `--out`, no bytecode included (`sys.dont_write_bytecode` is set before any harness module loads, so neither a run that writes nor one that fails leaves a `__pycache__`), never through a link: a link where a pack file goes, or `--out` itself a link, is refused before any write, and each file is opened with `O_NOFOLLOW`. The pack is the same bytes for the same inputs, whatever the observation files are called or the order of their lines: the only input it names is the gate JSON, by file name and digest.
650
649
-**Fix round 1** (the subagent's review of 39cccb09, CHANGES REQUIRED; Codex's pending): the core panic classed as an expected refusal (P1); a compare output read empty, truncated or prefixed as "no differences"; a gating value difference not classing the case; observations not tied to the gate's run; held-out file names shown and lines of no rule dropped uncounted; seams hiding rust-host and panics; per-cell runs not required, and a cell-less Python line matched as `ref-pandas3`; the docs overclaiming that the comparator's lines carry no values and that item 4 is complete. Each was fixed as above, on the test author's c1bb6a09.
650
+
-**Fix round 2** (the subagent approved 89d506d2; Codex's review of 39cccb09 raised two new P2s): a register entry naming a held-out case, by its case lists or its free text, carried that case's ID and text into `comparator.md`; the harness modules' bytecode was written beside them, outside `--out`. Both fixed as above, on the test author's 7d206b99, with the subagent's round-2 P3s: a gating difference naming no case refused, the failing step's header read, a dirty checkout's observation refused.
651
+
-**Fix round 3** (Codex approved f6fd89ef with one P3; the subagent's round 3, one P2): a held-out ID followed by `.` (sentence-final, or inside a longer token or path) was shown. Held-out IDs are now matched as plain substrings wherever they are matched (log lines, ratchet notes, register entries), the orchestrator's fail-safe rule, on the test author's 58cf7b1d; and this section's fix round 2 bullet, filed under T1.20 by mistake, moved here (Codex's P3).
Copy file name to clipboardExpand all lines: docs/design/rust-core/phase-1/tracker.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -291,6 +291,8 @@ None yet.
291
291
292
292
**T1.35, part 1, fix round 2** (the subagent APPROVED 89d506d2 with three P3s; Codex's review of 39cccb09: its P1 already fixed by fix round 1, two new P2s; the orchestrator's decisions, the P2s seeded by the test author at 7d206b99). Fixed: with `--heldout`, a register entry naming a held-out case in its `cases`, `case_outcomes` or any free text (an `h11.X.json` name included) is left out of the pack's listings and only counted; bytecode writing is off before any harness module loads, so nothing lands outside `--out` on success or failure; a gating difference naming no case fails the run naming `compare.log`; the compare step's failing header (`compare FAIL (exit 1)`) is read; an observation whose `identity.commit.clean` is not true is refused naming its case. Measured before the push: `tests/test_t135_evidence.py` 71 passed, its 5 pending ids taken out (the list empty again); the smoke run as in fix round 1, with the real held-out bundle (no register entry names a held-out case: 0 withheld; no `h11.` in any pack file), a failing gate's prefixed compare block read (a payload difference classes its case regression), and a no-case gating line and a dirty observation each refused with nothing written; `ratchet.py check --checkpoint 8af842b2`: ok, naming the same three protected paths; `coverage.py --check` (`--checkpoint 8af842b2` and `HEAD`): ok; `l0_scan.py`: ok; the cheap prepush tier: every check passed.
293
293
294
+
**T1.35, part 1, fix round 3** (Codex APPROVED f6fd89ef across the range with one P3; the subagent CHANGES REQUIRED, one P2; the orchestrator's decisions, seeded by the test author at 58cf7b1d). Fixed: a held-out ID followed by `.` (sentence-final, inside a token or a path) leaked; held-out IDs are now matched as plain substrings, which may over-hide and never under-hide; Codex's P3, as-built's T1.35 fix round 2 bullet, filed under T1.20, moved to T1.35's section unchanged. Measured before the push: `tests/test_t135_evidence.py` 92 passed, its 9 pending ids taken out (the list empty again); `ratchet.py check --checkpoint 8af842b2`: ok, naming the same three protected paths; `coverage.py --check` (`--checkpoint 8af842b2` and `HEAD`): ok; `l0_scan.py`: ok; the cheap prepush tier: every check passed.
0 commit comments