You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 05bddf2
Browse filesBrowse the repository at this point in the historyBrowse files
Docs: T1.11b as built, E36 to E39, register P1B-2 proposed
The T1.11b row's as-built notes and the tracker; contract version 6
(E36), the test host's ICU loaded explicitly (E37), describe_source's
thin scope and the schema bridge (E38), each provider's cursor per
request with the zone parity check and the harness changes (E39), all
for the owner to confirm; P1B-2, the thin route's refusals of inputs the
reference accepts, proposed and waiting for the owner.
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Copy file name to clipboardExpand all lines: docs/design/rust-core/phase-1/decisions.md
+5-1Lines changed: 5 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ Decisions made for Phase 1, newest last.
4
4
Rows marked "orchestrator, owner to confirm" were made by the orchestrator; implementation proceeds on them, and the owner confirms or changes them at the check-in that lists them (design, "Owner check-ins"; decision H12); only tasks depending on a listed row wait, and defect fixes never wait.
5
5
Rows marked "approved by owner 2026-10-08" record the owner's written answers to re-review rounds 1 and 2, the T1.7b check-in and checkpoint 1, and "approved by owner 2026-10-09" the owner's answers of that day (E27 to E31); no other row is owner-approved unless its status says so.
6
6
Only these wait for the owner: the 1b go or no-go, any new refusal of an input Python accepts (P1B-1), relaxing a comparison rule, adding an exclusion, downgrading a field to unavailable or adding one as unavailable, a new acceptance capability, an addition to `emulation-allowlist.txt`, the Q2 narrowing fallback, a same-way counter reaching 2 (a check failing twice the same way), and two fix rounds on the same product divergence (migration-plan.md section 9). A review's fix-round counter is reported to the owner but not waited on (E29).
7
-
Rows marked "revised R1" or "revised R2" were changed after that review round; rows C1 to C9 come from round 1, D1 from round 2, E1 to E15 from implementation, E16 and E17 from the T1.7b check-in, E18 and E19 from the owner at checkpoint 1, E20 to E25 from the checkpoint 1 review (T1.8b), E26 from T1.9's code review, E27 from the owner on 2026-10-09, E28 to E31 from the owner at checkpoint 2 (2026-10-09), E32 from the checkpoint 2 corrections (T1.10d), E33 to E35 from T1.11a, H1 to H26 and F1 to F7 from re-review round 1, H27 from the batch 2 code review. Re-review round 2 and its closure checks added no row: their corrections revised rows in place ("re-review R2") and the design text, and superseded rows are marked, not deleted.
7
+
Rows marked "revised R1" or "revised R2" were changed after that review round; rows C1 to C9 come from round 1, D1 from round 2, E1 to E15 from implementation, E16 and E17 from the T1.7b check-in, E18 and E19 from the owner at checkpoint 1, E20 to E25 from the checkpoint 1 review (T1.8b), E26 from T1.9's code review, E27 from the owner on 2026-10-09, E28 to E31 from the owner at checkpoint 2 (2026-10-09), E32 from the checkpoint 2 corrections (T1.10d), E33 to E35 from T1.11a, E36 to E39 from T1.11b, H1 to H26 and F1 to F7 from re-review round 1, H27 from the batch 2 code review. Re-review round 2 and its closure checks added no row: their corrections revised rows in place ("re-review R2") and the design text, and superseded rows are marked, not deleted.
8
8
The design is updated in the same change as a row that changes it.
9
9
Standing decisions are in [migration-plan.md](../migration-plan.md) section 2 and [phase-0/decisions.md](../phase-0/decisions.md).
10
10
@@ -73,6 +73,10 @@ Standing decisions are in [migration-plan.md](../migration-plan.md) section 2 an
73
73
| E33 | The data port and vocabulary as built at T1.11a, where the design's text could not be built as written or would break H9's statement count. (a) `DataLayer` has two methods: `capabilities(provider)` and `plan(request, provider)`, both over the run's `ProviderChoice`, in place of H14's `plan(request, capabilities)`: a host provider's choice carries the capabilities it declared, and for a SQL dialect the layer fills them itself (as T1.22 and T1.25 already require); capabilities alone cannot tell the layer which dialect to render. The core checks a request's own consistency (`DataRequest::check`, `INVALID_REQUEST` naming the request and field) and refuses an undeclared optional request by name before the layer plans it. (b) `Describe` reads one `part` of the catalog (`tables`, `columns`, `keys`, `foreign_keys`, `indexes`): the reference reads its catalog in one statement per part and table, and a request answered with one attachment cannot hold five result shapes, so a single `Describe` could not meet H9's statement count. (c) `KeyCounts` takes a list of `(table, key columns)` and returns one row per key (rows, distinct values among rows holding the whole key, rows missing part of it), the reference's single `UNION ALL`, as H15 reshaped `TimeBounds`; the old per-value counts matched no reference statement. `TimeBounds` keeps no filters, which no reference statement has. (d) A fetch step carries its `FetchPlan` (`sql` with statements and `expect`, or `host` with `expect`); a key-set parameter names the `relation` its statement reads it by. (e) Neighbors' `key` and Lookup's `key` are one column, as the reference's join is; the role-named seed columns are `key` and `batch`, and result helper columns keep the reference's names (`__kumo_batch__`, `__kumo_matches__`). (f) `Zones::zone` answers `UnknownZone` with no name; the core's error for it quotes none (H17). With (b) and (c), each of the 16 statements the capture records on `base.regression` maps to one request (design, T1.11a as built), so no register entry is needed before T1.11b | H14's signature had no way to name a dialect; H9 gates the statement count and each statement's result against the reference, which the round-1 `Describe` and `KeyCounts` could not meet | implementer (T1.11a), owner to confirm at the Journey check-in with H14 and H15's shapes |
74
74
| E34 | Contract version 5 (T1.11a): the data vocabulary changes (H15, E33), the fetch step's plan, the key-set parameter's relation, and the two new optional requests change the wire, so the working entry is rewritten under H4: `sdm-contract-4.json` becomes `sdm-contract-5.json` and the lock holds that one entry; host and core refuse any other version before execution (H25) | A shape change bumps the version, as E26 did for version 4 | implementer (T1.11a), owner to confirm at the Journey check-in |
75
75
| E35 | Proposed, a comparison exclusion: capture conservation (`contract/runners/capture_check.py`, `RANDOM_PATHS`) leaves out the value of `metadata.request_id` in each prediction payload, which the client draws at random for every request, so two runs of the same case never agree on it. Value only: the field must be present in both runs or absent in both, with a value of the same JSON type, and a run that deletes or retypes it fails (`contract/tests/test_capture.py`); every other payload field is compared parsed with numbers as text. Until the owner approves it, T1.11a's capture parity is stated as holding under this one proposed exclusion; T1.11e's `compare-policy.yaml` carries it as the policy entry if approved | Adding an exclusion waits for the owner (this file's preamble); T1.11a's review held the earlier form, which dropped the field and so also hid a missing request id, to be unapproved and too broad | proposed by the implementer (T1.11a review round 1); owner to approve at T1.11e, not approved |
76
+
| E36 | Contract version 6 (T1.11b): the answers whose shape the contract fixes get fixed expected schemas, for a rendering and a host provider alike: each `describe` part, `key_counts` and `time_bounds`, in the reference's own column names and types (`sdm_contract::data::results`, `DataRequest::expected`; a list's item is DuckDB's `l`), so check (b) (H16) holds every provider to one shape; a request whose columns are the source's own (`preview`, `scan`, `sample_entities`, `neighbors`, `lookup`) and `count_rows`, which nothing renders yet, still discover theirs. No shape changes on the wire; what a host provider's answer must be does, which H25 counts as a change of validation, so the working entry is rewritten under H4: `sdm-contract-5.json` becomes `sdm-contract-6.json` and the lock holds that one entry. The test-only fetch (not in the schema) may name a provider in place of a plan, so the root's data layer plans it | T1.11a left every expected schema a discovery marker, so a host provider could answer any shape, which its review noted; the shapes are the reference statements' results, which H9 gates on | implementer (T1.11b), owner to confirm at the Journey check-in |
77
+
| E37 | The test host's ICU is loaded explicitly, in place of `duckdb`'s `icu` feature, which `dependencies.md` planned: at `=1.10506.0` the feature needs `bundled-cmake`, which needs DuckDB's sources beside the crate (`duckdb-sources/CMakeLists.txt`), and the published crate carries neither them nor ICU's, so the build script panics. `sdm-testkit`'s `duck::open` turns autoloading and autoinstalling off, then runs `LOAD icu`, and, the first time on a machine, `INSTALL icu` from DuckDB's repository, the extension DuckDB signs for this exact version (signature checking stays on); it checks the loaded ICU's version is v1.5.6 and the linked DuckDB's too. The Python wheel of DuckDB links ICU statically, so both hosts then read `TIMESTAMPTZ` alike. The cost: `rust-test` and a developer's first run download one signed file (about 15 MB) | The design's mechanism cannot be built at the pinned version; a git-sourced `duckdb-rs` with its submodules would change the source rules `cargo-deny` holds and the DuckDB build, and a prebuilt `libduckdb` would change the linking; an explicit, version-checked load keeps the intent (no extension loads implicitly) | implementer (T1.11b), owner to confirm |
78
+
| E38 | `describe_source` is thin at T1.11b, by the reference's rules where it ports them: (a) each column's data type comes from the Arrow type its preview returned (bool, integers and decimals of scale 0 as `int`, floats and other decimals as `float`, `utf8` as `string`, dates and timestamps as `date`, `time64` as `time`), not from the catalog's spelling of its type, which is the dialect's and so `sdm-data`'s; for those types the two agree, and any other Arrow type is refused; (b) every input whose description takes a rule not ported is refused with `UNSUPPORTED_REQUEST` naming the rule (register entry P1B-2, proposed for the owner), never guessed: text that may hold dates (T1.25, DT-7) or several categories, a key tie or a key whose score depends on the reference's set order, keys that repeat or miss values, a keyless table the declared foreign keys would key, a time column other than the one plain timestamp, a preview of 1,000 rows (the reference then reads the whole table), an empty table, and the overrides (tables named, keys, time columns); (c) the schema bridge builds today's `Schema` with the client's own assembly after the description (relationships inferred by name over the declared links, edit stamps settled, key columns marked, the same warnings), as `Schema.from_source` does after the engine's reading; relationships passed in are refused there; the old materializer reads the source itself afterwards, from an empty `EngineTables`; (d) one data request per reference statement, in its order; `key_counts` counts an enforced unique key with `COUNT(DISTINCT …)` where the reference writes `COUNT(*) - nulls`, the same value, and numbers its keys among those it counts where the reference numbers them among every keyed table, which differ only when a key the source enforces comes before one it does not (H9 gates on results: T1.23 settles it with the comparator's rules) | The design gives the inference to T1.23 and the bridge's full form to T1.24; T1.11b's exit is that `base.regression`'s schema completes, and the owner's rule is that refusing a reference-accepted input waits for the owner, so the thin route says what it cannot do rather than answer differently | implementer (T1.11b), owner to confirm; P1B-2 for the owner |
79
+
| E39 | The DuckDB provider reads each request on its own cursor on both hosts (Python `connection.cursor()`, Rust `Connection::try_clone`), as the reference's reads do: a cursor takes the database's settings and none of the source connection's session. So the `TIMESTAMPTZ` parity check of T1.11b's exit is held under `SET GLOBAL TimeZone='America/New_York'`, where both hosts read New York's wall clock (`contract/provider-vectors.json`, the same rows in each host's test), and a session `SET TimeZone='America/New_York'` on the source's own connection is shown to be seen by no read, on either host, as in the reference. Harness changes with it: the observation's `core` records every fetch with its plan (the step trace) and `describe_source`'s description, which the comparator holds equal between `python-core` and `rust-host`; the case format names `describe_source`'s inputs; the runner's schema projection reads the schema's `relationships`, where it read a `links` attribute the schema never had, so every projection before recorded none | The exit check's session `SET` reaches no read through a cursor, in the reference or either host; read literally it would compare each host's local zone, not New York's | implementer (T1.11b), owner to confirm |
76
80
| H1 | Runtime cells: CI cells are named by runtime and pinned from `phase-0/support-matrix.md`, as local semantic cells, not Databricks runtime certification: `ref-pandas3` (the recorded reference environment) compared with the reference on all fields; `dbr17-pandas2` (the observed DBR 17.3 LTS set) where `python-core` equals `python-old`; `floor-pandas2.2` (Python 3.10, pandas 2.2.0, pyarrow 14.0.1, NumPy 1.26.4, the E16 floor), compared like `dbr17-pandas2`; revised by E16 from A8's S0-only pandas 1.5 floor. A runtime-only difference is a register entry naming the cell, never an exclusion; observation identity records platform and cell. Re-review R2 (owner answer 5): the reference stays recorded on macOS and reference comparisons run locally through `gate.sh`; CI gates platform-independent fields; a Linux reference is a later CI build | Locked check 4 never said which pandas it compares on; the first attempt ended with 165 pandas-2 exemptions | approved by owner 2026-10-08 (round 1 answer 1, round 2 answer 5); the cell versions are the orchestrator's reading of support-matrix.md |
77
81
| H2 | Gate integrity: the checkpoint is the last reviewed landing, with no tag, serialized promotion or separate accept commit (re-review R2): `CHECKPOINT_SHA = git merge-base CANDIDATE_SHA BASE_SHA`, with `BASE_SHA` the phase-branch tip resolved at package start and both SHAs recorded and passed to every gate run, never recomputed from a ref that contains the candidate (R2 closure check); each gate run checks it out into its own worktree, overlays only its observation projection and capture (`run_python.py`, the test kit's case loader and observation writer; the pure-Rust host, providers and scripted endpoint are protected candidate code, and `KNOWN_DIFFERING` is a candidate data file accepted only as a subset of the checkpoint's), and runs its harness, comparator, ratchet, L0 scripts and expectations against the candidate's observations; `contract/protected-paths.txt` is read at the checkpoint, and a protected change passes only when the reviewer's `Reviewed-by:` trailer names it, an outcome change only with a reviewer-approved register entry, a relaxation only with an owner-approved row; before T1.14 lands the candidate's harness runs under review, with no acceptance claim; `contract/availability.yaml` fixes availability per runner and field, and "unavailable" cannot pass a locked check; the owner's local merge is the only hard approval; no signing machinery | Re-review found the gate judged by code and expectation files the candidate can edit (learnings lesson 9), and round 2 found the tag procedure could not bootstrap and let the executor authorize its own protected changes | approved by owner 2026-10-08 (round 1 answer 2, round 2 answer 1); the bootstrap, the authorization rule and the narrowed overlay approved at the T1.7b check-in |
78
82
| H3 | A change is one reviewed work package: its proposed baselines are reviewed with it, one final gate run judges it against the checkpoint, it lands only with evidence for its exact SHA (one rerun after review fixes), and landing is the promotion; `rust-lint` clippy skips `sdm-testkit`, so only `rust-test` builds DuckDB | A CI round trip per commit and per baseline-only commit repeats the first attempt's validation overhead; the cold Rust jobs are dominated by the bundled DuckDB build | approved by owner 2026-10-08 (T1.7b check-in); revised re-review R2 |
|`arrow-*` (`arrow-schema`, `arrow-buffer`, `arrow-array`, `arrow-data`, `arrow-ipc`, `arrow-select`) | 58 | contract, host, later domain crates; `sdm-py` (`arrow-array` and `arrow-schema`, with `ffi`) | Use a library | The closed type set is Arrow's (plan section 3); one major across the workspace, because `duckdb =1.10506.0` and `arrow-pyarrow` 58 both need 58 and two majors in one process need FFI glue between them (decision B-D2); `cargo-deny` refuses a second version of any `arrow-*` crate |
10
+
|`arrow-*` (`arrow-schema`, `arrow-buffer`, `arrow-array`, `arrow-data`, `arrow-ipc`, `arrow-select`) | 58 | contract, host, `sdm-core`, `sdm-schema` (from T1.11b), later domain crates; `sdm-py` (`arrow-array` and `arrow-schema`, with `ffi`) | Use a library | The closed type set is Arrow's (plan section 3); one major across the workspace, because `duckdb =1.10506.0` and `arrow-pyarrow` 58 both need 58 and two majors in one process need FFI glue between them (decision B-D2); `cargo-deny` refuses a second version of any `arrow-*` crate |
11
11
|`pyo3`| 0.28 (`abi3-py310`) |`sdm-py` only | Use a library | The binding tool the plan names; 0.28 because `arrow-pyarrow` 58 needs it (B-D2); RUSTSEC-2026-0176 and RUSTSEC-2026-0177 are ignored with reasons until the next Arrow bump brings 0.29 (decision E1) |
12
12
|`arrow-pyarrow`| 58 |`sdm-py` only | Use a library | PyCapsule only, which the `pyarrow>=14.0.1` floor guarantees (E16), without our own FFI code; as built at T1.8 it reads Arrow in, after the binding checks for `__arrow_c_stream__`, and Arrow out is the binding's own `ArrowStream` over `arrow-array`'s safe FFI types, since `arrow-pyarrow` exports through pyarrow's private `_import_from_c`|
13
-
|`duckdb` (duckdb-rs, `bundled`; `icu` planned at T1.11b) |`=1.10506.0` (DuckDB 1.5.6) |`sdm-testkit` only | Use a library | The pure-Rust test host runs the core's DuckDB SQL at the reference's DuckDB version; `cargo-deny` refuses it anywhere else; `icu`, with extension autoload off, lets the host set a session `TimeZone` so `CAST(TIMESTAMPTZ AS TIMESTAMP)` matches the Python host (`time.timestamptz`) |
13
+
|`duckdb` (duckdb-rs, `bundled`) |`=1.10506.0` (DuckDB 1.5.6) |`sdm-testkit` only | Use a library | The pure-Rust test host runs the core's DuckDB SQL at the reference's DuckDB version; `cargo-deny` refuses it anywhere else. ICU, which the Python wheel links, lets a host set a `TimeZone` so `CAST(TIMESTAMPTZ AS TIMESTAMP)` matches the Python host (`time.timestamptz`): the `icu` feature planned for T1.11b cannot build from the published crate (it needs DuckDB's sources beside it), so the test host loads DuckDB's signed ICU extension for this version explicitly, autoloading and autoinstalling off, and checks its version (decision E37) |
14
14
|`jiff` (`std`, `tzdb-zoneinfo`, `tzdb-bundle-always`; no `tz-system`) | 0.2.38 | host, `sdm-engine` (the `Zones` port, T1.11a), `sdm-core` (resolving through it), the PQL parser | Use a library | The one calendar and zone library (decision A7): it meets DT-1 and DT-2 on all 2,808 zone answers and all 208 `DateOffset` answers of the T0.6 corpus, with a host directory and with its bundled copy; `tz-system` stays off so nothing reads the environment; from T1.10c `rust/clippy.toml` bans the global database (`jiff::tz::db`, `TimeZone::get`, the `in_tz` methods, `TimeZone::system`, the parsers that look zones up in it: `DateTimeParser::parse_zoned`, `Zoned::strptime`, `BrokenDownTime::to_zoned`) and the clocks, and `l0_scan.py` refuses a `Zoned` parsed through `str::parse` or `FromStr`, and `TimeZoneDatabase::from_dir` everywhere but `sdm-host`'s zone module, which reads the host's lookup roots (E24) |
15
15
|`jiff-tzdb`| 0.1.9 (tzdata 2026e) | host | Use a library | The bundled zone database behind `jiff`, the DT-2 fallback; its version (`jiff_tzdb::VERSION`) is recorded as the bundled source's version, and the zone vectors are checked against it |
16
16
|`regex`| 1 |`sdm-host` tests only | Use a library | Repeats the T0.6 probe's reading of the accepted shapes in the zone and calendar vectors |
0 commit comments