Skip to content

Commit 4052071

Browse files
committed
CI4 docs: E131, the tracker row and as built after fix round 1
E131 now records the phase-push check in the landing job, evidence followed below empty landing commits, verify's phase-branch check, why a moved landing keeps its CI (sound for a single move, not for a stack), the credentials left out until the move, the corrected timeouts and their measured maxima, and the checkpoint from the landed-on head. The tracker cites the tests commits on the branch, 76d5d24 and 9e9dfe6. Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
1 parent e0ba692 commit 4052071

3 files changed

Lines changed: 12 additions & 10 deletions

File tree

‎docs/design/rust-core/phase-1/as-built.md‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -698,11 +698,13 @@ Landing a candidate moved over landing commits only (E130, the owner's decision
698698

699699
## CI4
700700

701-
CI's images, its landing commits and a run's checkpoint (E131, the owner's items 29 and 30 of 2026-10-14), each claim with its mechanism:
702-
703-
- **No CI on a landing commit.** `ci.yml`'s first job, `landing`, checks out the pushed commit with its parent (`fetch-depth: 2`) and, on a push to `rust-core-phase-1` or a branch under it, writes `skip=true` when `git rev-list --parents -n 1 HEAD` names one parent, the subject starts `Land ` and `HEAD^{tree}` is `HEAD^^{tree}`; otherwise, and for every pull request, `skip=false`. The jobs that need nothing else (`lint`, `hygiene`, `rust-lint`, `rust-deny`, `rust-test`, `native-wheel`, `native-test-ops`) need `landing` and run only `if: needs.landing.outputs.skip != 'true'`; the rest are left out with the jobs they need; `cross-host`, whose `!cancelled()` would run it, needs `landing` and checks the output itself. A landing's run shows the landing job green and every other job skipped. `tests/test_ci_workflow.py` runs the step's text against real repositories (a landing commit on the phase branch, a lane and a topic branch skips; on `main`, `rust-core` or a pull request, a candidate, a landing commit changing a file, an empty commit with another subject, and a merge with its first parent's tree do not) and walks the jobs' `needs` and conditions.
704-
- **`land.py`.** `verify` checks the landing commit is empty with E25's trailers, then takes the newest gating run (`runs_on`: a push to the phase branch or a topic branch under it, on the exact SHA) on the candidate, the landing commit's parent, or, with `Moved-from`, on that commit once its tree is the landing's: green passes, naming the run; red fails with the revert instructions; cancelled or unfinished asks for a rerun (exit 3). It no longer waits for the landing commit's runs to appear (`--appear-timeout` is accepted and ignored); those listed are reported, a cancelled one with the green run of the same tree, and one that ran and failed fails. `--wait-for-landing-run` is refused before anything is read, naming the candidate's run as the evidence; the not-green refusal no longer suggests it; after the push `land.py` lists the landing runs already there, once, rather than waiting up to 180 s. The test author's 14 CI4 cases in `land_selftest.py` hold this (169 self-tests); `lane_selftest.py`'s red-landing case makes its earlier landing commit by hand, the flag that made it being refused.
705-
- **The checkpoint while the phase branch moves.** `ratchet.py resolve` with `--base` (a lane or topic run, the phase branch read live): when `git merge-base` gives a commit that is no landing commit, `landed_since` looks on the ancestry path from it to the base for a landing commit whose `Base-SHA` is a proper ancestor of it; with one, `landed_ancestor` takes the newest landing commit on the candidate's first-parent line that the base holds, with a note. Otherwise, and for `--phase-push` and a landing commit's recorded base, nothing changes. Checked on temporary repositories: a candidate stacked on X1 resolves to H0 before and after X1 lands on its own commits; a candidate on an unlanded commit pushed to the phase branch is refused, as before, also when a later landing's `Base-SHA` is that commit. Until CI4 is a checkpoint, the checkpoint's own `ratchet.py`, which `gate.sh` runs again inside the checkpoint's worktree, still resolves the old way.
706-
- **The images.** `POSTGRES_IMAGE` and `MANYLINUX` in the workflow's `env`, and each wheel-smoke lane's image in `wheel_smoke.py` (`wheel_smoke.py image LANE` prints it), are the one place each image is named, by digest. A service container's `image` cannot read `env`, so `landing` outputs `postgres-image` and one anchored service block serves `test` and `engine`, with `credentials` empty (no login) unless the image is on ghcr.io. `scripts/rust_core/pull_image.sh` runs before each `docker run`: it refuses an image without `@sha256:` and 64 hex digits, logs in to ghcr.io with `GITHUB_TOKEN` when the image is there, and tries `docker pull` up to four times, after 20, 60 and 120 s. `permissions: packages: read` is added. Nothing is published; every image comes from where it did.
707-
- **Timeouts and caches.** Every job has `timeout-minutes`, about twice its slowest green run over 260 phase and lane runs (`rust-test` max 56.7 min: 90; `contract-python (ref-pandas3)` 34.8: 60; `rust-lint` 36.8, `test` 56.7 (p99 22.6), `engine` 25.8: 45; every other job under 8: 20; `landing`: 10). pip's caches are saved only on a push to `main` (`SAVE_PIP_CACHES`), which every branch restores; `SAVE_CACHES` leaves out `rust-core-phase-1`, whose pushes are now landing commits.
708-
- **Dependencies at opt-level 1.** `rust/Cargo.toml`'s `[profile.dev.package."*"]` adds `opt-level = 1` (the test profile inherits it); the workspace's crates stay at 0. Measured on a 10-core laptop, cold, `cargo test -p sdm-testkit` for three DuckDB-backed files: unoptimized, build 2 min 22 s, `t132a_key_cast` 94 s, `t132b_decimal_vectors` 30 s, `duckdb_provider` 30 s, 2.4 GB of target; at opt-level 1 for every dependency, 2 min 14 s, 12 s, 4 s, 6 s, 1.4 GB; for `libduckdb-sys` alone, 2 min 13 s, 71 s, 27 s, 26 s, so the time is mostly Arrow's and the DuckDB crate's unoptimized Rust. Not taken, for the owner: linking the test kit to DuckDB's official prebuilt `libduckdb` 1.5.6 (checked by its published SHA-256) instead of compiling the bundled source: build 38 s and 48 s, the three files 89, 28, 24 s unoptimized and 7, 3, 4 s with opt-level 1; it changes the DuckDB build the Rust tests run against, and needs `duckdb`'s `bundled` feature made an option of the test kit, with the library on the test runs' library path. A Cargo.toml change leaves rust-cache's environment hash as it was, so the first run restores the branch's previous build and rebuilds the dependencies once.
701+
CI's images, its landing commits and a run's checkpoint (E131, the owner's items 29 and 30 of 2026-10-14), each claim with its mechanism, as of fix round 1:
702+
703+
- **No CI on a landing commit.** `ci.yml`'s first job, `landing`, checks out the pushed commit with its parent (`fetch-depth: 2`) and, on a push to `rust-core-phase-1` or a branch under it, writes `skip=true` when `git rev-list --parents -n 1 HEAD` names one parent, the subject starts `Land `, no `Moved-from` trailer is present (`%(trailers:key=Moved-from,valueonly)` is empty) and `HEAD^{tree}` is `HEAD^^{tree}`; otherwise, and for every other event, `skip=false`. The jobs that need nothing else (`lint`, `hygiene`, `rust-lint`, `rust-deny`, `rust-test`, `native-wheel`, `native-test-ops`) need `landing` and run only `if: needs.landing.outputs.skip != 'true'`; the rest are left out with the jobs they need; `cross-host`, whose `!cancelled()` would run it, needs `landing` and checks the output itself. A landing's run shows the landing job green and every other job skipped.
704+
- **The phase branch's check.** On every push to `rust-core-phase-1`, the `landing` job's second step deepens the clone (`git fetch --unshallow`, when shallow) and runs `ratchet.py checkpoint --repo . $(ratchet.py where)`, that is `--phase-push` with the event's previous tip: a fast-forward, exactly one landing commit, the tip, whose `Base-SHA` is the previous tip. A push failing it fails the job, so every other job is left out and the run is red. Tried on a temporary origin and a depth-2 clone: a sound landing passes, a previous tip that is not the landing's `Base-SHA` fails.
705+
- **A moved landing runs.** A landing commit carrying `Moved-from` is not skipped (E131 (c)): its runs judge it against its own `Base-SHA` by the phase push, as before E131, and a failed one fails `verify`.
706+
- **`land.py`.** `skip_shaped` reads a commit as CI does (subject `Land `, one parent, its parent's tree), and `tested_commit` follows such commits down to the first that is not one; `check_candidate_ci` (the candidate's run, and under `--moved-from` X's) and `verify` take the run of that commit, and a refusal names both commits. `verify` checks the landing commit is empty with E25's trailers and is an ancestor of the phase branch's head on the remote (fetched), then takes the newest gating run on that commit below the candidate, or, with `Moved-from`, below that commit once its tree is the landing's: green passes, naming the run; red fails with the revert instructions; cancelled or unfinished asks for a rerun (exit 3). It no longer waits for the landing commit's runs to appear (`--appear-timeout` is accepted and ignored); those listed are reported, a cancelled one with the green run of the same tree, and one that ran and failed fails. `--wait-for-landing-run` is refused before anything is read; the not-green refusal no longer suggests it; after the push `land.py` lists the landing runs already there, once. The test author's 14 CI4 cases in `land_selftest.py` hold this (169 self-tests); `lane_selftest.py`'s red-landing case makes its earlier landing commit by hand. Checked on temporary repositories: a candidate whose tip is an empty `Land` commit on a commit with no run is refused, and passes once that commit's run is green; a landing commit the phase branch does not hold fails `verify`.
707+
- **The checkpoint while the phase branch moves.** `ratchet.py resolve` with `--base` (a lane or topic run, the phase branch read live): when `git merge-base` gives a commit that is no landing commit, `landed_on` looks on the ancestry path from it to the base for a landing commit whose `Base-SHA` R is a proper ancestor of it, and, when R is a landing commit, the checkpoint is `git merge-base CANDIDATE R`, with a note. Otherwise, and for `--phase-push` and a landing commit's recorded base, nothing changes. The test author's five `test_ci4_*` cases in `tests/test_contract_ratchet.py` hold it: a stacked candidate before and after its package lands, and after another lands on top; a landing whose `Base-SHA` is an unlanded phase commit, still refused; a stack landed as moved copies; a landing on the candidate's line the phase branch lacks, passed over; a phase push and a recorded base, as before. Until CI4 is a checkpoint, the checkpoint's own `ratchet.py`, which `gate.sh` runs again inside the checkpoint's worktree, still resolves the old way.
708+
- **The images.** `POSTGRES_IMAGE` and `MANYLINUX` in the workflow's `env`, and each wheel-smoke lane's image in `wheel_smoke.py` (`wheel_smoke.py image LANE` prints it), are the one place each image is named, by digest. A service container's `image` cannot read `env`, so `landing` outputs `postgres-image`, and one anchored service block serves `test` and `engine`, with no `credentials` while the image is not on ghcr.io (an empty credential fails the workflow's template, run 38659551659). `scripts/rust_core/pull_image.sh` runs before each `docker run`: it refuses an image without `@sha256:` and 64 hex digits, logs in to ghcr.io with `GITHUB_TOKEN` when the image is there, and tries `docker pull` up to four times, after 20, 60 and 120 s, except that `Data limit exceeded` fails at once. `permissions: packages: read` is added. Every image comes from where it did until the four are published to ghcr.io and the references swapped.
709+
- **Timeouts and caches.** Every job has `timeout-minutes` above its slowest green run over 260 phase and lane runs: `rust-test` 120 (slowest 80.7 min, run 38640995251; to be lowered once runs with optimized dependencies are measured), `test` and `engine` 60 (56.7, run 38577013261, and 25.8), `contract-python (ref-pandas3)` 60 (34.8), `rust-lint` 45 (36.8), every other job 20 (each under 8), `landing` 10. pip's caches are saved only on a push to `main` (`SAVE_PIP_CACHES`), which every branch restores; `SAVE_CACHES` leaves out `rust-core-phase-1`, whose pushes are now landing commits.
710+
- **Dependencies at opt-level 1.** `rust/Cargo.toml`'s `[profile.dev.package."*"]` adds `opt-level = 1` (the test profile inherits it); the workspace's crates stay at 0, and debug assertions and overflow checks keep the dev defaults. Measured on a 10-core laptop, cold, `cargo test -p sdm-testkit` for three DuckDB-backed files: unoptimized, build 2 min 22 s, `t132a_key_cast` 94 s, `t132b_decimal_vectors` 30 s, `duckdb_provider` 30 s, 2.4 GB of target; at opt-level 1 for every dependency, 2 min 14 s, 12 s, 4 s, 6 s, 1.4 GB; for `libduckdb-sys` alone, 2 min 13 s, 71 s, 27 s, 26 s, so the time is mostly Arrow's and the DuckDB crate's unoptimized Rust. `cargo test --workspace --locked` passes at opt-level 1. A Cargo.toml change leaves rust-cache's environment hash as it was, so the first run restores the branch's previous build and rebuilds the dependencies once. Not taken, for the owner: linking the test kit to DuckDB's official prebuilt `libduckdb` 1.5.6 (checked by its published SHA-256) instead of compiling the bundled source: build 38 s and 48 s, the three files 89, 28, 24 s unoptimized and 7, 3, 4 s with opt-level 1; it changes the DuckDB build the Rust tests run against, and needs `duckdb`'s `bundled` feature made an option of the test kit, with the library on the test runs' library path.

0 commit comments

Comments
 (0)