Skip to content

fix(crosscheck): move the Lean image to v4.28.0 and name imports that exist in the Lean skills - #145

Merged
hnipps merged 6 commits into
mainfrom
fix/lean-image-v4.28-r2
Nov 7, 2026
Merged

hnipps merged 6 commits into
mainfrom
fix/lean-image-v4.28-r2

Conversation

@hnipps

@hnipps hnipps commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Tier: 3
Intent: intent/2026-10-27-lean-image-v4-28.md
Spec: intent/2026-10-27-lean-image-v4-28-spec.md
Plan: intent/2026-10-27-lean-image-v4-28-plan.md
Task: LT-1.1

What changes

This is for anyone running /lean-spec, /lean-impl, /correspondence-review or /drt-oracle. lean_check, lean_run and lean_test now judge files against Lean v4.28.0 and Mathlib v4.28.0, the toolchain the rest of the repository uses. Before, a core-only file that used List.Pairwise failed with "environment does not contain 'List.Pairwise'". That is the failure behind PR #144.

  • The pins. crosscheck/mcp-server/lean-harness/lean-toolchain and lakefile.lean pin v4.28.0.
  • One import. Mathlib.Data.Nat.Defs does not exist at v4.28.0, so Crosscheck/Program.lean and the timing snippet in crosscheck/scripts/build-lean-docker.sh import Mathlib.Data.Nat.Basic.
  • The skills. /lean-spec, /lean-impl and /drt-oracle now name only imports that compile in the new image. Six of the imports they named fail there, and five of those also fail in the old image.
  • The roadmap. A new item, LT-1, governs this change, and task LT-1.1 records it. The governance note is below.

Every host must rebuild with crosscheck/scripts/build-lean-docker.sh. On the old image, the corrected /lean-spec table names Mathlib.Data.Nat.Basic, which does not exist there.

Imports the skills name (measured 2026-10-28)

Each probe is import <module> followed by #eval 1. Each was run with docker run --rm --network none --memory 2g --cpus 2 -v <dir>:/work <image> check /work/program.lean, which is what lean_check runs.

Import named by a skill v4.10.0 (:latest) v4.28.0 (:v4.28-test) Now
Mathlib.Data.Nat.Defs pass fail Mathlib.Data.Nat.Basic (pass)
Mathlib.Tactic.DeriveDecidableEq fail fail no import, deriving DecidableEq is core (pass)
Mathlib.Analysis.SpecialFunctions.Basic fail fail Mathlib.Data.Real.Basic (pass)
Mathlib.Data.Array.Basic fail fail no import, Array.foldl and Array.mapIdx are core (pass)
Mathlib.Data.HashMap.Basic fail fail Std.Data.HashMap (pass)
Mathlib.Data.Json fail fail Lean.Data.Json (pass)
Mathlib.Data.List.Basic, Finset.Basic, Multiset.Basic, Tactic.Linarith pass pass unchanged

Results (image built as crosscheck-lean:v4.28-test, with :latest untouched)

  • Build (measured). It takes 360 s, and the image is 10.5 GB against 6.34 GB for the old one. The script's own timing check takes 6 s.
  • npm test. 382 passed and 25 skipped, measured with and without LEAN_DOCKER_IMAGE set.
  • RUN_E2E=1 npm run test:e2e. 20 passed. No e2e test uses the Lean image, so the Lean checks below drove docker run the way runLean in docker.ts does.
  • Core-only List.Pairwise probe (measured). The new image passes it in 4 s, and the old image fails it.
  • import Mathlib.Data.List.Basic with List.Pairwise by decide. Passes in 7 s.
  • formal-verification/lean/CrosscheckModel/Power.lean. Passes with 4 sorry warnings in 8 s.
  • A file with a type error. It still fails.
  • A 2-line Mathlib check (measured). 6 s on the new image against 4 s on the old.
  • Peak memory. 516 MiB on the new image against 275 MiB on the old, inside the 2 GB limit. The sample is coarse: docker stats once a second.

Not yet reached: CI does not build the Lean image, so CI does not catch the pins drifting again. The property that blocks it is the hosted-runner time and disk the Mathlib oleans need. LT-1's acceptance records the open question.

Action needed: Resolve markers and checklist before merging
You are being asked to verify this governance-note block as PR reviewer because the edit touches a protected surface. Approving means the protected-surface edit merges as governed and traceable; declining means the author must resolve every REQUIRES HUMAN VERIFICATION: marker and Review Checklist item first. Full explanation: https://github.com/nicholls-inc/crosscheck/blob/main/docs/gates/protected-surface-amendment.md.

Protected-Surface Amendment

Target file(s): docs/assurance/ROADMAP.md (+ 3 others, see Diff Plan)
Class: A (harness and workflow definitions, and governance documents the gates read)
Matched rule: docs/assurance/**, crosscheck/skills/*/SKILL.md
Date: 2026-10-28

Change Description

  1. docs/assurance/ROADMAP.md gains one item, LT-1, in the Next horizon. It covers keeping the Crosscheck Lean image on a Lean and Mathlib release that builds what the pipeline produces. No other item, status or section changes.
  2. Three skills get corrected import tables. In each, a module named in the skill's import guidance is replaced with one that exists in Mathlib v4.28.0 or in core Lean, or with "no import" where core Lean covers the case.
    • /lean-spec: three rows, for natural numbers, decidability and reals, plus the same decidability advice in its build-error table.
    • /lean-impl: two rows, for arrays and hash maps.
    • /drt-oracle: one line, for JSON.
  3. No stage, gate or step of any skill changes. Only the import names change.

Rationale

The trigger is PR #145, which moves the Crosscheck Lean image from Lean and Mathlib v4.10.0 to v4.28.0. The image could not build the OCPP 1.6 model in PR #144, because the model uses List.Pairwise from Lean core, and v4.10.0's core does not have it. With v4.28.0, /lean-spec's instruction to import Mathlib.Data.Nat.Defs fails, because that module no longer exists.

The skill tables were checked on 2026-10-28. Each module was compiled in both images with docker run --rm --network none -v <dir>:/work <image> check /work/program.lean, which is what lean_check runs. Five more imports named in the skills fail on both images:

  • Mathlib.Analysis.SpecialFunctions.Basic
  • Mathlib.Data.Array.Basic
  • Mathlib.Data.HashMap.Basic
  • Mathlib.Data.Json
  • Mathlib.Tactic.DeriveDecidableEq

So a model that follows those rows fails lean_check today, whichever image it uses. Every replacement passes in the v4.28.0 image. The plan lists each result: intent/2026-10-27-lean-image-v4-28-plan.md.

The maintainer asked on 2026-10-28 for everything #145 needs to be prepared for approval.

Governing Roadmap Item

  • Path: docs/assurance/ROADMAP.md (Next horizon, item LT-1, added by this change)
  • Title: Keep the Crosscheck Lean image on a Lean and Mathlib release that builds what the pipeline produces
  • Scope coverage: LT-1 covers the image's pins, and the imports that the skills tell a model to write, so that the code the pipeline produces passes lean_check.

No existing item covered the Lean image. ER-1 and VA-1 are the nearest, and citing either would stretch its scope. LT-1 is added by this change and authorises it. This follows restore-hooks-and-mr-1-2026-09-29.md, which added MR-1 under MR-1.

REQUIRES HUMAN VERIFICATION: The maintainer confirms that LT-1 may authorise its own addition, and the five skill-table corrections that were broken before this change.

Authority

Diff Plan

# File Lines Invariant ID / Stage Action
1 docs/assurance/ROADMAP.md Next horizon table and its item paragraphs LT-1 added
2 crosscheck/skills/lean-spec/SKILL.md 88, 90, 91 (import table); 165 (error table) Step "likely imports"; build-error table replaced: Mathlib.Data.Nat.Defs with Mathlib.Data.Nat.Basic; Mathlib.Tactic.DeriveDecidableEq with no import, in both tables; Mathlib.Analysis.SpecialFunctions.Basic with Mathlib.Data.Real.Basic
3 crosscheck/skills/lean-impl/SKILL.md 106, 107 (import table) Step "likely imports" replaced: Mathlib.Data.Array.Basic with no import; Mathlib.Data.HashMap.Basic with Std.Data.HashMap
4 crosscheck/skills/drt-oracle/SKILL.md 122 Lean runner requirements replaced: Mathlib.Data.Json with Lean.Data.Json

Test / Coverage Impact

  • No Class B invariant or property test changes.
  • node scripts/ci/skill-references.mjs checks the slash references in the skills. The change adds no slash reference.
  • No eval exercises the import tables. The probe compilations in the plan are the evidence, and they can be rerun with the command above.
  • A follow-up that is not yet reached: a CI job that compiles every import a skill names in the pinned image. The property that blocks it is that CI does not build the Lean image (10.5 GB). The open question is whether to build it in CI or to check the module names against Mathlib's index.

Review Checklist

Mechanical items. Tick each one only after running its command, and write the command and its output under the item. If a command cannot run, leave the item unticked and add REQUIRES HUMAN VERIFICATION: and the reason.

  • Authoriser is a named human, not a bot and not an agent. Command: the handle matches none of *-bot, *[bot]*, claude-* or an agent name.
    Output: hnipps matches none of the patterns.
  • Governing roadmap item exists. Command: grep -n "LT-1" docs/assurance/ROADMAP.md finds it.
    Output: lines 100 (Next horizon table), 128 (item paragraph) and 139.
  • Diff plan names every changed protected file. Command: git diff --name-only origin/main...HEAD, filtered by the protected globs, equals the diff plan's file list.
    Output: crosscheck/skills/drt-oracle/SKILL.md, crosscheck/skills/lean-impl/SKILL.md, crosscheck/skills/lean-spec/SKILL.md, docs/assurance/ROADMAP.md. These are the diff plan's four files.
  • This amendment block appears in the PR body. Command: the PR body contains the ## Protected-Surface Amendment heading.
    Output: gh pr view 145 --json body --jq .body | grep -c '^## Protected-Surface Amendment' prints 1.
  • All REQUIRES HUMAN VERIFICATION: markers above have been resolved. Command: the count of unresolved markers is zero.

Judgment items. A human decides these, or an agent ticks them on the maintainer's explicit instruction.

  • Rationale is anchored to a concrete trigger (not "cleanup" or "robustness").
  • Governing roadmap item actually covers this change, and LT-1 has the scope, acceptance and horizon the maintainer wants.
  • Diff plan covers every affected line range and invariant ID / stage.
  • Every added Class B invariant has a covering property test in this PR (or <!-- aspirational --> + linked issue). Not applicable: no Class B file changes.
  • Every removed Class B invariant has its covering test removed or re-pointed in this PR. Not applicable: no Class B file changes.
  • No invariant is being weakened purely to make a failing test pass.
  • Class A edits: downstream eval or prompt-tuning follow-ups are queued.

🤖 Generated with Claude Code

hnipps and others added 5 commits October 27, 2026 19:04
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The image checked files against Lean v4.10.0, whose core lacks List.Pairwise,
so a core-only file that builds with the repository's own toolchain failed
lean_check. Mathlib.Data.Nat.Defs no longer exists at v4.28.0, so the
placeholder and the timing snippet import Mathlib.Data.Nat.Basic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…overnance note for the Lean v4.28.0 image

LT-1 covers keeping the Lean image on a release that builds what the pipeline produces. It governs this change, following the MR-1 precedent, and the maintainer confirms that in the note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pl and /drt-oracle

Mathlib.Data.Nat.Defs is gone in Mathlib v4.28.0, so /lean-spec names Mathlib.Data.Nat.Basic. Five other imports the skills named exist in neither v4.10.0 nor v4.28.0: Analysis.SpecialFunctions.Basic, Data.Array.Basic, Data.HashMap.Basic, Data.Json and Tactic.DeriveDecidableEq. They become Mathlib.Data.Real.Basic, no import for Array, Std.Data.HashMap, Lean.Data.Json and no import for deriving DecidableEq. Each replacement compiles in the v4.28.0 image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nance note

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hnipps hnipps changed the title fix(crosscheck): pin the Lean image to Lean v4.28.0 and Mathlib v4.28.0 fix(crosscheck): move the Lean image to v4.28.0 and name imports that exist in the Lean skills Oct 28, 2026
…ance note

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hnipps
hnipps marked this pull request as ready for review October 28, 2026 13:28
@hnipps
hnipps merged commit c2df9e7 into main Nov 7, 2026
10 checks passed
@hnipps
hnipps deleted the fix/lean-image-v4.28-r2 branch November 7, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant