Skip to content

test(crosscheck): model the OCPP 1.6 composite schedule in Lean and test EVerest against it - #144

Merged
hnipps merged 26 commits into
mainfrom
worktree-ocpp-smart-charging-plan
Nov 7, 2026
Merged

hnipps merged 26 commits into
mainfrom
worktree-ocpp-smart-charging-plan

Conversation

@hnipps

@hnipps hnipps commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Tier: 1
Intent: intent/2026-10-24-ocpp-smart-charging-model.md
Plan: intent/2026-10-24-ocpp-smart-charging-model-plan.md
Spec: formal-verification/specs/ocpp16_composite_informal.md

Summary

This runs the Crosscheck Lean pipeline on an external code base for the first time. It models the OCPP 1.6 composite charging schedule in Lean, proves three of its properties, and tests EVerest's OCPP 1.6 implementation against the model on generated inputs.

It found one real EVerest bug. Recurring daily and weekly charging profiles restart one second early per leap second since the profile's startSchedule. The spec says a daily schedule restarts "at the same time as in the startSchedule". The physical impact is small, 1 to 3 seconds today, but the deviation is clear and reproducible. Apart from that, EVerest matched the model within 0.05 of the unit at every instant where the model gives a limit: 57,304 of 57,304 in the 2017 to 2025 run. The harness prints these counters on every run.

What is in the diff

  • Intent and plan. intent/2026-10-24-ocpp-smart-charging-model.md and -plan.md, steps S1 to S4.
  • Signed informal spec. formal-verification/specs/ocpp16_composite_informal.md, written with /informal-spec from OCPP 1.6 edition 2, sections 3.13 and 7.8 to 7.14. It records ten resolved ambiguities and four wording notes for the Open Charge Alliance.
  • Lean model and proofs. formal-verification/lean/CrosscheckModel/Ocpp16Composite.lean, written with /lean-spec and /lean-impl.
    • The five worked examples from the spec run as #guard checks.
    • q1 to q3 and the invariants I1, I2 and I3 are proved. I4 is stated and not proved.
    • check_ocpp16_composite.sh fails unless the proved theorems use only propext, Classical.choice and Quot.sound.
  • Correspondence review. formal-verification/correspondence/Ocpp16Composite.md, written by /correspondence-review on an independent model (Codex), followed by the pipeline owner's audit of its verdicts.
  • EVerest driver. formal-verification/tests/ocpp16_composite/everest_driver/ compiles EVerest's unmodified lib/everest/ocpp/lib/ocpp/v16/profile.cpp at dd696165 into a JSON command-line tool. It uses two shim headers, and build.sh downloads EVerest and its header-only dependencies.
  • DRT harness and report. drt_harness.py, Ocpp16CompositeRunner.lean and drt_report.md, which includes a draft EVerest issue. The issue has not been filed.

Results

Configuration Pass Warn Fail
Recurring anchors 2017 to 2025, 2,000 cases 2000 0 0
Recurring anchors 2010 to 2016, 2,000 cases 835 92 1073
Same cases with --leap-correct 2000 0 0
Self-test --mutate-stack, 300 cases 181 34 85

--leap-correct moves each recurring anchor in EVerest's input forward by the leap seconds since it. Every old-anchor divergence then disappears, which attributes all of them to leap seconds. The mechanism is profile.cpp:207-210, which does modulo arithmetic on date::utc_clock.

A warn means EVerest is below the model, or sets a limit where the model gives none. Q4 calls both conforming, but each one loses charging. The operator chose the second rule on 2026-10-27 (7d05c4e).

For review

  1. D4 correction after sign-off. /correspondence-review found that the signed D4 had lost "s ≤ t" when A5 and A6 were written in. Restored, and confirmed by the operator on 2026-10-27 (62163ab).
  2. Two reclassified verdicts. candidates and txResult are tested against signed resolutions A1 and A2, against the reviewer's approximation verdict. The operator approved this on 2026-10-27 (d6b51dd). A divergence there is a spec-reading gap, never an EVerest bug.
  3. Crosscheck Lean image. The model builds in the Crosscheck image (Lean v4.10.0) and with the local v4.28.0 toolchain (3c636c0). lean_check reports the same axioms on both, and lean_run prints the same section 3.13.7 composite. The DRT harness still calls local Lean, not lean_run.
  4. The draft EVerest issue at the end of drt_report.md needs human review before filing, per EVerest's policy on AI-assisted contributions.

How to reproduce

formal-verification/tests/ocpp16_composite/everest_driver/build.sh
formal-verification/lean/CrosscheckModel/check_ocpp16_composite.sh
python3 formal-verification/tests/ocpp16_composite/drt_harness.py --count 2000 --seed 20261024
python3 formal-verification/tests/ocpp16_composite/drt_harness.py --count 2000 --seed 20261024 --old-anchors
python3 formal-verification/tests/ocpp16_composite/drt_harness.py --count 2000 --seed 20261024 --old-anchors --leap-correct

The first needs curl, clang++ and Boost and fmt from Homebrew. The others need elan with toolchain leanprover/lean4:v4.28.0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA

hnipps and others added 12 commits October 24, 2026 14:26
…model and conformance harness

The intent states the problem, the outcome and the constraints, among
them the CC BY-ND licence on the spec and the upper-bound reading of the
composite schedule. The plan runs P0 to P8 from operator items through
a Lean model, a Quint lifecycle model, differential tests against
EVerest and MicroOcpp, a black-box harness, triage and publication.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
…plan

The plan is technical only. P0 now pins the spec source and installs
the toolchain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
The plan now builds a Lean model of the OCPP 1.6 composite schedule
with two proved properties and runs EVerest's 1.6 calculation against
it, in four steps behind a two-day build spike with a kill criterion.
The Quint model, the black-box harness, MicroOcpp, EVerest's 2.x code
and publishing move to a deferred list that returns only if the first
stage finds something.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
…hedule (S2)

Written with /informal-spec from OCPP 1.6 edition 2, sections 3.13.1 to
3.13.3, 3.13.7 and 7.8 to 7.14. It has 7 preconditions, 4 postconditions,
4 invariants, 5 worked examples and 10 open ambiguities, and waits for
human sign-off before /lean-spec. The plan's limit-precision citation
moves from 7.13 to 7.14, where the rule is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
…mposite schedule (S1)

The driver compiles EVerest's unmodified lib/ocpp/v16/profile.cpp at
dd696165 with two shim headers, re-implements the SmartChargingHandler
glue with line citations, and reads and writes JSON. build.sh downloads
EVerest, nlohmann/json 3.11.3 and date 3.0.1. On the section 3.13.7
example it reports the daily boundaries two seconds early, and controls
anchored in mid-2016 and 2020 give one and zero seconds, one per leap
second since startSchedule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
… record sign-off

The operator chose A1 to A3. A profile prevails only where its schedule
covers t, a TxProfile overrides per instant, and the model takes no
position on connector 0 versus connector c TxDefaultProfiles. A4 to A10
take the recommended readings. Four wording issues are recorded as notes
for the Open Charge Alliance, including recurrence across leap seconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
…chedule

Written with /lean-spec from the signed informal spec. Core Lean only, so
it checks with the local v4.28.0 toolchain while the Docker image is
unavailable. It type-checks with 19 sorry warnings: 12 definitions and 7
theorems, q1 to q3 and i1 to i4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
…rove I1 to I3

Written with /lean-impl. Every definition is filled, and the five worked
examples of the informal spec pass as #guard checks. q1 to q3 and the
invariants I1 (never above the ChargePointMaxProfile limit), I2 (a
prevailing TxProfile hides every TxDefaultProfile) and I3 are proved, and
check_ocpp16_composite.sh confirms they use only propext and Quot.sound.
It also fails on i4, which is still sorry. q2 now switches on the
prevailing TxProfile, as the signed Q2 says.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
…posite schedule (S3)

Ocpp16CompositeRunner.lean evaluates the Lean model on a JSON batch.
drt_harness.py generates profile sets within the signed preconditions,
runs EVerest's driver and the model, compares at every boundary and one
second either side, and shrinks flagged cases. --mutate-stack is a
self-test that must report fails, and --leap-correct attributes drift to
leap seconds by moving Recurring anchors in EVerest's input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
…port DRT against EVerest (S3, S4)

/correspondence-review ran on an independent model and found two
mismatches. scheduleStart now gives an Absolute profile its start
unconditionally, with covers requiring s <= t, and WellFormed states P6
over list positions. The signed D4 had lost s <= t when A5 and A6 were
applied, and is restored with a note pending operator confirmation. The
reviewer's approximation verdicts on candidates and txResult are
reclassified as abstraction, because they follow signed A1 and A2.

DRT over 2,000 cases with post-2016 recurring anchors finds no
divergence. With 2010 to 2016 anchors, 1,140 of 2,000 cases diverge, and
all of them pass once anchors are corrected for leap seconds: EVerest
moves daily and weekly boundaries one second earlier per leap second
since startSchedule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
…rence drift (S4)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
25 boxes are ticked, each with a pointer to its evidence. Four stay open
with a reason: the S1 and S2 example checks differ by the leap-second
finding, the operator has not yet reviewed the theorem statements or the
D4 correction, and the harness directory has no README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2XKM2iduWGdZ7nnVxVDMA
@hnipps
hnipps marked this pull request as ready for review October 26, 2026 15:07

@hnipps hnipps left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by pr-swarm — not written by a human

pr-swarm review complete. See inline comments.

@hnipps

hnipps commented Oct 26, 2026 •

Copy link
Copy Markdown
Member Author

Note

🤖 Automated comment by pr-swarm — not written by a human

Router-first swarm review. This is a comment, not an approval.

Verdict: ⚠️ REQUEST CHANGES (round 5 @ 3250cbe)

This round reviewed only the fix commit 3250cbe, which on its own carries LOW risk: no finding is above LOW. The verdict stays REQUEST CHANGES because the two HIGH findings from round 1 are still open and unchanged. They are the post-sign-off D4 edit with no recorded confirmation (formal-verification/specs/ocpp16_composite_informal.md:41), and the candidates/txResult reclassification with no named human authority (formal-verification/correspondence/Ocpp16Composite.md:35).

Tally over this round's 3 findings: 🟢 LOW ×1 · ⚪ NIT ×2

Key findings

LOW

  • formal-verification/tests/ocpp16_composite/drt_harness.py:88: int(f[1]) and int(f[3]) are unguarded. A Leap line with a non-numeric year or day, such as Leap 2016 Dec 3x ..., raises a bare ValueError traceback instead of reaching the `cannot check this Lea…

NIT

  • formal-verification/tests/ocpp16_composite/drt_harness.py:60: The except OSError branch covers a missing /etc/localtime, a permission error, and a path that exists but is not a symlink. 'is not a readable symlink' describes only the permission case.
  • formal-verification/tests/ocpp16_composite/everest_driver/build.sh:28: The new FAIL text is accurate, but it no longer mentions that a codeload mismatch may be a regenerated archive, which the comment at lines 14-17 still says. That is fine if the intent was only to drop…

Convergence

  • leap-line-int-parse-traceback, raised by router + crosscheck/byfuglien/fault-path + crosscheck/hellebuyck/intent, at formal-verification/tests/ocpp16_composite/drt_harness.py:88

Coverage

Lens Ran? Assessment
🧭 router (haiku) yes danger LOW, confidence HIGH, delegated nothing; Leap-line predicate now matches date's parser on what counts; one unguarded int parse remains
crosscheck/byfuglien (sonnet) yes, mandatory override (PR > 400 lines), scoped to 8faef29..3250cbe the check now agrees with date on which lines are Leap lines and on month case, and is stricter elsewhere (fail-closed); int parse is unguarded and the refusals are untested
crosscheck/hellebuyck (sonnet) yes, mandatory override, scoped to 8faef29..3250cbe nothing protected touched, and drt_report.md is still accurate; the commit title 'exactly as date does' overstates
Previous rounds (4)

round 1 @ 8cde273 — REQUEST_CHANGES: 21 findings (2 HIGH, 8 MEDIUM, 10 LOW, 1 NIT). Review-triage fixed 5 in b09a165, resolved 10 as nits, and deferred 6 to the author.
round 2 @ b09a165 — REQUEST_CHANGES (round-1 HIGHs carried): 5 findings on the fix commit (1 MEDIUM, 4 LOW), all fixed in 732e78b.
round 3 @ 732e78b — REQUEST_CHANGES (round-1 HIGHs carried): 6 findings on the fix commit (5 LOW, 1 NIT), all fixed in 8faef29.
round 4 @ 8faef29 — REQUEST_CHANGES (round-1 HIGHs carried): 4 findings on the fix commit (3 LOW, 1 NIT), 3 fixed in 3250cbe and 1 resolved as a nit.


Automated by pr-swarm — not a human review

The axiom gate printed nothing when Lean failed, because set -e exited
before the error branch. It now prints the Lean output and the status.

build.sh checks every download against the sha256 of the copy the DRT
results were produced with, so a moved tag or a truncated file fails.

The harness writes its report to the gitignored drt_report_latest.md by
default, so a default run no longer overwrites the curated drt_report.md,
and the header says when --mutate-stack or --leap-correct altered the
SUT's input. It also refuses to run when the host leap-second table, which
EVerest reads through USE_OS_TZDB, differs from LEAP_SECONDS.

The Lean model notes that q1 to q3 restate their definitions.

PR-Swarm-Automated: true
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hnipps hnipps left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by pr-swarm — not written by a human

pr-swarm review complete (round 2). See inline comments.

The leap-second check read $TZDIR/leapseconds, but date 3.0.1 ignores
TZDIR: it takes the zoneinfo directory from the /etc/localtime symlink on
macOS, and /usr/share/zoneinfo (uclibc first) on Linux. The harness now
finds the directory the same way, and parses month names without strptime.

build.sh removes a download whose hash does not match, so a re-run fetches
it again, and notes that codeload archives are not promised byte-stable.
The driver README describes the pins, drt_report.md says where a run's
report goes and that the harness checks the leap table, and the Lean note
names I1 to I4.

PR-Swarm-Automated: true
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hnipps hnipps left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by pr-swarm — not written by a human

pr-swarm review complete (round 3). See inline comments.

build.sh stamps each extracted tree with its archive's sha256 and
re-extracts when the stamp is missing or different, so a partial
extraction or a tree from before the hash check is replaced. The mismatch
message says a repeat failure means the upstream bytes changed.

The leap-second check exits with a message, not a traceback, when
/etc/localtime is not a symlink, and refuses a Leap line it cannot check
(a negative leap second, which date would count as inserted, or an unknown
month). The Lean note says I4's proof is open, and the drt_report.md line
no longer claims how the report was written.

PR-Swarm-Automated: true
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hnipps hnipps left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by pr-swarm — not written by a human

pr-swarm review complete (round 4). See inline comments.

The leap-second check now examines every line whose first word is Leap,
refuses one that is short, negative, or whose month is not one of date's
case-sensitive Jan to Dec names, and says why in its comment. The
/etc/localtime message covers a missing file as well as a regular one, and
build.sh's mismatch message no longer guesses at the cause.

PR-Swarm-Automated: true
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hnipps hnipps left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by pr-swarm — not written by a human

pr-swarm review complete (round 5). See inline comments.

hnipps and others added 7 commits October 26, 2026 11:39
A Leap line whose year or day is not a number now gets the same
"cannot check this Leap line" exit as the other refusals instead of a
ValueError traceback, and the /etc/localtime message covers every reason
readlink can fail.

PR-Swarm-Automated: true
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion

The operator confirmed on 2026-10-27 that D4 requires s ≤ t, as restored after /correspondence-review CR-OC16-01.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d txResult reclassification

The operator approved on 2026-10-27 testing both definitions against signed resolutions A1 and A2, with divergences there reported as spec-reading gaps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…warn/fail

A pass is the Q4 inequality (EVerest at or below the model), so it did not
show how often the two agree. The harness now prints and reports instants
compared, instants with a model limit, exact matches, EVerest below, EVerest
above, EVerest limit where the model gives none, neither gives a limit, and
cases where the model gives no limit anywhere. drt_report.md records the
measured counters for the three configurations and the seed-1 run, and one
minimal case of EVerest limit where the model gives none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The image pins Lean v4.10.0, whose core has no List.Pairwise, so lean_check rejected periodsWellFormed and WellFormed. A local pairwise with the same meaning replaces it. lean_check now passes in the image with the same axioms as the local v4.28.0 gate, and lean_run prints the same section 3.13.7 composite as the local runner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…audit

The header said DRT readiness BLOCKED and the checklist said CR-OC16-01 and
CR-OC16-02 stayed open, while the audit section said READY and both were fixed
before DRT ran. The reviewer's original verdicts stay visible, labelled as
such, and each affected section now states its revised class and status. The
candidates and txResult reclassification is marked as approved by the operator
on 2026-10-27.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…PP 1.6 docs said out of scope

The signed spec's module boundary and A9, the DRT report, the intent doc and
the Lean Kind doc comment called Relative profiles, phases, amp and watt
conversion, connector-0 aggregation, offline behaviour, transactionId matching
and OCPP 2.x out of scope or excluded. Each now says not yet reached and names
the blocking property and the open question, per docs/VISION.md. The operator
approved the wording change on 2026-10-27, and the signed spec records that it
changes no rule. The Lean change is a doc comment only. The DRT report's
Dafny-verified row now reads skipped as redundant, since those slices are not
unreached.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hnipps and others added 2 commits October 27, 2026 16:48
…raction

The correspondence doc cited about 55 line ranges of a pdftotext extraction in
/tmp, which no reader can open. Each citation is now an OCPP 1.6 edition 2
section number, with the field name where one field is meant and a verbatim
quote (checked against the extraction) where a classification depends on the
wording. A Source provenance section records the PDF's SHA-256, the retrieval
date, the extraction command, that the origin is not recorded, and the command
that fetches the pinned EVerest commit. The PDF and extractions stay out of the
repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s none

Q4 counts it as conforming, but it loses charging, as EVerest below the model does. The operator chose a warn on 2026-10-27. The 2010 to 2016 run moves from 860/67/1073 to 835/92/1073. The other runs and the mutate-stack self-test are unchanged. drt_witnesses_old_anchors.md is regenerated from the new run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…CPP 1.6 DRT report

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hnipps
hnipps merged commit bdffc38 into main Nov 7, 2026
5 checks passed
@hnipps
hnipps deleted the worktree-ocpp-smart-charging-plan branch November 7, 2026 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant