test: the conveyor corpus has no independent numeric oracle -- a lockstep VM+wasm drift would pass every test

[bpowers]

test/conveyors/reference_prototype.py is presented by test/conveyors/README.md and docs/design/conveyors.md sections 13/15 as the acceptance oracle for the core continuous conveyor. It is an executable transcription of the spec's per-DT algorithm (sections 4-7) and asserts the section 15 worked-example trajectories.

Nothing in CI or any test runs it. It is a standalone script (python3 test/conveyors/reference_prototype.py). The only automatic pin on the VM's belt numerics is src/simlin-engine/src/conveyor_tests.rs, which transcribes the prototype's expected values by hand into Rust asserts. The prototype is therefore a source document, not a live oracle, and the two can silently diverge.

Why it matters

This got sharper with #924. The corpus gate (simulate_special_path in src/simlin-engine/tests/integration/simulate.rs) runs each test/conveyors/ fixture through the VM, the protobuf round-trip, the XMILE round-trip, and the wasm backend, and asserts they all agree column-for-column. Its stated job is round-trip fidelity and VM-vs-wasm parity -- not numeric truth. The VM is the oracle for that harness. Consequences:

• A bug in the VM's belt pass that the wasm lowering faithfully mirrors -- which it is designed to do; src/simlin-engine/src/wasmgen/belt.rs reproduces conveyor.rs including its known quirks, see engine: <leak_integers/> is silently ignored on an exponential-leak conveyor #942 -- passes the corpus gate, the parity tests, and the round-trip tests. Every test is green and every answer is wrong.
• Four of the six corpus fixtures (leaky_conveyor, conveyor_containers, discrete_conveyor, queue_coupled_conveyor) were added by wasm: drop the conveyor Unsupported reject and put conveyor fixtures under the parity harness #924 with the VM as their sole oracle. Their trajectories are not checked against anything derived from the spec.
• conveyor-engine: a queue-coupled discrete conveyor's slat contents are not integral, contra conveyors.md 6.4 rule 1 #950 is a live instance of exactly this failure mode: the queue-coupled belt's slat contents are fractional where conveyors.md section 6.4 rule 1 says they must be integral, both backends agree, and no test notices.

Components

test/conveyors/ (reference_prototype.py, README), src/simlin-engine (conveyor_tests.rs, tests/integration/simulate.rs), docs/design/conveyors.md.

Requested work

1. Extend reference_prototype.py beyond the core continuous conveyor to cover:

   • leak zones (section 5.3, the zone_start/zone_end path schedule)
   • discrete admission and the per-time-unit <in_limit> budget (sections 6.3/6.4)
   • the queue-coupled serve (section 11 / queues.md section 9)

   The prototype already covers transport, capacity, inflow limit, linear + exponential leak, and non-integer-transit rounding.

2. Run it from a Rust test against the four new fixtures: parse each .xmile, feed the belt parameters to the prototype, and compare its trajectory to the VM's at the corpus comparator's existing epsilons. This is the check that catches a lockstep VM+wasm drift; nothing else does.

3. Keep it cheap. cargo test --workspace runs under a hard 3-minute wall-clock cap in pre-commit and CI, and individual unit tests must finish in a few seconds on a debug build.

Acceptable alternative if shelling out to python from a Rust test is too slow or too fragile a dependency: port the prototype's extended scenarios into conveyor_tests.rs as hand-derived closed forms. But the value lies entirely in the derivation being independent of the implementation -- a mechanical transcription of what the VM currently prints is worthless and must not be what lands.

Two stale doc claims to fix as part of this

Both currently assert the prototype pins the VM, which it does not:

• test/conveyors/README.md (~line 8): "the VM's belt pass is itself pinned by reference_prototype.py (below) and by hand-derived closed forms in the Rust unit tests"
• docs/design/conveyors.md section 13 (~line 1258): "the bytecode VM is the oracle, pinned in turn by reference_prototype.py"

#924 is rewording both to the truth -- that the automatic pin is conveyor_tests.rs, which transcribes the prototype -- and will reference this issue from the README's oracle paragraph. So the reword is handled; what remains here is the actual wiring.

Related

• conveyor-engine: a queue-coupled discrete conveyor's slat contents are not integral, contra conveyors.md 6.4 rule 1 #950 -- the spec-vs-VM contradiction this class of gap allowed to persist.
• engine: <leak_integers/> is silently ignored on an exponential-leak conveyor #942 -- a VM quirk the wasm lowering mirrors deliberately, invisible to parity tests by construction.
• test: <arrest> and <sample> have no conveyor corpus fixture (proto/XMILE round-trip untested) #949 -- corpus fixture coverage gaps. Different axis: test: <arrest> and <sample> have no conveyor corpus fixture (proto/XMILE round-trip untested) #949 is "features with no fixture"; this is "fixtures with no independent oracle". Note test: <arrest> and <sample> have no conveyor corpus fixture (proto/XMILE round-trip untested) #949's suggested approach explicitly says "No expected-output CSV is needed -- the VM is the oracle for that harness," which is the assumption this issue asks to stop relying on.
• conveyor-engine: wasm backend does not support conveyor/queue models (loud reject today; needs plan + parity coverage) #884, wasm: publish conveyor container-access values at step start #923, wasm: drop the conveyor Unsupported reject and put conveyor fixtures under the parity harness #924 -- the wasm conveyor parity work whose harness this issue backstops.

How it was discovered

Identified during review of #924 on branch conveyor-engine.