Skip to content

Commit 396d22f

Browse files
doublegateclaude
andauthored
docs: sync coverage figures to the generated report (351/443) (#311)
* docs: sync coverage figures to the generated report (351/443) The generated `docs/accuracysnes-coverage.md` reads 297 on-cart + 54 scene-only = 351; the hand-maintained figures in STATUS.md and accuracysnes-plan.md still said 350 (297 + 53). PR #302's "unchanged at 350" claim was wrong and the CHANGELOG says so. Three other hand-maintained lines had drifted behind this cycle's work and are corrected in the same pass: - the reference list — Mesen2, snes9x and ares, three independent references as of v1.29.0, with MesenCE named separately as the per-dot compositor's blueprint rather than as a cross-validation reference; - Mesen2's known-failure rationale — F1.03 alone (it clocks both ports out of one latch, so it needs the port-2 input the Lua runner cannot drive), not "two PAD2_CONTRACT rows"; - the scene tally — 54 declared, all matching on both scene hosts. STATUS.md also gains a paragraph on the headless ares host: why a third reference was needed (the provenance rule counts ares and bsnes as one, so several rows sat at 2-vs-1 with no tiebreaker) and the defect it found in ares rather than in this emulator. No code or ROM change; the battery and its pass rate are untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: the scene tally in plan.md §5 had drifted too (53 -> 54) Caught by the bot review, which flagged the internal contradiction between the updated table at the top and this line further down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 08361ad commit 396d22f

3 files changed

Lines changed: 34 additions & 12 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
## [Unreleased]
1111

12+
### Fixed
13+
14+
- **CORRECTION: the battery covers 351 of 443 assertions, not 350.** The `E5.06` entry below says
15+
coverage was "350 either way"; the regenerated `docs/accuracysnes-coverage.md` reads **297 on-cart
16+
+ 54 scene-only = 351**. `docs/STATUS.md` and `docs/accuracysnes-plan.md` are synced to the
17+
generated figure, along with three other hand-maintained lines that had drifted behind this
18+
cycle's work: the reference list (now **Mesen2, snes9x and ares**, with MesenCE named separately as
19+
the compositor blueprint rather than as a cross-validation reference), Mesen2's known-failure
20+
rationale (`F1.03` alone, and *why* — it clocks both ports out of one latch), and the scene tally
21+
(54, on both scene hosts). The generated report is the authority precisely because these drift;
22+
this is the periodic reconciliation, not a coverage change.
23+
1224
### Added
1325

1426
- **`E3.06` rewritten to poll and accumulate — and ares agrees with it exactly.** The row read

‎docs/STATUS.md‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@ compositor (`docs/adr/0014`, T-CA-10) becomes the sole renderer — its compose/
2323
per-dot against live registers (mid-line CGRAM/OAM/`INIDISP` at dot resolution; BG-fetch-ahead is
2424
Phase 4c, not yet landed) — and the first-party AccuracySNES
2525
cartridge matures into a usable instrument (AccuracyCoin-style paged menu + automatic skyline
26-
results + per-test B-skip + a Select WRAM debug viewer) with its battery at **350 of 443** dossier
27-
assertions (297 on-cart + 53 rendered scenes).
26+
results + per-test B-skip + a Select WRAM debug viewer) with its battery at **351 of 443** dossier
27+
assertions (297 on-cart + 54 rendered scenes).
2828
**`v1.22.0 "Horizon"` fixes the long-standing DSP-1 continuous-mode Mode-7 flat floor** (Pilotwings
2929
flight, SMK track): the shared NEC-DSP host-sync (`run_until_rqm`) stopped one host-write too early,
3030
so reading a host input word raised RQM before the firmware cleared DRC to 16-bit — mis-framing the
@@ -216,6 +216,13 @@ pass rate, with `Contested`/`Novel` recorded but never counted. Behaviour hardwa
216216
define is captured as a **golden vector**, never scored. **Real-hardware validation has not been
217217
done and is the honest ceiling on its authority.**
218218

219+
As of `v1.29.0` a **headless ares host** (`scripts/accuracysnes/ares_host/`) runs the battery as a
220+
third reference, which matters because this project's provenance rule counts ares and bsnes as one:
221+
several rows sat at 2-vs-1 with no way to break the tie. It also produced the first defect
222+
AccuracySNES has found in a *reference* emulator rather than in this one — ares' `$F1` handler
223+
negates the timer-2 counter reset, so any later `$F1` write zeroes `T2OUT`, while its own timers 0
224+
and 1 use the un-negated form.
225+
219226
The composed multi-suite battery below remains in force alongside it — AccuracySNES adds a layer,
220227
it does not replace the per-opcode oracles. The accuracy story here is instead a **composed multi-layer battery** across
221228
independently-sourced suites (`docs/testing-strategy.md`). Rather than force these heterogeneous
@@ -234,7 +241,7 @@ tracked here, always current, reaffirmed every release:
234241
| Core/Curated coprocessors (oracle-gated) | ✅ **3 / 3, honesty gate green** | DSP-1 (4 commercial ROMs), Super FX/GSU (58 Krom ROMs + per-opcode suite), SA-1 (18 commercial carts) — `ORACLE_COPROCESSORS` |
235242
| BestEffort coprocessors, real-title validated | ✅ **10 / 11** | DSP-2, DSP-4, ST010, S-DD1, CX4, OBC1 boot a real commercial title to real gameplay content. **DSP-3 (SD Gundam GX) + ST011 (2-dan Morita Shougi)** are liveness/determinism-validated — detection + `host_accesses > 0` + bit-identical framebuffer (`dsp3_st011_oncart`). **ST018 (Nidan Morita Shogi 2) + S-RTC (Daikaijuu Monogatari II)** are detection + boot + determinism-validated against their real carts (`srtc_st018_oncart`), but their coprocessor's core function is **usage-gated** (the ST018 shogi AI runs only on the computer's move; the S-RTC clock is read at specific moments), so it is not exercised in a headless boot |
236243
| BestEffort coprocessors, no booting dump | ⚠️ **1 / 11** | SPC7110 — **two** local Tengai Makyou Zero dumps both fail to boot to content: the 7 MiB one is a fan-translation ROM hack needing a patch-only memory region no cartridge has (`docs/audit/spc7110-boot-crash-2026-07-08.md`); a 5 MiB dump (sha256 `8620203d…`) freezes at a near-blank screen with zero coprocessor activity, and does not match the documented-good original (`69d06a3f…`). The correct original-cartridge dump remains the ROM-sourcing gap (`docs/rom-test-corpus.md`) |
237-
| AccuracySNES (first-party battery) | ✅ **350 / 443 assertions covered, 100% on-cart pass** | A 338-test self-scoring battery spanning Groups A-G. **350 of 443** dossier assertions covered: **297 on-cart** + **53 via rendered scenes** (`docs/adr/0013`), kept as separate columns on purpose (`docs/accuracysnes-coverage.md`, regenerated with the ROM so it cannot drift). Every scored row is inject-verified for non-vacuity and cross-validated headlessly against **Mesen2, snes9x, and MesenCE** (the last built this cycle as the per-dot compositor's blueprint and exact-frame oracle). Provenance gate green. Ships an AccuracyCoin-style on-cart UI as of `v1.21.0` (paged menu + automatic skyline results + per-test B-skip + a Select WRAM debug viewer). Remaining ~93 assertions are compositor-gated hi-res, second-image G-rows, treacherous CPU/dot-model timing, or provably uncoverable — though "uncoverable" has twice proved too strong (`E5.06` and `B2.02`/`B2.03` all landed after being parked) (`docs/accuracysnes-coverability-audit-2026-07-23.md`) |
244+
| AccuracySNES (first-party battery) | ✅ **351 / 443 assertions covered, 100% on-cart pass** | A 338-test self-scoring battery spanning Groups A-G. **351 of 443** dossier assertions covered: **297 on-cart** + **54 via rendered scenes** (`docs/adr/0013`), kept as separate columns on purpose (`docs/accuracysnes-coverage.md`, regenerated with the ROM so it cannot drift). Every scored row is inject-verified for non-vacuity and cross-validated headlessly against **Mesen2, snes9x and ares** — three independent references as of `v1.29.0`, with MesenCE additionally serving as the per-dot compositor's blueprint and exact-frame oracle. Provenance gate green. Ships an AccuracyCoin-style on-cart UI as of `v1.21.0` (paged menu + automatic skyline results + per-test B-skip + a Select WRAM debug viewer). Remaining ~92 assertions are compositor-gated hi-res, second-image G-rows, treacherous CPU/dot-model timing, or provably uncoverable — though "uncoverable" has twice proved too strong (`E5.06` and `B2.02`/`B2.03` all landed after being parked) (`docs/accuracysnes-coverability-audit-2026-07-23.md`) |
238245
| Determinism contract | ✅ **proven** | bit-identical framebuffer/audio across runs; save-state round-trip proven across all three board tiers (no-coprocessor, Curated, BestEffort) |
239246

240247
**Named residuals, tracked not hidden:** the 65816 `e1.e` divergence (`docs/adr/0002`);

‎docs/accuracysnes-plan.md‎

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,14 @@ AccuracySNES closed ticket **T-04**. The follow-on tickets minted here are **T-0
1414
| | |
1515
|---|---|
1616
| Tests | **338** (scoring + golden vectors + region SKIP per image) — *tests, not assertions; see the note below the table* |
17-
| Assertion coverage | **350 of 443** dossier assertions — **297 on-cart** + **53 rendered scenes**, kept as separate columns (`docs/accuracysnes-coverage.md`) |
18-
| Rendered scenes | **54** declared and all cross-validated (`docs/adr/0013`); **53** are the dossier rows they are the only cover for |
17+
| Assertion coverage | **351 of 443** dossier assertions — **297 on-cart** + **54 rendered scenes**, kept as separate columns (`docs/accuracysnes-coverage.md`) |
18+
| Rendered scenes | **54** declared, all blessed and matching on both scene hosts (`docs/adr/0013`); **54** dossier rows have a scene as their only cover |
1919
| Pass rate | **100.00%** on-cart, floor enforced at 1.00 by `tests/accuracysnes.rs` |
20-
| Cross-validated | RustySNES and Mesen2 agree on every test but two `PAD2_CONTRACT` rows the Lua runner cannot drive; snes9x agrees on every test but a handful of recorded reference bugs with citations in `scripts/accuracysnes/crossval.sh`; a headless **MesenCE** (this cycle) is the per-dot compositor's blueprint + exact-frame oracle. All images. |
20+
| Cross-validated | **Three references** as of `v1.29.0`. Mesen2 agrees on every test but `F1.03`, which clocks both ports out of one latch and so needs the port-2 input its Lua runner cannot drive; snes9x has 14 recorded divergences and **ares** 4, each with a citation in `scripts/accuracysnes/crossval.sh`. A headless **MesenCE** is separately the per-dot compositor's blueprint + exact-frame oracle. All images. |
2121
| Groups shipped | **A** (65C816) · **B** (5A22) · **C** (PPU, on-cart and rendered) · **D** (DMA/HDMA) · **E** (SPC700 + S-DSP) · **F** (controller ports) · **G** (cartridge/memory map) — all seven, all partial |
2222
| On-cart UI | AccuracyCoin-style paged menu + automatic skyline results + per-test B-skip + a Select WRAM debug viewer (`v1.21.0`) |
2323
| Defects found in this emulator | **12+** — see §5 |
24+
| Defects found in a *reference* emulator | **1** — ares' `$F1` handler negates the timer-2 counter reset (`scripts/accuracysnes/ares_host/README.md`) |
2425

2526
These counts are maintained by hand and will drift. **`docs/accuracysnes-coverage.md` is the
2627
authority**: it is regenerated with the ROM, so it cannot.
@@ -742,10 +743,12 @@ Chosen 2026-08-01, now that both halves of the Mesen2 oracle arbitrate. Two corr
742743
| `E9.02` | noise output is **highpass-filtered** `[ERRATA]` | DSP-observable via OUTX on a NON voice; errata-flagged |
743744
| `E5.06` | BRR 15-bit wrap: clamp to 16 bits, then `+4000h..+7FFFh → -4000h..-1` | reuses the landed `E5` decoder scaffolding |
744745

745-
**All three landed 2026-08-01.** Battery 338 tests, 100% on-cart, and both references agree:
746-
`snes9x: OK (14 known divergence(s))`, `Mesen2: OK (2 known divergence(s))`, 53/53 scenes on each.
747-
`MESEN2_KNOWN_FAILURES` came **down** from 3 to 2 in the process, because the redesign below removed
748-
the one Mesen2 divergence that was ours rather than the harness's.
746+
**All three landed 2026-08-01.** Battery 338 tests, 100% on-cart, and the references agree:
747+
`snes9x: OK (14 known)`, `Mesen2: OK (1 known)`, `ares: OK (4 known)`, 54/54 scenes on each scene
748+
host. `MESEN2_KNOWN_FAILURES` came **down** from 3, first to 2 because the redesign below removed the
749+
one Mesen2 divergence that was ours rather than the harness's, and then to 1 when `F1.10`'s Mesen2
750+
verdict proved phase-fragile (it flipped to passing when an unrelated `E3.06` rewrite moved the
751+
cart's execution phase) and was retracted rather than re-recorded.
749752

750753
### `E8.01` — two rejected drafts, and what they were really measuring
751754

@@ -2337,8 +2340,8 @@ exists.
23372340
stay in their own tier. `crossval.sh` gates on them, and per rule 4 a golden is committed only
23382341
once the references agree.
23392342

2340-
**Status: 53 scenes blessed** (the regenerated `docs/accuracysnes-coverage.md` is authoritative — this
2341-
hand-maintained figure had drifted from an earlier "41"), covering the scene tier across `C4`-`C8`,
2343+
**Status: 54 scenes blessed** (the regenerated `docs/accuracysnes-coverage.md` is authoritative — this
2344+
hand-maintained figure had drifted from an earlier "41", and again from "53"), covering the scene tier across `C4`-`C8`,
23422345
`C10`, `C11` and `C12`. The
23432346
first three disagreed with the references on first run, and in all three cases RustySNES was
23442347
wrong: the BG vertical fetch was a line late, and mosaic quantised the BG row instead of the

0 commit comments

Comments
 (0)