Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 24 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
status `$01`), so with RustySNES and snes9x it is **3 against 1 with Mesen2 the outlier**, and the
row comes off the "unexplained, needs a fourth opinion" list it has been on since the oracle fix.

**Five rows where ares disagrees with the cart** — `C7.05`, `C7.10`, `E8.02`, `E3.06`, `F1.10` —
are recorded and **not** adjudicated. `F1.10` is suspect-of-the-host first: it is a
`PAD2_CONTRACT` row and this host's port detection is assumed rather than verified. Not wired into
`crossval.sh` for that reason — an `ARES_KNOWN_FAILURES` constant encoding unexamined
disagreements would be worse than no third reference.
**Five rows where ares disagrees with the cart** — and **three of them are rows snes9x already
fails**, which the tally alone does not show. `C7.10` and `F1.10` become **2-vs-2** (RustySNES +
Mesen2 against snes9x + ares); `C7.05` is 2-vs-2 with the two dissenters failing on *different*
codes; only `E8.02` and `E3.06` are ares-only. So ares is corroborating snes9x more than it is
standing alone. Not wired into `crossval.sh` — an `ARES_KNOWN_FAILURES` constant encoding
unadjudicated disagreements would be worse than no third reference.

**`F1.10` deserves a hard look.** fullsnes puts the automatic read's start ~dot 32.5-95.5 into the
first vblank line rather than at the vblank edge. snes9x fails that row, ares fails it, and
`crossval.sh` records Mesen2 failing it too — which would leave **RustySNES passing alone**, on a
row it passes only because of a deliberate fix. The standing heuristic says RustySNES failing
alone means a real bug; it should say the same about RustySNES *passing* alone.
Comment on lines +27 to +38

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Keep the F1.10 result explicitly conditional.

This entry states that only E8.02 and E3.06 are ares-only, then treats Mesen2's F1.10 failure as established. The later entry from Line 819 through Line 824 marks that attribution as doubtful. If Mesen2 really fails F1.10, that row is not ares-only.

Proposed wording
-  codes; only `E8.02` and `E3.06` are ares-only. So ares is corroborating snes9x more than it is
+  codes; only `E8.02` and `E3.06` are ares-only in the measured set; `F1.10` remains unresolved.
+  So ares is corroborating snes9x more than it is

-  `crossval.sh` records Mesen2 failing it too — which would leave **RustySNES passing alone**, on a row
+  `crossval.sh` attributes a failure to Mesen2 — if that attribution is correct, **RustySNES would be
+  passing alone**, on a row

As per path instructions, **/*.md: “Docs are the spec, not a changelog. Flag prose that has drifted from the code it describes rather than style nits.”

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
**Five rows where ares disagrees with the cart** — and **three of them are rows snes9x already
fails**, which the tally alone does not show. `C7.10` and `F1.10` become **2-vs-2** (RustySNES +
Mesen2 against snes9x + ares); `C7.05` is 2-vs-2 with the two dissenters failing on *different*
codes; only `E8.02` and `E3.06` are ares-only. So ares is corroborating snes9x more than it is
standing alone. Not wired into `crossval.sh` — an `ARES_KNOWN_FAILURES` constant encoding
unadjudicated disagreements would be worse than no third reference.
**`F1.10` deserves a hard look.** fullsnes puts the automatic read's start ~dot 32.5-95.5 into the
first vblank line rather than at the vblank edge. snes9x fails that row, ares fails it, and
`crossval.sh` records Mesen2 failing it too — which would leave **RustySNES passing alone**, on a
row it passes only because of a deliberate fix. The standing heuristic says RustySNES failing
alone means a real bug; it should say the same about RustySNES *passing* alone.
**Five rows where ares disagrees with the cart** — and **three of them are rows snes9x already
fails**, which the tally alone does not show. `C7.10` and `F1.10` become **2-vs-2** (RustySNES +
Mesen2 against snes9x + ares); `C7.05` is 2-vs-2 with the two dissenters failing on *different*
codes; only `E8.02` and `E3.06` are ares-only in the measured set; `F1.10` remains unresolved.
So ares is corroborating snes9x more than it is
standing alone. Not wired into `crossval.sh` — an `ARES_KNOWN_FAILURES` constant encoding
unadjudicated disagreements would be worse than no third reference.
**`F1.10` deserves a hard look.** fullsnes puts the automatic read's start ~dot 32.5-95.5 into the
first vblank line rather than at the vblank edge. snes9x fails that row, ares fails it, and
`crossval.sh` attributes a failure to Mesen2 — if that attribution is correct, **RustySNES would be
passing alone**, on a row it passes only because of a deliberate fix. The standing heuristic says RustySNES failing
alone means a real bug; it should say the same about RustySNES *passing* alone.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CHANGELOG.md` around lines 27 - 38, Revise the CHANGELOG discussion of F1.10
so the Mesen2 failure remains explicitly conditional rather than established.
Update the statement that identifies E8.02 and E3.06 as the only ares-only rows
to acknowledge that F1.10 is also not ares-only if Mesen2’s recorded failure is
confirmed, while preserving the later attribution doubt.

Source: Path instructions


Three setup steps turned out to be mandatory and each cost a round, because all three fail as the
*same* segfault inside `System::load` with a backtrace pointing at memory setup rather than at
Expand Down Expand Up @@ -804,6 +811,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- **`crossval.sh`'s Mesen2 known-failure comment is keyed on rows, not catalogue indices, and one of
its attributions is flagged as doubtful.** The comment named `idx279 F1.03` and `idx286 F1.10`; in
the current catalogue those indices are `F1.01` and `F1.08`, because an index moves whenever a test
is added ahead of it. An index in a comment is a fact with a shelf life.

Separately, that comment attributes Mesen2's `F1.10` failure to the port-2 limitation, and
`f1_require_contract` reads `$4016` only — port 1 — while `F1.10` code 2 means "`$4212` read busy
at the very start of the vblank line", which does not involve controller state. The snes9x block a
few lines above independently says Mesen2 *passes* `F1.10`. Both cannot be true. Marked as doubted
rather than quietly rewritten: resolving it needs Mesen2's failing set read at `DONE` and mapped
through `SOURCE_CATALOG.tsv`, a measurement nobody has taken since the catalogue grew.

- **AccuracySNES: `E9.01` and `E9.02` stand down as SKIP on a menu restart.** `E9.02` steps the noise
LFSR by design and nothing can put it back: `FLG` bit 7 is the DSP's *soft* reset and does not
re-seed the shift register (checked against ares, which re-seeds only in `DSP::power`). A comment
Expand Down
45 changes: 28 additions & 17 deletions scripts/accuracysnes/ares_host/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,23 +31,34 @@ the outlier**, and the row comes off the "unexplained, needs a fourth opinion" l

## The five rows ares disagrees with the cart about

New information, and **not yet adjudicated** — the cart, snes9x and Mesen2 all pass these:

| idx | row | code |
|---:|---|---:|
| 116 | `C7.05` | 1 |
| 120 | `C7.10` | 1 |
| 265 | `E8.02` | 3 |
| 276 | `E3.06` | 2 |
| 288 | `F1.10` | 2 |

**Treat `F1.10` as suspect-of-this-host first.** It is a `PAD2_CONTRACT` row, and this host's port
detection (`port->name().find("2")` on the button's grandparent) is *assumed* to work, not verified.
Check that before concluding anything about ares.

The standing heuristic — three implementations agreeing usually means a broken test, one disagreeing
usually means a real bug — cuts an unfamiliar way here: it is **ares** alone, on rows the other three
pass.
**Three of the five are rows snes9x already fails**, with rationales in `crossval.sh`. That was not
obvious from the tally and it changes what each row means:

| row | ares code | who else fails it | so the split is |
|---|---:|---|---|
| `C7.05` | 1 | snes9x (code **2** — a different failure) | 2-vs-2, and the two dissenters disagree with each other |
| `C7.10` | 1 | snes9x | **2-vs-2** — RustySNES + Mesen2 against snes9x + ares |
| `F1.10` | 2 | snes9x | **2-vs-2** (but see the caveat below) |
| `E8.02` | 3 | nobody | ares alone |
| `E3.06` | 2 | nobody | ares alone |

So ares is *corroborating snes9x* on three rows rather than standing alone, and only `E8.02` and
`E3.06` are genuinely ares-only. **Nothing here is adjudicated** — this is the shape of the
disagreement, not a verdict on it.

**A correction to this file's first version:** it said `F1.10` was a `PAD2_CONTRACT` row and to
suspect this host's port detection first. That is wrong. `f1_require_contract` reads `$4016` only —
port 1 — and `F1.10` code 2 means "`$4212` read busy at the very start of the vblank line", which
does not involve controller state at all. The claim came from the Mesen2 known-failure grouping in
`crossval.sh`, which attributes *its* `F1.10` failure to the port-2 limitation and is itself now
flagged as doubtful there.

**`F1.10` is the interesting one.** fullsnes says the automatic read begins ~dot 32.5–95.5 of the
first vblank line rather than at the vblank edge. snes9x fails it (documented instant-latch), ares
fails it, and if the Mesen2 attribution is right then Mesen2 fails it too — which would leave
**RustySNES passing alone**, on a row it only passes because of a deliberate fix. This project's
heuristic says RustySNES failing alone means a real bug; it should say the same about RustySNES
*passing* alone. Worth a hard look before treating it as settled either way.

## Three setup steps that are not optional, each of which cost a debugging round

Expand Down
17 changes: 15 additions & 2 deletions scripts/accuracysnes/crossval.sh
Original file line number Diff line number Diff line change
Expand Up @@ -171,14 +171,27 @@ SNES9X_KNOWN_FAILURES=14
# read at `R_DONE == $A5` and reproduced exactly across runs, both completing at frame 479. A count
# alone would have hidden the composition, which is the whole point of enumerating them:
#
# idx279 F1.03 code 2 | Group F input rows. mesen_crossval.lua makes exactly ONE emu.setInput
# idx286 F1.10 code 2 | call because in this build the port argument does not select a
# **Key these on the ROW, not the index.** The catalogue index of a row moves whenever a test is
# added ahead of it, and this comment has already gone stale once: it named `idx279 F1.03` and
# `idx286 F1.10`, and in the current catalogue those indices are `F1.01` and `F1.08`. An index in a
# comment is a fact with a shelf life; the row name is not. Map with SOURCE_CATALOG.tsv.
#
# F1.03 code 2 | Group F input rows. mesen_crossval.lua makes exactly ONE emu.setInput
# F1.10 code 2 | call because in this build the port argument does not select a
# | controller -- 0/1/2 all land on controller 1 -- so a second call for
# | port 2 overwrites port 1 with a mask containing no Start and the cart
# | never leaves its menu. The cost is that PAD2_CONTRACT-dependent rows
# | cannot pass here. Covered by the in-repo harness and snes9x, which do
# | drive both ports. See that script's comment for the full account.
#
# **`F1.10`'s attribution above is DOUBTED, and stated as doubted rather than quietly fixed.**
# `f1_require_contract` reads `$4016` only — port 1 — so `F1.10` does not depend on `PAD2_CONTRACT`
# at all, and its code 2 means "$4212 read busy at the very start of the vblank line", which has
# nothing to do with which controllers are held. The snes9x block above independently says Mesen2
# *passes* `F1.10`. Both cannot be right. Resolving it needs Mesen2's failing SET read at `DONE` and
# mapped through SOURCE_CATALOG.tsv, which is a measurement nobody has taken since the catalogue
# grew; until then treat the count (2) as trustworthy and this row attribution as not.
#
# E8.01 used to be a third entry here, and is not any more. Its two rejected drafts asserted
# something a phase the cart cannot control got to decide -- the delay from a KON write to the
# voice starting is a sawtooth in that phase -- so the row passed on the PAL image and failed on
Expand Down
Loading