Add Tests-fix CI-revert fallback ladder and scratch-file guidance

RESEARCH.md

@@ -47,6 +47,7 @@ Last reviewed: 2026-07-25.
 - PR #34 (Structure Phase 9 self-audit prompts and label gap issues): shipped 2026-05-25. Observed effect: pending — needs N cycles of data. Extends the rubric-over-free-form pattern from Phase 4 (self-review) to Phase 9 (self-audit), with a 6-item structured checklist and a 5-label taxonomy for filed gap issues.
 - PR #54 (Apply consolidated template-rule backlog, #44/#46/#47): shipped 2026-06-14. Observed effect: pending — needs N cycles of data. Sharpens the external-anchor definition (#44: UNVERIFIED handling when the anchor can't run; #46: green CI is not verification when CI's scope excludes the changed files) and grounds the Tests-fix rubric item empirically (#47: stash-and-run FAIL→PASS, not scored from reasoning alone) — directly targeting the self-preference failure mode this finding's evidence documents for judgment-based rubric scoring.
 - PR #57 (Fix documentation drift across the three sources of truth): shipped 2026-07-25. Observed effect: pending — needs N cycles of data. Restores `{{EXTERNAL_SIGNAL_LABEL}}` in the Phase 1 carried-over-PR bullet, where PR #54 had reintroduced a hardcoded "CI must be green first" after PR #28 generalized the anchor. An anchor named `CI` in a repo that has no CI is an anchor the loop cannot check, which degrades the self-review to exactly the unanchored critique this finding shows is neutral-to-harmful.
+- PR #67 (Add Tests-fix CI-revert fallback ladder): shipped 2026-07-26. Observed effect: pending — needs N cycles of data. Closes a gap left by PR #54: the Tests-fix rubric item required a local `git stash` experiment with no substitute when the local anchor can't run, even though the same PR's UNVERIFIED handling fully anticipates that case. Without a defined fallback, Tests-fix silently degraded back to unanchored judgment on exactly the repos this finding is about (confirmed twice on `medieval-factions-dev-loop`, PRs #1976 and #1980).
 
 ---
 

create-dev-loop.md

@@ -253,6 +253,7 @@ Follow project conventions:
 Universal rules:
 - **Match sibling structure.** Before creating a new file in a directory, read the section headers / structure of every existing file in the same directory and conform to the established pattern. Example: `grep "^##" path/to/dir/*.md` for docs, or read 2–3 neighboring source files for code.
 - **Rename siblings together.** When renaming a heading or identifier that is part of a parallel pair or series (e.g. `Required X` / `Optional X`, `loadConfig` / `saveConfig`), scan for the siblings and rename them in the same commit.
+- **Scratch-file handling in a sandboxed harness.** When a step needs a scratch file for inspection or transformation (not a project source file — e.g. redirecting `git show` output for byte-level inspection, or a throwaway helper script), prefer the `Write` tool over `> file` shell redirection, and prefer `python3 -c "import os; os.remove(path)"` over `rm` to clean it up afterward. Some harness sandboxes statically block plain `>` redirection and `rm` outright — even for files the same session just created inside the working directory — while `Write` and `os.remove` are not pattern-matched the same way.
 
 Write or update tests for every change (and see Stage B in Phase 2 when the *whole cycle* is dedicated to expanding coverage of existing functionality):
 {{TEST_GUIDANCE}}
@@ -330,12 +331,17 @@ Perform a self-review. This step is anchored on external signals ({{EXTERNAL_SIG
    Universal rubric:
    - **Scope:** every file modified is necessary for one of the issues in `Closes #N` (no unrelated formatting, renames, or comment churn).
    - **Tests-new:** every new public method/function has at least one test that exercises it.
-   - **Tests-fix (empirical, not judged):** for each bug fix, temporarily revert the fix (`git stash push -- <src files>`), run the new/changed tests and confirm they **FAIL**, then `git stash pop` and confirm they **PASS**. A regression test that still passes with the fix stashed is a false-negative (common when the "after" state is indistinguishable from the "before") — use distinct/sentinel data so the failure is observable. Do not score this from reasoning alone.
+   - **Tests-fix (empirical, not judged):** for each bug fix, temporarily revert the fix (`git stash push -- <src files>`), run the new/changed tests and confirm they **FAIL**, then `git stash pop` and confirm they **PASS**. A regression test that still passes with the fix stashed is a false-negative (common when the "after" state is indistinguishable from the "before") — use distinct/sentinel data so the failure is observable. Do not score this from reasoning alone. When the local anchor cannot run (see UNVERIFIED above), use the fallback ladder below instead of skipping this item.
    - **Sibling structure:** every new file matches the section/structure conventions of its directory siblings (Phase 3 rule).
    - **Sibling renames:** every renamed identifier in a parallel pair/series has its siblings renamed in the same commit (Phase 3 rule).
    - **Docs:** every row in the Phase 7 documentation sources-of-truth table reflects the new behavior.
    - **Issue resolution:** every `Closes #N` issue's named surface area is actually changed; no issue is partially resolved while claiming closure.
    - **{{EXTERNAL_SIGNAL_LABEL}}:** the external anchor is green on the PR head (re-confirms step 1).
+
+   **Tests-fix fallback ladder when the local anchor cannot run.** The stash-and-run experiment above needs a live local anchor. CI on the fixed tree alone cannot substitute for it — CI only ever runs the *fixed* state and can never reproduce the stashed-revert half. When the UNVERIFIED case above applies, score Tests-fix using this ladder instead:
+   1. **CI-based temporary revert.** Push a commit that reverts only the production fix while keeping the new/changed tests; confirm CI goes **red** naming exactly the new regression tests; then `git reset --hard` back to the verified fix commit and force-push, confirming CI goes **green** again. Strongest available substitute — costs two CI round-trips.
+   2. **Pre-existing test changed by the fix.** If a test written before the fix asserted the old behavior and the fix's diff changes that test's assertion to the new behavior, that diff is itself a recorded FAIL→PASS — quote the test name and the changed assertion instead of re-running it.
+   3. **Neither is available.** Score Tests-fix **FAIL** and do not auto-merge — hand to a human. A bug-fix PR whose regression evidence can't be established by either rung above does not clear the regression gate.
    {{#if SELF_REVIEW_RUBRIC}}
 
    Repo-specific rubric items:
@@ -423,7 +429,7 @@ Only proceed when a complete pass finds nothing wrong.
 
 ### Phase 8 — Merge
 
-**Regression gate.** For each issue in `Closes #N` that is a bug fix or describes incorrect behavior, verify the diff includes a new or modified test (or, for projects whose external anchor is manual validation, a new validation step) that exercises the fix — and that it was confirmed empirically (the Phase 4 stash-and-run: FAIL with the fix reverted, PASS with it restored), not by reasoning alone. If absent, do not merge — either add the regression coverage or reclassify the issue. Per RESEARCH.md §3, regression evidence is the only way to distinguish a real fix from a coincidental patch.
+**Regression gate.** For each issue in `Closes #N` that is a bug fix or describes incorrect behavior, verify the diff includes a new or modified test (or, for projects whose external anchor is manual validation, a new validation step) that exercises the fix — and that it was confirmed empirically (the Phase 4 stash-and-run, or its fallback ladder when the local anchor cannot run: FAIL/RED with the fix reverted, PASS/GREEN with it restored), not by reasoning alone. If absent, do not merge — either add the regression coverage or reclassify the issue. Per RESEARCH.md §3, regression evidence is the only way to distinguish a real fix from a coincidental patch.
 
 **Do-not-auto-merge path check.** Before invoking `gh pr merge`, list the files this PR modifies and check them against the do-not-auto-merge list. If any modified path matches, do not merge automatically — leave the PR open and report to the user for manual review.