Skip to content

fix(enforce-tdd): stop the red step being blocked for not failing first - #52

Open
wazum wants to merge 1 commit into
nizos:mainfrom
wazum:fix/enforce-tdd-red-step-circular-block
Open

fix(enforce-tdd): stop the red step being blocked for not failing first#52
wazum wants to merge 1 commit into
nizos:mainfrom
wazum:fix/enforce-tdd-red-step-circular-block

Conversation

@wazum

@wazum wazum commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #51.

Problem

The red-phase rule that gates production code reads:

  - A test added to drive new behavior must be observed failing for
    the right reason (an assertion, not a syntax or import error) in
    a prior test run before production code may be written to satisfy
    it.

The subject is "A test added to drive new behavior". On a diff that only adds a
test, the leading clause matches and the trailing "before production code may be
written" gets lost, so the validator blocks the write that creates the test. That
is impossible to satisfy: the test cannot fail before it exists, and retrying
gives the same block, so the cycle deadlocks.

The trigger is a deletion followed by a green run (an obsolete test removed
before its replacement is written). No failing run is in the window, and no
completed red->green cycle anchors the judgment.

Change

Rephrase so the subject is the production write, and name the circular case:

  - Production code that satisfies a new test may not be written until
    that test has been observed failing for the right reason (an
    assertion, not a syntax or import error) in a prior test run. This
    gates the production write, never the write that adds the test: a
    test cannot fail before it exists, so "no failing run yet" is never
    a reason to block adding one.

The gate on production code is unchanged.

Evidence

Replaying a captured hook payload (same transcript window, same file before and
after), 8 runs per variant:

variant denied
released prompt 4/8
only this bullet reworded 0/8

Tests

Adds an integration test pinning the case: an obsolete test is deleted, the suite
runs green, then one new test is added. Fixtures are generic (renderBanner).

npm run checks passes. The AI suite passes, including the deny cases. Note the
refactor-enforcement deny test is flaky independently of this change — 3/5 on
main and 4/5 with the change in my runs.

Because the misfire is probabilistic and current main already allows this
scenario in my sampling, the new test pins the behavior rather than failing
before the change. Happy to reshape it if you'd prefer a different form.

The red-phase rule that gates production code on a prior failing run was
phrased with the test as its subject: "A test added to drive new behavior
must be observed failing ... before production code may be written to
satisfy it." Read on a diff that only adds a test, the leading clause
matches and the trailing qualifier gets lost, so the validator blocks the
very write that would create the test. The demand is impossible: the test
cannot fail before it exists, and retrying reproduces the block, so the
cycle deadlocks.

The session shape that triggers it is a deletion followed by a green run
(an obsolete test removed before its replacement is written). No failing
run sits in the window and no completed red->green cycle anchors the
judgment, so the misread has nothing to correct it.

Rephrase the rule so its subject is the production write, and state the
circular case explicitly: a test cannot fail before it exists, so "no
failing run yet" is never a reason to block adding one. The gate on
production code is unchanged.

Replaying a captured hook payload 8 times per variant: 4/8 denied on the
released prompt, 0/8 with only this bullet reworded. Adds an integration
test pinning the case (new test written after a deletion, last run green).

Closes nizos#51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

enforceTdd can block adding a test, asking it to fail before it exists

1 participant