feat(dev-loop): require live behavioral proof, not just review - #1075
Merged
Merged
Conversation
a clean pr-review pass proves the diff, not the behavior. dev-loop has never run the thing it built - review just reads code and checks ci. adds create-prove: after review (and the bounded repair pass, if it ran) succeeds, it dispatches the changed skill for real at the pr's immutable head, waits for a completed run, and requires a sha-bound aeon-proof receipt before the chain can report success. no receipt, no claimed success - chain_status=proof-missing instead, excluded from skill-health's lifetime ratio the same way no-action/invalid- dispatch already are, since create-prove's v1 scope (single skills/<slug>/skill.md prs only) means most real feature prs will land here every run and that's not evidence of a regression. scoped narrowly on purpose: aeon-shaped prs only, one changed skill file, no conventional app launching, no guessing which skill represents a workflow-only change. unsupported shapes fail closed without posting a receipt. never proves itself recursively. also fixes two real gaps this needed: - codex couldn't write to aeon's notification staging dir outside the checkout (--add-dir), so proof runs that need to notify were silently missing their queued payload - bumped dev-loop's max_dispatches 4 -> 5 for the new proof step live-tested against a real repo before this was written down as working: dispatched create-prove for real, got a genuine sha-bound receipt back, confirmed the target branch head never moved.
ci caught both: README/skill-packs.md still said 82 skills, and create-prove had no eyebrowlock.json entry yet (needs the real eyebrow binary, not available in the sandbox that wrote the previous commit).
aaronjmars
added a commit
that referenced
this pull request
Sep 18, 2026
Follow-up nits from reviewing #1075/#1076/#1077 (all merged): - aeon.yml "Commit read-only failure log" step now also skips prove-* dispatches. A proof run re-runs a read-only skill on the PR's attested head branch; on a failed run this step would commit memory/logs and push a stray commit onto the very head the proof must not move. The sibling "Commit results" step already had this guard; this closes the asymmetry. - telegram-route.sh dev-loop target regex tightened from (#[0-9]+)? to (#[1-9][0-9]*)?, matching the workflow validator (dev-loop-pr.sh). The router no longer accepts #0 / zero-padded refs that the workflow would reject downstream, wasting an Actions run. Header side-effects note now mentions the chain-runner.yml dispatch path. - test_idea_pipeline_dev_loop_offer.sh: the pick: no-force-reply guard was vacuous - its sed range start pattern omitted the backticks around ${var} that SKILL.md actually has, so it captured 0 lines and could never fail. Fixed the pattern (now captures the block, still passes). - test_telegram_route.sh: added a regression case asserting a #0 issue ref is rejected at the router edge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
why
a clean review only proves the diff. nothing in dev-loop has ever launched the thing it built. review reads code and checks ci status - that's it.
what
adds
create-prove: after review (and the bounded repair pass, if one ran) comes back clean, it dispatches the changed skill for real at the pr's immutable head, waits for a completed run, and requires a sha-boundaeon-proofreceipt before the chain reports success. no receipt, no claimed success -chain_status=proof-missinginstead.proof-missingis excluded from skill-health's lifetime ratio, same asno-action/invalid-dispatch. reasoning: create-prove's v1 scope is narrow (aeon-shaped prs that change exactly oneskills/<slug>/skill.md, nothing else), so most real feature prs will land here every run, and that's not evidence of a regression.scoped on purpose:
two real gaps this needed, fixed along the way:
--add-dir), so a proof run that needed to notify silently lost its queued payloaddev-loop'smax_dispatchesbumped 4 -> 5 for the new proof stepverification
catalogs regenerated (
bin/generate-skills-json,bin/generate-packs-json) to pick up the new skill.live-tested on a real fork before this was written down as working: dispatched create-prove for real, got a genuine sha-bound receipt, confirmed the target branch head never moved during the proof run. also independently reviewed on that fork before landing here - the first pass caught a real bug (the commit-skip guard never engaged because the nested dispatch didn't set the
prove-prefix it checks for), fixed and re-verified before this port.