Repository navigation
ci: guard the edit-accuracy gate against a failed changes job - #5182
Conversation
|
Re-verified the premise today against the live rulesets API: Worth stating the mechanism without reference to any tool, because GitHub documents all of it. From Handling skipped but required checks, three cases with three different outcomes:
The Without it, a failed So the failure needs no unusual conditions. The six sibling required checks already encode exactly this, each pairing One thing I should correct from the description above: it says the finding came from deadgate, and the detector that produced it has since been removed from that tool. I measured its output across 275 repositories and about 80% of what it reported was redundant with GitHub's needs-skip rule, the third row in the table, where the dependent skips anyway and the claim adds nothing. Deleting it was the right call on volume. This repository is in the remainder, and |
miguel-heygen
left a comment
There was a problem hiding this comment.
Thanks for the careful write-up. The hole is still on main: "Studio: edit accuracy gate" is still a required check, and its condition is still !cancelled() && needs.changes.outputs.edit_accuracy == 'true' with no needs.changes.result guard. A failed changes job (for example the accidental-deletion check) makes the job skip on its own if:, and that counts as a pass for a required check. Build and the other required jobs already use the needs.changes.result != 'success' || guard.
I also checked what the gate does once your guard lets it run after a failed changes. The shard jobs still skip, so no results are downloaded, ratchet.mjs gate reports "0 of N shards reported results", and the final step fails the job. So it fails closed, which is what you want.
This needs a rebase. ci.yml conflicts because main has since reworked this job: it now also needs the retry job, and the github.event_name == 'pull_request' clause is gone (the gate runs in the merge queue too). On current main the condition would become:
if: ${{ !cancelled() && (needs.changes.result != 'success' || needs.changes.outputs.edit_accuracy == 'true') }}The commit also needs signing, since main requires signed commits. CI has not run on this fork branch yet, so there is no CI result to go on. Happy to see this approved once it is rebased.
Verdict: COMMENT
Reasoning: The fix is correct and still needed, but the branch conflicts with the reworked job on main and has no CI run yet.
6f2fc62 to
f180ffc
Compare
|
Thanks, Miguel. Rebased onto current main with your condition exactly as written, and the commit is now signed. Sorry for the brief close and reopen while I reset the branch. CI on the fork needs a maintainer's approval to run; ready whenever you are. |
miguel-heygen
left a comment
There was a problem hiding this comment.
Thanks for the rebase. This head is exactly the condition from my last review, on top of current main, and the commit is signed. The diff against main is now just the one if: line on "Studio: edit accuracy gate", and the gate still fails closed when changes fails (no shard results, so the final step fails the job).
CI hasn't run on this fork branch yet; it needs a maintainer to approve the workflow run before the required checks can report.
Reviewed head 3c98c1c8.
Verdict: COMMENT
Reasoning: The rebased one-line guard matches the pattern the other required jobs use and closes the skipped-equals-success hole; I'll approve once the workflow run is approved and green.
jrusso1020
left a comment
There was a problem hiding this comment.
Approved. The edit-accuracy gate now runs when change detection fails, so a failed changes job can't silently satisfy the required check. Unrelated changes and cancelled runs behave as before, and a zero-report run exits 2 and is rejected by the final step.
jrusso1020 merged it through the merge queue on 2026-10-26 00:58 UTC, after approving it. The résumé's deadgate line now ends ", fixed by my merged PR #5182", and the PDF is rebuilt from the same print template as 0aa0337: a word diff against the previous PDF shows only that clause added, still 2 pages, US Letter, no phone. The findings entry's status reads MERGED, the same markup findings-ledger's generate.py emits for it now. findings/ was not regenerated from findings-ledger: the deployed page carries corrections the ledger source does not have yet (the ruff outcome, corrected test counts, the light-palette scoping, the Orchestration link), and a full regeneration would undo them.
Studio: edit accuracy gateis a required status check onmain, and it is the only gated required check without aneeds.changes.resultguard.If the
changesjob fails, its outputs are unset,needs.changes.outputs.edit_accuracy == 'true'is false, the job skips, and GitHub reports a skipped job as Success. The required check is satisfied with nothing run.changescan fail. It runsnode scripts/check-no-main-deletions.mjs --base origin/main, which exists to fail.Six sibling required checks already handle exactly this:
This applies the same guard to the edit-accuracy gate. When
changessucceeds the condition is unchanged, so there is no difference on the happy path; the only new behaviour is that a failedchangesno longer silently satisfies the gate.Found with deadgate, which reads workflow files for checks that structurally cannot fail and then resolves each one against the branch's required checks. It flags other jobs here on the same shape, but per the public rulesets API those are not required checks, so they are hygiene rather than a merge-gate hole and I have left them out of this PR.