Repository navigation
fix(skill-health): detect critical-incident recovery without waiting on lifetime success_rate - #1018
Merged
aaronjmars merged 2 commits intoSep 3, 2026
Conversation
…on lifetime success_rate a critical issue's HEALTHY-driven auto-resolve only fires once the skill's classification clears CRITICAL/DEGRADED/WARNING, which is gated on lifetime success_rate — so a skill that recovers with several straight clean runs can sit "still unhealthy" and its issue stays open for a long stretch of history to age out of the average, even though the specific incident is long over. add skill_health_recovery.py: a small, fail-closed, independently tested check — a run newer than the issue's detected_at with last_status=success and consecutive_failures=0 proves that incident recovered, regardless of what the lifetime average says. wired into skill-health's issue-resolution step, scoped narrowly: only status: open critical issues (flapping/high still need real flap-free time to prove recovery, and fix-pending issues already have their own reconcile step above this one). Claude-Session: https://claude.ai/code/session_01SbipDt7VmuDvosRPS8AfEx
skill_health_recovery.py fails closed (prints active) on a naive detected_at, silently reverting to the slow lifetime success_rate path. The issue template gave no timezone hint, so an agent-written naive stamp would no-op the recovery shortcut. Pin the template to UTC Z. Co-Authored-By: Svector-anu <svector.anu@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
what
a critical skill-health issue only auto-resolves once the skill's classification clears CRITICAL/DEGRADED/WARNING — which is gated on lifetime
success_rate. so a skill that recovers cleanly (several straight successful runs) can still sit "unhealthy" and its issue can stay open for a long stretch of history to age out of the average, even though the specific incident is long over.fix
scripts/skill_health_recovery.py: a small, independently-tested, fail-closed check. a run newer than the issue'sdetected_at, withlast_status=successandconsecutive_failures=0, proves that specific incident recovered — regardless of what the lifetime average still says.wired into skill-health's issue-resolution step, deliberately scoped narrow:
status: opencritical issuesstatus: fix-pendingissues — those already have their own reconcile step (the fix-pending → merged/reopened logic from don't mark an issue resolved when the repair PR is only opened #997)invalid or missing state/timestamps fail closed (
active, never a false recovery).tests
scripts/tests/test_skill_health_recovery.py— 5 cases: recovery ignores lifetime success_rate, a success before the incident'sdetected_atdoesn't resolve it, an in-flightdispatchedstatus waits for the actual outcome, invalid/missing timestamps fail closed, and the CLI's stdin→stdout contract. wired intoci-tests.ymlalongside the other health-suite steps.https://claude.ai/code/session_01SbipDt7VmuDvosRPS8AfEx