Skip to content

fix(triage): say why, where and which log in BOT_FAILED comments - #5430

Merged
springfall2008 merged 1 commit into
mainfrom
tools/triage-failure-reasons
Oct 9, 2026
Merged

springfall2008 merged 1 commit into
mainfrom
tools/triage-failure-reasons

Conversation

@chalfontchubby

Copy link
Copy Markdown
Collaborator

Posted by Claude on behalf of @chalfontchubby.

In short

When the triage bot gives up and adds BOT_FAILED, the comment now says why it failed, where in the daemon it failed, and which log file to open. Before, the issue-side comments only said "see the triage bot's logs", and those logs live on the bot's host, so nobody else could tell a spent turn budget from a crash.

What a failure comment now carries

All four paths (first-pass triage, issue follow-up, PR review, PR cleanup):

  • Reason: a classified cause. From the run's log: out of turns, spend limit, out of credit or at the usage limit, expired login, rate limited, API overloaded, network lost. Otherwise: killed by a signal (named), a failed gh/git subcommand (leading words only, never the arguments, since a comment body can be one), or the exit status.
  • Failed at: triage_daemon.py:<line> in <function>() plus the daemon's short commit, so the line can be read against the right file. It is the innermost frame in this file, not the subprocess module's frame beneath a check=True. For a run that exited 0 and did nothing there is no traceback, so it names the process_* function that noticed.
  • Log: the log file's name, not its path.

The log is only classified, never quoted, because it can hold a reporter's attachment or a token.

How

  • FlowFailed is a CalledProcessError that also carries the log path. triage(), triage_followup(), review_pr() and cleanup_pr() raise it, so the cause can be read from the run that failed.
  • FAILURE_LOG_SIGNATURES is matched only against the last 4000 characters after the final started marker. The log is appended to across retries, and the agent quotes the issue it is reading, so an issue about "Reached max turns" must not classify its own triage.
  • The four mark_*_failed() functions take exc=, and the callers pass the exception they caught.

Known limits

  • The Claude Code strings in FAILURE_LOG_SIGNATURES are from memory and have not been checked against a real failed log. A wrong or stale entry gives the less specific exit-status message, never a wrong label. Worth checking against a real file in ~/predbat-triage-bot/logs/.
  • An issue triage that exits 0 having posted nothing is still not detected (PR review catches this by counting comments before and after). Adding that is a behaviour change, so it is left out here.

Testing

  • 325 daemon unit tests, including each signature, the fallback, last-attempt-only classification, a signature quoted mid-run, a missing log, signals, gh arguments not leaking, the origin frame for both a FlowFailed and a check=True failure, the clean-exit origin, and all four comments.
  • coverage/run_pre_commit passes.

🤖 Generated with Claude Code

The issue-side failure comments only said "see the triage bot's logs",
and the PR-side ones did so for any failure without a hand-written
reason. The logs live on the bot's host, so nobody else could tell a
spent turn budget from a crashed run.

Each of the four failure comments now carries:
- Reason: a classified cause - out of turns, spend limit, credit or
  usage limit, expired login, rate limit, overloaded API, network loss,
  a signal, a failed gh/git subcommand, else the exit status. The log
  is only classified, never quoted, and only its last few thousand
  characters of the final attempt are searched.
- Failed at: file, line and function in triage_daemon.py where the
  failure arose, with the daemon's short commit.
- Log: the log file's name, not its path.

The claude runs raise FlowFailed, a CalledProcessError that carries the
log path, so the cause can be read from the run that failed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@chalfontchubby chalfontchubby added the enhancement New feature or request label Oct 7, 2026
@springfall2008
springfall2008 merged commit 6c57d83 into main Oct 9, 2026
2 checks passed
@springfall2008
springfall2008 deleted the tools/triage-failure-reasons branch October 9, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants