Repository navigation
Tell agents not to quote signal markers - #467
Merged
Merged
Conversation
detectSignal matches a marker anywhere in the output, so an agent that writes the full marker inside a sentence explaining it is not sending it sends the signal anyway. In custom_eval.txt this ended the external review loop with the fixes uncommitted and never re-reviewed. each prompt that defines signals now says a marker takes effect wherever it appears, and that a signal is referred to by its bare name (REVIEW_DONE), never the full marker. review_first.txt and review_second.txt extend their existing irreversible-marker paragraph; codex.txt, custom_eval.txt and task.txt get a short SIGNALS paragraph. This is prompt mitigation, consistent with how #92 and #293 were handled; the detection contract is unchanged. Related to #466
golangci-lint 2.15.0 reports the discarded receives from errCh in the dashboard port-conflict test and from done in the watcher close test as unchecked errors. Both channels deliver nil on these paths, so the tests now assert it. CI pins 2.13.0, which does not flag them.
review_first.txt and review_second.txt define both REVIEW_DONE and TASK_FAILED, but the bare-name rule gave only REVIEW_DONE as its example. A quoted TASK_FAILED marker is detected the same way and would fail the review, so the rule now names both, matching task.txt. Related to #466
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
an agent explaining why it is not sending a signal can quote the full
<<<RALPHEX:...>>>marker in that sentence.detectSignalmatches a marker anywhere in the output, so the quote counts as sent. In #466 this ended the external review loop throughcustom_eval.txt, with the fixes uncommitted and never re-reviewed.the fix is prompt only, same as #92 and #293. Every prompt that defines signals now says a marker takes effect wherever it appears, and that a signal is referred to by its bare name (
CODEX_REVIEW_DONE), never the full marker:review_first.txtandreview_second.txtextend the existing irreversible-marker paragraph, namingREVIEW_DONEandTASK_FAILEDcodex.txt,custom_eval.txtandtask.txtget a short SIGNALS paragraphthe wording proposed in #466 said never to name a signal at all. The bare name gives the agent something safe to write instead, and detection never matches it.
make_plan.txtis left out since no plan-mode case was reported; it is in the backlog.also in this branch:
docs/custom-providers.md, and themake_plan.txtfollow-upRelated to #466