You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The sandbox CLI blocks plain sleep waits and its guidance message steers the agent toward run_in_background: true. In interactive Claude Code, that workflow ends with the harness re-invoking the agent when the task completes. Under our driver the CLI runs non-interactively: when the model ends a turn with no tool call, the session is treated as complete. An agent that backgrounds a test and ends its turn "waiting for the notification" silently ends its own run with budget unspent, and the driver logs a normal completion (error=False).
Observed
Two Sonnet 5 ClutteredStorage runs (2026-08-05, cluster):
sonnet5_clutteredstorage_2026-08-05/bilevel/.../s42: 83 turns, $3.80 of $20. Last agent messages: no-op Bash("true") calls, then "I'll stop issuing commands and wait for the background task notification", then Apptainer session done.
sonnet5_clutteredstorage_2026-08-05/noprims/.../s42: 277 turns, $13.00 of $20, same pattern ("I'll pause here and resume analysis when that notification arrives").
Opus runs in the same campaign avoided it by waiting inside a blocking Bash poll (until grep -q ... ; do sleep 10; done), which keeps the turn alive. The failure is model-behavior-dependent, so it can silently skew model comparisons: affected runs look like voluntary early stops.
Possible fixes
Prompt line: background-task notifications are not delivered in this environment; wait for backgrounded commands with a blocking until loop in a single Bash call.
Driver-side safety net: on session end with unspent budget and a live background task in the sandbox, re-invoke with a continuation prompt carrying the task output (the resume machinery from Resume Claude Code sessions after retryable interruptions #123 already does the re-invocation half).
Disable run_in_background in the sandbox CLI settings.
Option 1 is cheap and immediate; option 2 is the robust fix; option 3 loses genuinely useful parallelism (agents run long test sweeps in the background while editing).
What happens
The sandbox CLI blocks plain
sleepwaits and its guidance message steers the agent towardrun_in_background: true. In interactive Claude Code, that workflow ends with the harness re-invoking the agent when the task completes. Under our driver the CLI runs non-interactively: when the model ends a turn with no tool call, the session is treated as complete. An agent that backgrounds a test and ends its turn "waiting for the notification" silently ends its own run with budget unspent, and the driver logs a normal completion (error=False).Observed
Two Sonnet 5 ClutteredStorage runs (2026-08-05, cluster):
sonnet5_clutteredstorage_2026-08-05/bilevel/.../s42: 83 turns, $3.80 of $20. Last agent messages: no-opBash("true")calls, then "I'll stop issuing commands and wait for the background task notification", thenApptainer session done.sonnet5_clutteredstorage_2026-08-05/noprims/.../s42: 277 turns, $13.00 of $20, same pattern ("I'll pause here and resume analysis when that notification arrives").Opus runs in the same campaign avoided it by waiting inside a blocking Bash poll (
until grep -q ... ; do sleep 10; done), which keeps the turn alive. The failure is model-behavior-dependent, so it can silently skew model comparisons: affected runs look like voluntary early stops.Possible fixes
untilloop in a single Bash call.run_in_backgroundin the sandbox CLI settings.Option 1 is cheap and immediate; option 2 is the robust fix; option 3 loses genuinely useful parallelism (agents run long test sweeps in the background while editing).