Harden agent runs: work-practice policy, actionable headless block, unhelped SWE-bench measurement / 加固 agent 运行：工作实践策略、可行动的 headless 拦截、不加特殊照顾的 SWE-bench 测量

[SivanCola]

Summary

Three small, independently reviewable hardening changes, each grounded in failures observed in real SWE-bench Verified runs (50-instance subset, official images, official harness) — and each one applies to ordinary headless/interactive runs just as much as to the benchmark.

The benchmark is an instrument, not a target. It exists to read what the shipped software actually does, so nothing here is tuned for the score:

• The container runs a default install. No benchmark-only configuration, no environment hints, nothing that helps the agent in a way a user's own machine would not. Pinned by a test.
• The harness does not tidy up after the agent. Whatever the run left behind goes into the submitted diff.
• A failure only earns a product change if it would also bite a user on their own machine. Where a failure turned out to be a property of the benchmark container rather than of agents in general, it is not written into the shipped persona — see change 1.

A consequence worth stating plainly: some of this is expected to lower the measured score relative to earlier runs, because earlier runs were quietly helped. That is the point — the lower number is the honest one.

1. feat(agent): work-practice policy in the stable system prompt, environment facts kept out of it

The default system prompt is three lines of persona. Runs repeatedly showed the same avoidable behavior failures: substituting a "better" approach for the one the issue explicitly named (the agent's own tests all passed; the requested behavior was never delivered), leaving repro/probe files and build output in the repo, and declaring done with known regressions.

Adds config.WorkPracticePolicy, appended to every system prompt next to UserDecisionPolicy (so custom personas cannot drop it either): implement the requested approach exactly and state disagreements explicitly; keep scratch out of the repo and review the final diff before declaring done; scale verification to the change.

Every clause in the global policy has to hold for any user on any machine. Two candidate clauses did not, and are deliberately not in it:

• "avoid repeated full-suite runs; if broad failures look unrelated, confirm once that they pre-exist and move on" — that is a 30-minute-per-instance run budget talking. Real users more often complain about the opposite, and the clause sits badly next to "no known regressions" in the same policy.
• "if network calls fail with proxy errors, treat the environment as offline and stop retrying" — true in a container with no off-box route, wrong on a flaky laptop network where one retry succeeds.

Both are properties of where the agent is running, not of good engineering. The second becomes [environment] offline, a declaration an air-gapped deployment makes for itself, rendered as config.OfflineEnvironmentNote next to the environment block. It is deliberately a declaration and not a probe: testing connectivity on every boot would cost a request, leak the startup, and let one transient flap rewrite the cache-stable prompt prefix. It sits outside EnvironmentEnabled because that switch only turns off tool probing, while a blocked network is a fact the agent still needs. The run budget gets no prompt clause at all — that belongs to the harness (timeouts, step caps).

The SWE-bench containers do not set it; see change 3.

Cache impact (deliberate): the stable prefix changes once at upgrade; per-turn cache stability is unaffected. The offline note is off by default, so it never enters an ordinary user's prefix. Golden provider-request / prefix-shape baselines regenerated via REASONIX_UPDATE_GOLDEN=1.

2. fix(control): actionable headless dynamic-shell block

In non-interactive auto mode, a blocked python -c / node -e told the agent only to "use an interactive session or YOLO mode" — impossible in a headless run. Agents burned dozens of calls retrying inline-code variants (88 blocked retries in one benchmark arm). The gate stays exactly as strict (inline code is not auditable); the message now names the in-session workaround: write the code to a file with write_file and run that file, or use read_file/grep for inspection.

This one is not benchmark-specific in any way: the same gate serves ordinary -p runs (internal/control/controller.go) and desktop subagents (desktop/subagents_app.go), which hit the identical dead end. The benchmark only made it countable.

3. fix(e2ebench): keep unappliable binaries out of the SWE-bench patch — and nothing else

A submission was zeroed when a Sphinx repro build left a binary .pickle in /testbed: git add -A + git diff --cached degraded it to a Binary files differ placeholder that git apply rejects wholesale, so the correct source fix in the same patch never reached the grader. That is a defect in the submission format rather than in the agent, and repairing it is the only reason the extractor exists. Paths are passed as plain argv, never through a shell.

Binary entries are the only thing dropped. Build output, __pycache__, .egg-info and leftover repro scripts all go into the patch as-is: they apply cleanly and do not change the grade, so filtering them would only hide the agent's mess from its own diff and report a cleanliness the run did not have. (An earlier revision of this PR dropped them by name; that was harness cleanup and has been removed.)

The container config also drops [environment] offline = true for the same reason. Egress genuinely is blocked — runSwebench requires -network with no off-box route — but a real user whose network is blocked does not know to set that flag, so declaring it measured a better-configured agent than the one that ships. An agent that burns its budget retrying dead requests is a real weakness; the benchmark's job is to surface it, and the fix belongs in the product where it helps everyone. [sandbox] bash = "off" stays: bubblewrap is absent from the official images and Reasonix would otherwise refuse to run bash at all, so it is a necessity, not a favor.

Two consequences for reading results: scores from before the binary fix are not comparable with scores after it (and the bias it removes is uneven — repos whose builds emit artifacts were penalized far more than pure-Python libraries), and any movement in a run that also carries change 1 must not be attributed to behavior alone.

Verification

• New tests: work-practice policy appended to custom prompts with deterministic ordering; the global policy may not assert offline/proxy/full-suite/pre-exist claims (pins the split so environment facts cannot drift back into the persona); the offline note is absent by default, present when declared, and survives enabled = false while the probed section does not; gate-level test pins python -c still blocked, workaround named, and script-file execution actually allowed by the same gate; the SWE-bench container config decoded with the real config.Config and asserted to carry no setting other than sandbox.bash; patch keep/drop classification incl. binaries dropped, generated-looking text kept, and malformed numstat.
• go build ./..., gofmt, and go test ./... (root module) green apart from three failures that reproduce identically on the base commit in this container: two depend on chmod-based unwritability (bypassed when running as root) and one needs an X display.

Not changed, deliberately

• No loosening of the inline-interpreter permission gate.
• No read-only repeat guard: an existing pinned test explicitly allows repeated non-writing bash commands (re-running tests is the core agent workflow).
• No run-budget guidance in the prompt: budget enforcement belongs to the harness, not to a clause every user reads.
• No connectivity probe: offline is declared by the deployment, never detected at boot.
• No benchmark-only configuration or post-run tidying, per the principle above.
• No per-turn update_goal tool gating: goal turns are stamped per-turn within ordinary sessions, so per-turn tool-list changes would break the prompt-cache prefix on every transition; the existing absorb-and-repair path already bounds the cost of a stray call.

CI Metadata

Documentation-impact: updated - documented [environment].offline in reasonix.example.toml and docs/GUIDE.md, docs/GUIDE.zh-CN.md, and docs/SPEC.md.
Cache-impact: medium - WorkPracticePolicy changes the stable system prompt once; the optional offline note is excluded by default; per-turn prefix stability is unchanged and golden baselines are updated.
Cache-guard: focused prompt/config tests in internal/boot and internal/config; go test ./internal/boot ./internal/config; go run ./tools/repolint on the PR merge tree.
System-prompt-review: self-review completed; policy ordering and custom/offline prompt isolation are covered by tests.