fix(tools): resolve workflow:<id> before image vision gates and self-heal wedged replays - #2412
Conversation
…heal wedged replays
## Summary
- `browser_screenshot_handler` and `_read_image` keyed their `supports_vision`
inline/degrade gate on the raw model string from `get_session_model`, without
resolving `workflow:<id>` bindings to their declared `output_model` (as the
loop does via `_resolve_capability_model`). `supports_vision("workflow:…")`
→ `litellm.get_model_info` raises → `None` → the post-551ce0fd `is False`
gate fell through and inlined the image, wedging any workflow-bound session
whose inner model is text-only (the frozen `image_url` tool result 400s on
every replay wake, latching terminal `errored`). Both handlers now resolve
the binding via `_resolve_capability_model` (function-local import to avoid
the browser → loop → tools → browser module-load cycle) before the gate.
- Added a build-time safety net `_strip_image_parts_for_non_vision_model` in
`context.build_messages`, sibling to `_clamp_oversize_image_data_urls` /
`_correct_image_data_url_mimes`: it downgrades any persisted `image_url`
part to a text marker when `supports_vision(resolved_model) is False`, so
already-wedged sessions self-heal on the next wake. `None`/True are left
untouched to preserve the optimistic-inline policy (a genuinely
vision-capable uncatalogued/model-unresolved bound model is not silently
stripped). Replaces part dicts with fresh copies, never mutates the
aliasing `Event.data` in place (the immutability contract).
## Test plan
- [x] `uv run mypy src tests` (1084 files, clean)
- [x] `ruff check src tests` + `ruff format --check src tests` (clean)
- [x] unit: `test_browser_tools.py::TestScreenshot` (10) + `test_read_image.py::TestImageBranch`
(20) + `test_context.py` replay-pass/immutability/lazy-litellm (8) +
`test_model_binding.py` (27) — workflow-binding resolution mocks the
resolver; the build-time pass exercises text-only strip / vision keep /
unknown keep / no-model / any-role / multi-image / immutability.
- [x] integration: new `test_workflow_screenshot_vision_gate.py` (3) drives
the REAL `_resolve_capability_model` DB lookup against a testcontainer
Postgres — text-only `output_model` degrades, vision inlines, and a
persisted `image_url` tool result is stripped by the real
`compose_step_context` on the next build. Pre-existing
`test_model_workflow_park_sweep.py` (4) still green.
- [x] e2e (live Docker sandbox): `test_workspace_image_read.py` +
`test_extensionless_image_read.py` (2) — read-image inline path unchanged.
- [ ] browser e2e (`test_browser_image_contract.py` /
`test_browser_driver_behavior.py`): the `aios-browser:dev` image builds and
`docker run` works, but the host cgroup `/sys/fs/cgroup/user/workload` is in
`domain threaded` mode, so Docker cannot apply the resource-limit
controllers (`--cpus`/`--memory`/`--pids-limit`) that
`start_browser_container` hardcodes; 4 metadata tests pass, the rest are
blocked by that host restriction (not a fix defect). CI's docker shard runs
these green.
Co-authored-by: Detail <detail@aios.local>
Code reviewVerdict: pass. The diagnosis is correct, the fix is at the right two layers (trigger + backlog), and the regression tests are genuinely load-bearing. Three non-blocking observations below; none of them blocks the merge. Scope verified
What I checked and howThe trigger fix is correct. Both handlers now route through The tests are real, not tautological. I reverted the three Focused runs on head, all green (repo-wide suites left to CI):
The build-time pass respects the invariants it claims to. It replaces part dicts with fresh copies and rebuilds the message via No new litellm import cost. I initially suspected the unconditional The self-heal reaches the wedged sessions it claims to. Non-blocking observations
ConventionsDocstrings match the density and the "state the failure mode and why this posture" style of the surrounding passes; the function-local imports carry the cycle rationale inline, as the neighbouring lazy imports do; the integration test's |
|
@cursor update to latest master |
|
This branch is already on latest
PR #2412 already targets that master SHA. CI is running on the merge commit (lint, unit, integration, both e2e shards, plus eumemic-bot review). I did not push or change the PR. |
…id-before-image-vision-c8e05e
Code reviewVerdict: pass. This head ( Scope verified
7 files, +944/−1. What I checked and howThe trigger diagnosis holds. Both gates test The tests are load-bearing, not tautological. I reverted only the three Focused runs on the merged head (repo-wide suites left to CI per scope):
The build-time pass respects the invariants it claims. It never mutates in place — it copies the part list, replaces the offending dicts, and rebuilds the message via The self-heal reaches the wedged sessions. Perf. Unevaluable, and why it does not gate the verdict. The new integration test Non-blocking observationsThese carry forward from the prior round unchanged — the diff did not change, and I re-confirmed each still reads as described. None blocks the merge.
ConventionsDocstrings match the surrounding "state the failure mode and the reason for this posture" density; the function-local imports carry their cycle rationale inline as neighbouring lazy imports do; the integration fixture's |
…id-before-image-vision-c8e05e
Code reviewVerdict: pass. Head Scope verified
7 files, +944/−1 vs. base. What I checked and howThe trigger diagnosis holds on this tree. The tests are load-bearing, not tautological. I reverted only the three Focused runs on the merged head (repo-wide suites left to CI per scope):
The build-time pass respects the invariants it claims. It never mutates in place — it copies the part list, replaces the offending dicts, and rebuilds the message via Degenerate shape checked directly. I ran the case the unit tests do not cover — an image-only tool result with no sibling text part — against the real The self-heal reaches the wedged sessions. Unevaluable, and why it does not gate the verdict. The new integration test Standing properties re-checkedBoth carry forward from the prior rounds; I re-verified each against this head rather than inheriting the earlier assessment. Neither is newly violated, and neither blocks.
ConventionsDocstrings match the surrounding "state the failure mode and the reason for this posture" density; the function-local imports carry their cycle rationale inline as neighbouring lazy imports do; the integration fixture's |
…id-before-image-vision-c8e05e
|
Labelled Review state: three PASS verdicts, none inheritedWhat makes this PR unusually well-reviewed is that the reviewer refused to inherit its own prior verdicts across master merges:
That is the correct discipline: a green earned against an older tree is a stale green, and the reviewer explicitly re-derived rather than assuming the merge was inert. The gap I found, stated preciselyThe live head is I checked what that unreviewed merge actually contains — The PR itself touches 7 files: Overlap: NONE. The unreviewed merge and the PR modify entirely disjoint file sets. What that does and does not establishIt is good evidence the verdict carries — the same reasoning the reviewer applied at rounds 2 and 3, and it held both times. But I want to be exact about its strength: "no file overlap" is weaker than "re-verified against this tree." Disjoint files can still interact through shared behaviour, import-time effects, or a changed config default. So this is not a green at the live head; it is a strong prior that the green survives. Under the standing rule, merge requires an uncorrelated review green at the live head and CI green at that head. The honest state is: two rounds of evidence say the merge is inert, and nobody has actually checked. RecommendationCheapest correct path is a re-verify of Both facts point the same way: one more verification round at whatever head the rebase produces, then merge. Blocked on the aios#2396 freeze until then. |
…id-before-image-vision-c8e05e
Code reviewVerdict: pass. Head Scope verified
7 files, +944/−1 vs. base. The triage comment correctly flagged that "disjoint files" is weaker evidence than re-verification, and named What I checked and howThe trigger diagnosis holds on this tree. Both gates test The tests are load-bearing, not tautological. I reverted only the three Focused runs on this head (repo-wide suites left to CI per scope):
Working tree restored to a clean The build-time pass respects the invariants it claims. It copies the part list, replaces the offending dicts, and rebuilds the message via Degenerate shape re-checked directly on this tree. An image-only tool result with no sibling text part — not covered by the unit tests — yields The self-heal reaches the wedged sessions. Unevaluable, and why it does not gate the verdict. Standing properties re-checkedAll three carry forward from earlier rounds. Re-verified against this head rather than inherited; none newly violated, none blocking.
ConventionsDocstrings match the surrounding "state the failure mode and the reason for this posture" density; the function-local imports carry their cycle rationale inline as neighbouring lazy imports do; the integration fixture's |


Detail bug report: View on Detail
Summary
browser_screenshot_handlerand_read_imagegated image inlining onsupports_vision(model), but passed the raw model string fromget_session_model— includingworkflow:<id>bindings — without resolving the binding to its declaredoutput_model(the loop's capability gates do this via_resolve_capability_model).supports_vision("workflow:…")callslitellm.get_model_info, which raises → returnsNone; after 551ce0f the gate only degrades onis False, soNonefell through and inlined the screenshot. For a workflow-bound session whose inner model is text-only, the replayedimage_urltool result 400s every wake, latching the session terminalerroredwith no retry/strip._resolve_capability_modelbefore the vision gate (function-local import, avoiding thebrowser → loop → tools → browsermodule-load cycle), so a catalogued text-only inner model degrades to a text marker and a vision-capable one inlines. Added a build-time safety net_strip_image_parts_for_non_vision_modelinbuild_messages, sibling to the oversize-clamp / mime-correction passes: it downgrades any persistedimage_urlpart to a text marker whensupports_vision(resolved_model) is False, self-healing already-wedged sessions on the next wake.None/True are left untouched to preserve the optimistic-inline policy.Substrate state changes
None — code-only change.
Test plan
uv run mypy src testsclean on 1084 files;ruff check/ruff format --checkclean). Added regression coverage:test_browser_tools.py::TestScreenshot— workflow binding with text-only inner model degrades; vision-capable inlines; unresolved/unknown inlines optimistically; raw model passes through the resolver unchanged.test_read_image.py::TestImageBranch— same four shapes for the read path.test_context.py::TestVisionCapabilityReplayPass— text-only strips persistedimage_url(incl. any-role, multi-image, ordering preserved, immutability ofEvent.data); vision/unknown/no-model keep the part (optimistic-inline policy preserved).test_workflow_screenshot_vision_gate.py, testcontainer Postgres): drives the real_resolve_capability_modelDB lookup — text-onlyoutput_modeldegrades the screenshot, visionoutput_modelinlines it, and a persistedimage_urltool-result event is stripped by the realcompose_step_contexton the next build. Pre-existingtest_model_workflow_park_sweep.pystill green.test_workspace_image_read.py+test_extensionless_image_read.pypass — read-image inline path unchanged.test_browser_image_contract.py,test_browser_driver_behavior.py) could not be fully run: theaios-browser:devimage builds and a plaindocker runworks, but the host cgroup/sys/fs/cgroup/user/workloadis indomain threadedmode, so Docker cannot apply the resource-limit controllers (--cpus/--memory/--pids-limit) thatstart_browser_containerhardcodes to mirror production. 4 image-metadata tests pass; the rest are blocked by that host restriction (not a fix defect). CI's docker shard runs these green.Risk / rollback
Low-risk, additive. The trigger fix only changes behavior for
workflow:<id>bindings (the raw-passthrough path is unchanged). The new build-time pass only acts on an explicitsupports_vision is False(vision/unknown/no-model are untouched), so it cannot strip a legitimate image from a vision-capable or uncatalogued model. If a regression appears, revert the commit — no schema/env/migration changes are involved.Automatic Fixes PRs can be configured here.