You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b31eecc
Browse filesBrowse the repository at this point in the historyBrowse files
fix(node): an attach replays the agent's stderr past the line it last read (#977)
## Summary
Follow-up to LivXue's review note on #957 (the pick-up's binding at
`cli_agent.py:1393`). The machine's daemon kept an agent's stderr in the
task's record (`stderr.log`) but sent it only to the connection holding
the
task when a line was written, and `agent/attach` replayed `frames.jsonl`
alone. So what an agent said only on stderr while no connection held it
never
reached the host:
- **The session id a CLI names only on stderr** (hermes does). The
pick-up
#957 added binds a create's handle to the id in its output; for such a
CLI
it found none, warned "not resumable", and the next turn to the instance
opened a new session.
- **The reason a command failed.** A run picked up after this computer
stopped was reported as `exited 3` where the dispatch would have said
`exited 3: token expired`. This part predates #957.
- **The stderr written while a link was down**, on a run still being
watched, the same way.
The fix is in the node and the host's link to it (`raven/node/`):
- The daemon numbers stderr lines as it numbers frames (`agent/stderr`
carries `n`), and `agent/attach` takes an optional `stderr_after`: the
stderr lines past that one are replayed after the frames, under the lock
that forwards a new line, so none is sent twice or missed at the
handover.
It works the same for a live task, a finished one, and a record a daemon
that never ran the task reads. An attach that does not ask hears no
stderr,
as before; a `stderr_after` that is not a whole number is refused like a
bad `after`.
- The host's `ChannelEndpoint` counts the stderr lines it got
(`stderr_received`). A cli pick-up (`NodePool.attach`, a process that
read
none of them) asks from the first line; an acp pick-up from the line the
run's record kept as the prompt went out (`RemoteRun.stderr_received`,
follow-up commit below); a re-attach after a lost link
(`NodePool._reattach`) asks from the last one it read.
The cli dispatch and pick-up read stderr off the endpoint as before and
now
get all of it, so the reason and the session id are there. An acp
pick-up's
connection drains the same endpoint into the stderr tail it quotes in
its
errors and into its journal, so what the agent wrote during the turn
while
this computer was away reaches them too; the acp run record keeps the
stderr
line count as the prompt goes out, so the earlier prompts' lines are not
read again (follow-up commit below). No protocol bump:
the new fields are optional, and a node runs exactly the host's code
anyway
(protocol and digest are checked at every connection).
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `tests/test_node_stderr_replay.py`, new, 22 tests: the daemon numbers
and
keeps stderr lines; an attach replays exactly the lines past
`stderr_after`
(from a live task, a finished one, and a record a daemon that never ran
it
reads), none when not asked, none from a record with no `stderr.log`,
and
refuses a bad `stderr_after`; the endpoint counts by the node's numbers;
a
re-attach and a pick-up ask from the right line. End to end on the
stand-in machine (real daemon, stand-in ssh): a left cli run's reason on
stderr reaches its pick-up (said before and after this computer
stopped),
a left create that names its session only on stderr is bound to it by
the
pick-up (both orders), and a reason written while the link was down
reaches the run watching it. 18 fail on main (2c27181); the 4 that
pass
there pin what does not change (no `stderr_after`, past the last line,
past the end, no `stderr.log`).
- Changed assertions: the four attach-parameter assertions in
`tests/test_node_client.py` and the forwarded-stderr one in
`tests/test_node_server.py` include the new fields.
- Full suite: 32065 passed, 173 skipped, 2 xfailed, 3 failed -- the same
3
that fail on main (2c27181) on this computer when run alone there: a
host
without LibreOffice, and two simulation cases.
- Diff coverage: 100.00% (30/30 executable changed lines; `coverage
json`, then
`scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`).
- `ruff check`, `ruff format --check`, `ty check`, `lint-imports`,
pre-commit on the diff; `scripts/check_commit_messages.py`,
`scripts/check_source_language.py` and `scripts/check_large_files.py` on
`origin/main..HEAD`: clean.
- Docs: CONTEXT.md's Raven node entry, the protocol module's notes, and
the
machines guide's "Disconnects and restarts" section in English and
Chinese
now say the daemon keeps and replays stderr; the changelog has an entry
under Fixed. `tests/test_docs_site.py` and `tests/test_living_docs.py`
pass
(18), `mkdocs build --strict` passes.
- Real machine, ravenx2, an isolated node directory per tree. The host
process that dispatched was killed with SIGKILL and the run picked up by
a
new process; the link was dropped by killing the local ssh. The command
writes `token expired` (or `session slow-A1`) on stderr:
| Probe | main (2c27181) | this branch |
|---|---|---|
| reason written after the host process was killed | `exited 3` |
`exited 3: token expired` |
| reason written before it was killed | `exited 3` | `exited 3: token
expired` |
| reason written while the link was down | `exited 3` | `exited 3: token
expired` |
| create left before its commit, session id only on stderr | "not
resumable", nothing bound | bound to `slow-A1`, no warning |
The machine was cleaned after: the probe's two daemons ended by pid, its
directory removed.
- The runs above are on main 2c27181 (#957 merged). The branch was
then
rebased onto 400b30d, whose two new commits touch only TUI copy and
i18n; the node, docs and range checks were run again there (238 passed,
1
skipped). It merges cleanly with the open #964 and #965 (`git
merge-tree`).
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
- Follow-up commit e0468fa, after the agent review's F1 (an acp
pick-up asked
for the stderr from the agent's first line, across every earlier prompt
of
its process): `RemoteRun.stderr_received`, taken in
`Tracker.prompt_sent`
off the endpoint's count, and passed through `NodePool.attach(...,
stderr_after=...)` by the pick-up; a cli pick-up keeps asking from 0; a
record from before the field reads with 0. Regression on the stand-in
machine
(`test_a_pick_up_reads_the_turn_s_own_stderr_and_not_what_the_prompts_before_it_wrote`):
two turns of 150 stderr lines, a third interrupted after its prompt, the
next start's pick-up -- before the change its tail held 200 lines, 50 of
them the second turn's; after it, the third turn's 150 and nothing else.
Unit tests pin the record, the pick-up's attach arguments and the
default
of 0, and the tracker's wiring. The related files 259 passed; full suite
32067 passed with the three failures this Mac always has, plus one test
whose stand-in `Tracker` lacked the new keyword, fixed in the same
commit.
Diff coverage of the changed lines: see CI.
## Risk
The replayed lines are the agent's own stderr, already kept in the
task's
owner-only record and already sent live to whichever connection held the
task; nothing new leaves the machine. `stderr_after` and `n` are
optional: an
attach without `stderr_after` behaves as before, and a line without a
number
counts on from the last. A cli pick-up replays the whole `stderr.log`,
as it
already replays every frame of the command; an acp pick-up replays the
lines
past the count its record kept as the prompt went out, so the replay is
bounded to the turn, and the connection keeps a bounded tail of it.
Rollback
is reverting the commit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
Follow-up to #957, for this review note:
#957 (comment)
---------
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
0 commit comments