Skip to content

Commit b31eecc

Browse files
Handsome-wzwclaude
andauthored
fix(node): an attach replays the agent's stderr past the line it last read (#977)
## Summary Follow-up to LivXue's review note on #957 (the pick-up's binding at `cli_agent.py:1393`). The machine's daemon kept an agent's stderr in the task's record (`stderr.log`) but sent it only to the connection holding the task when a line was written, and `agent/attach` replayed `frames.jsonl` alone. So what an agent said only on stderr while no connection held it never reached the host: - **The session id a CLI names only on stderr** (hermes does). The pick-up #957 added binds a create's handle to the id in its output; for such a CLI it found none, warned "not resumable", and the next turn to the instance opened a new session. - **The reason a command failed.** A run picked up after this computer stopped was reported as `exited 3` where the dispatch would have said `exited 3: token expired`. This part predates #957. - **The stderr written while a link was down**, on a run still being watched, the same way. The fix is in the node and the host's link to it (`raven/node/`): - The daemon numbers stderr lines as it numbers frames (`agent/stderr` carries `n`), and `agent/attach` takes an optional `stderr_after`: the stderr lines past that one are replayed after the frames, under the lock that forwards a new line, so none is sent twice or missed at the handover. It works the same for a live task, a finished one, and a record a daemon that never ran the task reads. An attach that does not ask hears no stderr, as before; a `stderr_after` that is not a whole number is refused like a bad `after`. - The host's `ChannelEndpoint` counts the stderr lines it got (`stderr_received`). A cli pick-up (`NodePool.attach`, a process that read none of them) asks from the first line; an acp pick-up from the line the run's record kept as the prompt went out (`RemoteRun.stderr_received`, follow-up commit below); a re-attach after a lost link (`NodePool._reattach`) asks from the last one it read. The cli dispatch and pick-up read stderr off the endpoint as before and now get all of it, so the reason and the session id are there. An acp pick-up's connection drains the same endpoint into the stderr tail it quotes in its errors and into its journal, so what the agent wrote during the turn while this computer was away reaches them too; the acp run record keeps the stderr line count as the prompt goes out, so the earlier prompts' lines are not read again (follow-up commit below). No protocol bump: the new fields are optional, and a node runs exactly the host's code anyway (protocol and digest are checked at every connection). ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - `tests/test_node_stderr_replay.py`, new, 22 tests: the daemon numbers and keeps stderr lines; an attach replays exactly the lines past `stderr_after` (from a live task, a finished one, and a record a daemon that never ran it reads), none when not asked, none from a record with no `stderr.log`, and refuses a bad `stderr_after`; the endpoint counts by the node's numbers; a re-attach and a pick-up ask from the right line. End to end on the stand-in machine (real daemon, stand-in ssh): a left cli run's reason on stderr reaches its pick-up (said before and after this computer stopped), a left create that names its session only on stderr is bound to it by the pick-up (both orders), and a reason written while the link was down reaches the run watching it. 18 fail on main (2c27181); the 4 that pass there pin what does not change (no `stderr_after`, past the last line, past the end, no `stderr.log`). - Changed assertions: the four attach-parameter assertions in `tests/test_node_client.py` and the forwarded-stderr one in `tests/test_node_server.py` include the new fields. - Full suite: 32065 passed, 173 skipped, 2 xfailed, 3 failed -- the same 3 that fail on main (2c27181) on this computer when run alone there: a host without LibreOffice, and two simulation cases. - Diff coverage: 100.00% (30/30 executable changed lines; `coverage json`, then `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`). - `ruff check`, `ruff format --check`, `ty check`, `lint-imports`, pre-commit on the diff; `scripts/check_commit_messages.py`, `scripts/check_source_language.py` and `scripts/check_large_files.py` on `origin/main..HEAD`: clean. - Docs: CONTEXT.md's Raven node entry, the protocol module's notes, and the machines guide's "Disconnects and restarts" section in English and Chinese now say the daemon keeps and replays stderr; the changelog has an entry under Fixed. `tests/test_docs_site.py` and `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes. - Real machine, ravenx2, an isolated node directory per tree. The host process that dispatched was killed with SIGKILL and the run picked up by a new process; the link was dropped by killing the local ssh. The command writes `token expired` (or `session slow-A1`) on stderr: | Probe | main (2c27181) | this branch | |---|---|---| | reason written after the host process was killed | `exited 3` | `exited 3: token expired` | | reason written before it was killed | `exited 3` | `exited 3: token expired` | | reason written while the link was down | `exited 3` | `exited 3: token expired` | | create left before its commit, session id only on stderr | "not resumable", nothing bound | bound to `slow-A1`, no warning | The machine was cleaned after: the probe's two daemons ended by pid, its directory removed. - The runs above are on main 2c27181 (#957 merged). The branch was then rebased onto 400b30d, whose two new commits touch only TUI copy and i18n; the node, docs and range checks were run again there (238 passed, 1 skipped). It merges cleanly with the open #964 and #965 (`git merge-tree`). - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed - Follow-up commit e0468fa, after the agent review's F1 (an acp pick-up asked for the stderr from the agent's first line, across every earlier prompt of its process): `RemoteRun.stderr_received`, taken in `Tracker.prompt_sent` off the endpoint's count, and passed through `NodePool.attach(..., stderr_after=...)` by the pick-up; a cli pick-up keeps asking from 0; a record from before the field reads with 0. Regression on the stand-in machine (`test_a_pick_up_reads_the_turn_s_own_stderr_and_not_what_the_prompts_before_it_wrote`): two turns of 150 stderr lines, a third interrupted after its prompt, the next start's pick-up -- before the change its tail held 200 lines, 50 of them the second turn's; after it, the third turn's 150 and nothing else. Unit tests pin the record, the pick-up's attach arguments and the default of 0, and the tracker's wiring. The related files 259 passed; full suite 32067 passed with the three failures this Mac always has, plus one test whose stand-in `Tracker` lacked the new keyword, fixed in the same commit. Diff coverage of the changed lines: see CI. ## Risk The replayed lines are the agent's own stderr, already kept in the task's owner-only record and already sent live to whichever connection held the task; nothing new leaves the machine. `stderr_after` and `n` are optional: an attach without `stderr_after` behaves as before, and a line without a number counts on from the last. A cli pick-up replays the whole `stderr.log`, as it already replays every frame of the command; an acp pick-up replays the lines past the count its record kept as the prompt went out, so the replay is bounded to the turn, and the connection keeps a bounded tail of it. Rollback is reverting the commit. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues Follow-up to #957, for this review note: #957 (comment) --------- Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
1 parent 547c1ab commit b31eecc

18 files changed

Lines changed: 670 additions & 72 deletions

‎CHANGELOG.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,19 @@ All notable changes to Raven are documented here.
262262

263263
### Fixed
264264

265+
- What an agent on a registered machine writes on stderr reaches Raven even
266+
when no link held the agent at the time. The node kept those lines but, when
267+
a link attached, replayed only the agent's output: a run picked up after a
268+
restart was reported as `exited 3` without the reason after it, a CLI that
269+
names its session only on stderr (hermes) could not be bound to it by the
270+
pick-up, so its next turn opened a new session, and what was written while a
271+
link was down was lost the same way. The node now numbers stderr lines as it
272+
numbers output lines and replays the ones the link has not seen. An ACP
273+
agent is one task across every prompt its process is sent, so its pick-up
274+
asks the replay from the line its record kept as the prompt went out, not
275+
from the agent's first line; a CLI pick-up, whose task is the command's
276+
own, reads from the first line.
277+
265278
- A task on a registered machine ends the way it ended, whatever happened on
266279
either side. A Raven node killed outright (`kill -9`) used to leave its
267280
agents running with nobody reading them and the host reading a bare

‎CONTEXT.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -443,10 +443,14 @@ whose entries name the machine, each under a task id of the host's, in its own p
443443
group in the machine user's login environment, forwards what they write (an ACP agent's
444444
frames, a CLI agent's output lines) on numbered channels of the connection that holds them
445445
-- a CLI agent's prompt reaches its stdin the other way (`agent/stdin`), which also closes
446-
it -- and keeps those lines in the task's record
446+
it -- and keeps those lines and the agent's stderr, each line numbered, in the task's record
447447
(`node_dir/tasks/<task>/`, owner-only like the host's journal), so a connection that goes
448448
leaves the agent running and the next
449-
one takes it over from the frame the host last read (`agent/attach`); and answers probes of
449+
one takes it over from the frame and the stderr line the host last read (`agent/attach`; a
450+
cli pick-up, which read none of it, gets the stderr from its first line -- a session id or a
451+
failure a CLI agent says only there is not lost with a link -- and an acp pick-up from the
452+
line its run record kept as the prompt went out, so the earlier prompts' lines are not read
453+
again); and answers probes of
450454
the machine and of an agent's command. It holds no conversation, calls no model and opens
451455
no network port. It runs exactly the host's code, checked by protocol and digest at every
452456
connection (`raven/node/bundle.py`); `raven ops connection install-node` puts it on a

‎docs-site/docs/machines.md‎

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -304,19 +304,21 @@ address it by id thanks to the alias written at registration.
304304
Raven keeps one ssh link per machine and watches it with a heartbeat: a ping
305305
every 5 seconds, and 15 seconds without an answer means the link is gone. The
306306
agents are not: they run under the machine's daemon, which keeps every frame
307-
they write. Raven tries the link again on a backoff of 2, 4, 8, then 16
308-
seconds for as long as any agent is waiting, and a link that comes back
309-
attaches each agent by its task id, from the frame Raven last read. The turn
310-
goes on as if nothing had happened; a cancel sent while the link was down is
311-
delivered when it is back.
307+
they write and every line of their stderr. Raven tries the link again on a
308+
backoff of 2, 4, 8, then 16 seconds for as long as any agent is waiting, and a
309+
link that comes back attaches each agent by its task id, from the frame and
310+
the stderr line Raven last read. The turn goes on as if nothing had happened;
311+
a cancel sent while the link was down is delivered when it is back.
312312

313313
Stopping Raven does the same on purpose. An agent running on a machine is left
314314
to the daemon; what Raven keeps is a record under
315315
`<raven home>/machines/runs/`. The next start reads those records, attaches
316-
each run from where it was, and announces the result into the conversation
317-
that asked, as a finished turn. Keep the machine's registration and the agent
318-
entry in place for that. `/restart` and a Raven that was killed are covered
319-
the same way.
316+
each run from where it was, reads what it wrote on stderr during that turn
317+
too (a command's whole stderr: the reason it failed, or the session id a CLI
318+
names only there), and announces the
319+
result into the conversation that asked, as a finished turn. Keep the
320+
machine's registration and the agent entry in place for that. `/restart` and a
321+
Raven that was killed are covered the same way.
320322

321323
What is not recoverable: a machine that rebooted, or a daemon that was ended
322324
by hand, no longer knows the task. The turn is then announced as an error

‎docs-site/docs/machines.zh.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -245,14 +245,15 @@ grep, write_file and edit_file take that path as it is, and deliver_files fetche
245245
## 7. 断线与重启 { #7-disconnects-and-restarts }
246246

247247
Raven 对每台机器保持一条 ssh 链路,用心跳看着它:每 5 秒 ping 一次,15 秒没有回应就算
248-
链路断了。Agent 没有断:它们在机器的守护进程下继续运行,写出的每一帧都被守护进程保存。
248+
链路断了。Agent 没有断:它们在机器的守护进程下继续运行,写出的每一帧和 stderr 的每一行都被守护进程保存。
249249
只要还有 Agent 在等,Raven 就按 2、4、8、16 秒的退避不断重试;链路回来后按任务 id
250-
把每个 Agent 接回,从 Raven 上次读到的那一帧继续,这一轮像什么都没发生一样进行下去;
250+
把每个 Agent 接回,从 Raven 上次读到的那一帧和那一行 stderr 继续,这一轮像什么都没发生一样进行下去;
251251
链路断开期间发出的取消,链路回来时送达。
252252

253253
停止 Raven 的效果是一样的,而且是有意为之。机器上的 Agent 留给守护进程,Raven 这边
254254
留下的是 `<Raven 主目录>/machines/runs/` 下的一条记录。下次启动时读这些记录,把每次
255-
运行从原来的位置接回,再把结果通告到发起它的那个对话,就像一轮正常结束一样。为此要保留
255+
运行从原来的位置接回,连同这一轮写在 stderr 里的内容一起读到(命令的整个 stderr:失败的原因,或只在 stderr
256+
里报会话 id 的 CLI 给出的 id),再把结果通告到发起它的那个对话,就像一轮正常结束一样。为此要保留
256257
机器的登记和 Agent 条目。`/restart` 和被强制结束的 Raven 也按同样方式处理。
257258

258259
接不回来的情况:机器重启过,或者守护进程被手动结束,任务就不再被机器认识。这一轮会以错误

‎raven/acp_client/acp_agent.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2037,9 +2037,10 @@ def _track_on_machine(
20372037
session_cwd=session_cwd,
20382038
stateful=self.is_stateful,
20392039
received=int(getattr(endpoint, "received", 0) or 0),
2040+
stderr_received=int(getattr(endpoint, "stderr_received", 0) or 0),
20402041
)
20412042
try:
2042-
tracker = remote_runs.Tracker(run)
2043+
tracker = remote_runs.Tracker(run, stderr_count=lambda: getattr(endpoint, "stderr_received", 0))
20432044
except OSError as exc:
20442045
logger.warning(
20452046
"acp agent {!r}: the run on machine {} could not be recorded: {}", self.name, self.machine, exc

‎raven/acp_client/remote_resume.py‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,11 @@
22
33
The run's record (:mod:`raven.acp_client.remote_runs`) says which machine,
44
which task its daemon runs the agent under, which ACP session the turn is in,
5-
which prompt is awaited and how many frames that process had read when the
6-
prompt went out. This module turns that into the reply the turn owes: the
7-
machine's daemon replays the turn from there, whole, and hands the rest over
8-
as it comes (``agent/attach``),
5+
which prompt is awaited and how many frames, and how many stderr lines, that
6+
process had read when the prompt went out. This module turns that into the
7+
reply the turn owes: the machine's daemon replays the turn from there, whole,
8+
with the stderr the turn wrote, and hands the rest over as it comes
9+
(``agent/attach``),
910
a connection over that channel is taken into the pool as if started here, the
1011
turn's collector reads the agent's messages off it, and the answer to the
1112
prompt -- sent by the other process, awaited by this one -- ends the wait. What
@@ -66,7 +67,9 @@ async def pick_up(run: RemoteRun, backend: AcpAgentBackend, *, pool: Any = None)
6667
if run.prompt_id <= 0:
6768
return PickedUp("error", f"the run of {run.agent!r} on machine {target.label} ended before its prompt was sent")
6869
try:
69-
endpoint, attached = await link.attach(run.machine, run.task, name=run.agent, after=run.received)
70+
endpoint, attached = await link.attach(
71+
run.machine, run.task, name=run.agent, after=run.received, stderr_after=run.stderr_received
72+
)
7073
except remote_path.NodeCallError as exc:
7174
if exc.reason in ("unknown_task", None) and "no longer runs" in str(exc):
7275
return PickedUp("error", str(exc))

‎raven/acp_client/remote_runs.py‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
import json
3131
import os
3232
import time
33-
from collections.abc import Iterator
33+
from collections.abc import Callable, Iterator
3434
from dataclasses import asdict, dataclass, field
3535
from pathlib import Path
3636
from typing import Any
@@ -73,6 +73,11 @@ class RemoteRun:
7373
received: int = 0
7474
"""How many of the agent's frames this computer had read when the prompt went out: where an attach goes on
7575
from, so a pick-up reads the turn from its first frame."""
76+
stderr_received: int = 0
77+
"""How many of the agent's stderr lines had reached this computer when the prompt went out: where a pick-up
78+
asks the stderr replay from. An ACP agent is one task across every prompt its process is sent, so what the
79+
earlier prompts wrote is not read again into this turn's tail and journal; a cli run, whose task is the
80+
command's own, keeps 0 and reads all of it."""
7681
started: float = field(default_factory=time.time)
7782
kind: str = "acp"
7883
"""``acp`` for a turn in an ACP session, ``cli`` for a command whose whole output is the reply
@@ -344,18 +349,26 @@ class Tracker:
344349
It moves the record only until ``prompt_sent``: the turn's own frames are
345350
never checkpointed, since the reply is put together from all of them --
346351
the chunks before the answer as much as the answer -- and a process that
347-
picks the run up has to read the turn from its first frame.
352+
picks the run up has to read the turn from its first frame. ``stderr_count``
353+
is read once, as the prompt goes out: the stderr lines before it are the
354+
earlier prompts', and a pick-up asks the replay from past them.
348355
"""
349356

350-
def __init__(self, run: RemoteRun) -> None:
357+
def __init__(self, run: RemoteRun, *, stderr_count: Callable[[], Any] | None = None) -> None:
351358
self.run = run
352359
self.kept = False
360+
self._stderr_count = stderr_count
353361
record(run)
354362

355363
def prompt_sent(self, request_id: int) -> None:
356-
"""The prompt went out under ``request_id``: what a later process awaits the answer to."""
364+
"""The prompt went out under ``request_id``: what a later process awaits the answer to; the stderr line
365+
count is taken now."""
357366
self.run.prompt_id = request_id
358367
self.run.next_id = request_id + 1
368+
if self._stderr_count is not None:
369+
n = self._stderr_count()
370+
if isinstance(n, int) and not isinstance(n, bool) and n > self.run.stderr_received:
371+
self.run.stderr_received = n
359372
try:
360373
record(self.run)
361374
except OSError as exc:

‎raven/node/client.py‎

Lines changed: 40 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,13 @@
1313
The link is watched from here, not by ssh: a ``node/ping`` every
1414
:data:`HEARTBEAT_S` seconds, and one unanswered for :data:`HEARTBEAT_LOST_S`
1515
means the link is gone -- not the agents. They run on under the machine's
16-
daemon, which keeps their frames; each agent's endpoint here waits, holding
17-
what the host writes meanwhile, and the link is tried again on a backoff
18-
(:data:`RECONNECT_BACKOFF_S`) for as long as any agent is waiting. A link
19-
that comes back attaches each agent again by its task id, from the frame the
20-
host last read, and the turn goes on as if nothing had happened. Only a task
21-
the machine no longer knows ends an endpoint with the sentence that says so.
16+
daemon, which keeps their frames and their stderr; each agent's endpoint here
17+
waits, holding what the host writes meanwhile, and the link is tried again on
18+
a backoff (:data:`RECONNECT_BACKOFF_S`) for as long as any agent is waiting. A
19+
link that comes back attaches each agent again by its task id, from the frame
20+
and the stderr line the host last read, and the turn goes on as if nothing had
21+
happened. Only a task the machine no longer knows ends an endpoint with the
22+
sentence that says so.
2223
2324
An MCP server the host lends an agent on the machine is carried on the link
2425
too: the agent runs the node's bridge as a stdio server, the daemon tells this
@@ -183,6 +184,9 @@ def __init__(
183184
self.pid: int | None = None
184185
self.received = 0
185186
"""How many of the agent's frames reached here: where an attach goes on from."""
187+
self.stderr_received = 0
188+
"""How many of the agent's stderr lines reached here: where an attach goes on from for those. What a CLI
189+
agent says only there -- a session id, the reason it failed -- is replayed past it, not lost with a link."""
186190
self.watchers: list[Callable[[int], None]] = []
187191
"""Told, after each frame the connection has dealt with, how many it has read: what a record of the
188192
run keeps until its prompt is out, so a process that comes after this one attaches from there and reads
@@ -324,8 +328,9 @@ def feed_frame(self, line: Any, n: Any = None) -> None:
324328
self.received = n if isinstance(n, int) and not isinstance(n, bool) else self.received + 1
325329
self._frames.put_nowait((self.received, line.encode("utf-8") + b"\n"))
326330

327-
def feed_stderr(self, text: Any) -> None:
331+
def feed_stderr(self, text: Any, n: Any = None) -> None:
328332
if isinstance(text, str) and not self._done.is_set():
333+
self.stderr_received = n if isinstance(n, int) and not isinstance(n, bool) else self.stderr_received + 1
329334
self._stderr.put_nowait(text.encode("utf-8") + b"\n")
330335

331336
def exited(self, rc: Any, reason: Any = None) -> None:
@@ -609,15 +614,20 @@ async def open_channel(
609614
return endpoint
610615

611616
async def attach(
612-
self, machine_id: str, task: str, *, name: str, after: int = 0
617+
self, machine_id: str, task: str, *, name: str, after: int = 0, stderr_after: int = 0
613618
) -> tuple[ChannelEndpoint, dict[str, Any]]:
614-
"""``task`` on the machine, onto a channel of its link, from the frame past the ``after``-th.
619+
"""``task`` on the machine, onto a channel of its link, from the frame past the ``after``-th and the
620+
stderr line past the ``stderr_after``-th.
615621
616622
What a process that comes after the one that started the agent uses
617623
to pick the run up: the daemon replays the frames past the record's
618-
count -- the turn whole, from where its prompt went out -- and hands
619-
the rest over as it comes, or, for a task that has ended, replays its
620-
record and says so. The endpoint returned is what an
624+
count -- the turn whole, from where its prompt went out -- and the
625+
agent's stderr past ``stderr_after``: from its first line for a cli
626+
run, whose task is the command's own and none of whose stderr this
627+
process has read, and from the line the run's record kept as the
628+
prompt went out for an acp agent, which is one task across every
629+
prompt its process is sent; and hands the rest over as it comes, or,
630+
for a task that has ended, replays its record and says so. The endpoint returned is what an
621631
:class:`AcpClient` runs over; the answer says the pid, the frames
622632
and the exit code (``None`` while the agent runs).
623633
"""
@@ -627,8 +637,9 @@ async def attach(
627637
node.next_channel += 1
628638
endpoint = ChannelEndpoint(node.client, channel, target=node.target, agent=name, task=task)
629639
endpoint.received = after
640+
endpoint.stderr_received = stderr_after
630641
node.channels[channel] = endpoint
631-
params = {"channel": channel, "task": task, "after": after}
642+
params = {"channel": channel, "task": task, "after": after, "stderr_after": stderr_after}
632643
try:
633644
answer = await node.client.request(protocol.AGENT_ATTACH, params, timeout=_ATTACH_TIMEOUT_S)
634645
except AcpRemoteError as exc:
@@ -650,7 +661,13 @@ async def attach(
650661
if isinstance(answer.get("pid"), int):
651662
endpoint.pid = answer["pid"]
652663
node.used = time.monotonic()
653-
logger.info("acp agent {!r}: picked up on {} from frame {}", name, endpoint.describe(), after)
664+
logger.info(
665+
"acp agent {!r}: picked up on {} from frame {} and stderr line {}",
666+
name,
667+
endpoint.describe(),
668+
after,
669+
stderr_after,
670+
)
654671
return endpoint, answer
655672

656673
def status(self, machine_id: str) -> MachineStatus:
@@ -950,7 +967,7 @@ async def _notified(
950967
if method == protocol.AGENT_FRAME:
951968
endpoint.feed_frame(params.get("line"), params.get("n"))
952969
elif method == protocol.AGENT_STDERR:
953-
endpoint.feed_stderr(params.get("text"))
970+
endpoint.feed_stderr(params.get("text"), params.get("n"))
954971
elif method == protocol.AGENT_EXIT:
955972
channels.pop(channel, None)
956973
endpoint.exited(params.get("rc"), params.get("reason"))
@@ -1020,7 +1037,8 @@ async def _reconnect(self, machine_id: str) -> None:
10201037
self._reconnects.pop(machine_id, None)
10211038

10221039
async def _reattach(self, machine_id: str) -> None:
1023-
"""Every endpoint parked for the machine onto a channel of its new link, from the frame it last read.
1040+
"""Every endpoint parked for the machine onto a channel of its new link, from the frame and the stderr line
1041+
it last read.
10241042
10251043
A link that goes while this runs -- under the attach, or under the
10261044
replay of what an endpoint held -- parks that endpoint and every one
@@ -1039,7 +1057,12 @@ async def _reattach(self, machine_id: str) -> None:
10391057
channel = node.next_channel
10401058
node.next_channel += 1
10411059
node.channels[channel] = endpoint
1042-
params = {"channel": channel, "task": endpoint.task, "after": endpoint.received}
1060+
params = {
1061+
"channel": channel,
1062+
"task": endpoint.task,
1063+
"after": endpoint.received,
1064+
"stderr_after": endpoint.stderr_received,
1065+
}
10431066
try:
10441067
await node.client.request(protocol.AGENT_ATTACH, params, timeout=_ATTACH_TIMEOUT_S)
10451068
if endpoint.returncode is None:

‎raven/node/protocol.py‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,9 @@
1515
them, a blank line included, since they are its answer; its prompt reaches
1616
its stdin as ``agent/stdin``) -- and a *task* id of the host's
1717
choosing, which outlives the connection: an agent keeps running when the ssh
18-
goes, its frames are kept on the machine, and a new connection picks them up
19-
from where the host stopped reading (``agent/attach``). Requests are answered
18+
goes, its frames and its stderr (``agent/stderr``) are kept on the machine,
19+
every line numbered, and a new connection picks them up from where the host
20+
stopped reading (``agent/attach``; the stderr when it asks). Requests are answered
2021
as they finish, not in the order they arrive: a search of a large tree does
2122
not hold a heartbeat behind it.
2223
@@ -51,7 +52,7 @@
5152
SESSION_DIR = "session/dir"
5253

5354
AGENT_LAUNCH = "agent/launch"
54-
AGENT_ATTACH = "agent/attach"
55+
AGENT_ATTACH = "agent/attach" # ``channel``, ``task``, ``after``; ``stderr_after`` replays the stderr past that line
5556
AGENT_FRAME = "agent/frame"
5657
AGENT_STDIN = "agent/stdin"
5758
AGENT_STDERR = "agent/stderr"

0 commit comments

Comments
 (0)