Skip to content

feat(*): pick up a turn left on a registered machine when this computer stops - #934

Merged
0xKT merged 53 commits into
EverMind-AI:mainfrom
Handsome-wzw:feat/subagent_server_resume
Oct 14, 2026
Merged

0xKT merged 53 commits into
EverMind-AI:mainfrom
Handsome-wzw:feat/subagent_server_resume

Conversation

@Handsome-wzw

@Handsome-wzw Handsome-wzw commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Registered machines are now available without a debug environment switch.
RAVEN_DEBUG_MACHINES previously defaulted to off: it kept machine-backed
agents out of the dispatch table, refused node/installer access, hid remote
preset destinations, and treated file paths locally while off. It was a
trial feature gate, not a verbose-logging option. At the owner's request,
this PR removes the variable and all of those checks. Normal startup and
restart recovery use the configured machine entries; even a leftover value
of 0 is ignored. Machine registration, SSH authentication, file permission
prompts, and concurrency limits still apply.

Fourth change in the series that lets a Raven dispatch work to the
machines in its machine registry. #920, #921 (merged through #922) and
#922 are on main; this change stands on main alone now, and the diff is
its own.

What this does: a turn running on a registered machine survives this
computer stopping, and the next start picks it up and reports it.

  • While a turn runs on a machine, this computer remembers what the
    machine does not know (raven/acp_client/remote_runs.py): one
    owner-only file per run under <raven home>/machines/runs/, with the
    machine's registry id, the task id its daemon runs the agent under, the
    agent, the session key, the instance handle, the dispatch id, the ACP
    session, the id of the prompt whose answer is awaited, the ids this
    side has used, how many of the agent's frames were read here, the
    agent's answer to initialize and the connection's launch key. Kept
    current after every frame the connection has dealt with, removed when
    the turn ends here. No address is in it.
  • When this process stops -- the gateway, serve, the TUI, the page
    stack -- the
    agents on machines stay with their daemons
    (remote.leave_agents_running() before the drain): a cancelled turn
    on a machine sends no session/cancel, closing its connection ends the
    endpoint only, and the manager leaves the dispatch's record open rather
    than finishing it as cancelled. A hard kill or /restart leaves the
    same record, written as the turn ran.
  • At the next start -- the gateway once its generation is bound; the
    serve, TUI and page stacks once -- raven/agent/subagent/resume_remote.py
    picks every run up: task/list on the machine, agent/attach from the
    frame last read (NodePool.attach), the connection adopted into the
    pool with request ids carried on past the other process's
    (AcpConnectionPool.adopt, AcpClient.continue_ids_from), the
    prompt's answer awaited (AcpClient.await_response), the reply built
    as a turn ended here is -- the partial notice when the agent stopped
    early, the machine note saying it was picked up after this computer
    restarted -- then the dispatch's record finished, the registry row and
    the node claim closed, and the result announced into the conversation
    that asked, as the manager announces a turn that ended here
    (raven/acp_client/remote_resume.py). A machine that cannot be asked
    keeps its record for a later start; a run the machine no longer has,
    one whose agent ended while this computer was away, one whose agent
    left the agent table, and one whose turn says nothing are announced as
    errors that say so. The adopted agent process serves later dispatches
    of its entry, as one started here would.
  • Seams: AcpClient.request(sent=) hears the id of what goes out;
    ChannelEndpoint.watchers hear the frames the connection has dealt
    with (consumed, called by the read loop); ChannelEndpoint.stays
    reads the leaving flag.
  • A cli entry's run on a machine is recorded while it runs (kind cli,
    no ACP part) and left the same way, the manager reading the dispatch
    as left; the entry picks it up (CliAgentBackend.pick_up): the daemon
    replays the command's whole output from its first line, the exit code
    ends it, and the reply closes with the machine note.
  • The record carries the task as the model wrote it, and -- once the
    manager has read the run as left -- the dispatch's origin and summary
    (remote_runs.annotate): the announce after a restart names the
    instance, node and tool call as the announce here does and quotes the
    task as authored, never the rendered prompt on disk; a host killed
    outright gets an origin made up from the spawn record and the run.
  • The record goes last, once the outcome has reached its conversation:
    a conclusion that fails, a stop under it, or an announce the scheduler
    refuses (_announce_result says so now) leaves the record, its claim
    released, for the next start to try again.
  • A run that returned here keeps its record the same way, until the
    manager's announce of the reply was taken: the manager dispatches with
    reported_by_caller=True, the backend then leaves the record behind
    with the reply instead of removing it as it returns, and the manager
    removes it after the announce (remote_runs.forget_dispatch) -- or,
    when the draining scheduler refused the announce, annotates it with
    the dispatch's origin and summary for the next start to report. A host
    killed between the agent's answer and the announce therefore still has
    the way back to the run. A caller that takes the reply as its own
    result (a graph node, a probe, a direct chat) dispatches without the
    keyword, and the record goes as it did. The record's frame count stops
    at the prompt: the turn's own frames, the chunks before the answer as
    much as the answer, are never checkpointed, so a pick-up replays the
    turn whole and the reply it reports is complete; a record that had
    moved past the answer would have resumed after it and waited forever.

Not in this change: a run from the DAG lane has no spawn record to
finish; it is announced from the origin the manager added to its record
when it was left, or, for a host killed outright, from the run's record
alone. The Tasks page reads the finished record on its next load rather
than live.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

Machine availability and restart checks after removing the trial switch:

  • New default-behavior regressions were run before the removal: 8 failed,
    with the enabled-value control passing. After removal, all 9 passed,
    including ACP/CLI recovery through the real AgentRegistry with the
    variable absent or set to 0.

  • uv run --no-sync pytest tests/test_acp_remote.py tests/test_node_client.py tests/test_node_server.py tests/test_node_install.py tests/test_ops_remote_path.py tests/test_permissions_remote_read.py tests/test_remote_file_tools.py tests/test_deliver_files_tool.py tests/test_raven_config_tool.py tests/test_rpc_subagents.py tests/test_shell_machine_channel.py tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py tests/test_resume_remote.py tests/test_remote_resume.py tests/test_remote_runs.py tests/test_subagent_manager.py tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short
    -- 1799 passed on the combined tree, including the latest published
    result-retention fix. An earlier run had a missing stdio response in the
    minimal-node dependency test and an idle-gate hit; both passed a focused
    recheck, and this final complete selection passed with the strict gate.

  • uv run --no-sync python scripts/coverage_gate.py --coverage-json <coverage-json> diff --base-ref origin/main --threshold 90
    -- 92.92% (656/706 changed executable lines) for this PR; the gate-removal
    commit alone covers 2/2 new executable lines (the remainder deletes gates
    and updates descriptions).

  • UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types, the
    commit/source-language/large-file scope gates and git diff --check pass.
    Both RPC generators' --check runs pass; npm test -- scripts/gates/fixture-shape.test.mjs
    from ui-web passes all five tests.

  • Real Linux machine, isolated node installations, separate dispatch and
    restart processes: before removal, a task started with the trial switch
    on was reported as an absent agent and its recovery record deleted when
    restarted without it. After removal, both absent and old 0 settings
    recover the full CLI result and read its output file by address. The
    absent-variable case kills the relay before host shutdown. The combined
    final tree was reinstalled and both scenarios repeated successfully.

  • With the variable absent, 12 further timing/boundary probes pass: early
    CLI exit and blank lines, relay loss before heartbeat with normal-call
    recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection,
    11 launches admitting 8 and refusing 3, occupied tunnel port, and an
    unwritable prompt file. Three attachment, POSIX-command and IPv6 checks
    also pass; Windows command parsing is simulated, not native Windows.
    No test task or tunnel remains; the dedicated node directories are removed.

  • CONTEXT.md, configuration and RPC descriptions, transport/path docstrings,
    both collaboration guides and the owner's design notes were checked
    against the code. The guides explain the former trial switch and normal
    startup without it. The existing SSH, permission and limit tests continue
    to exercise their checks with no test-only switch setup.

  • env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short --cov=raven --cov-branch --cov-report=json:<full-coverage-json> --cov-report=
    -- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate current-main
    checkout reproduces four failures: CLI color rendering on this host,
    LibreOffice detection, and the two simulation cases. The remaining failure
    was the direct-chat test stub rejecting reported_by_caller from the
    latest result-retention change; the published wrapper fix now forwards only keywords declared by its fake backend. The full
    suite was not rerun for that test-only correction. Final direct-chat and
    manager recheck: 277 passed with strict idle accounting. Full-PR changed-line coverage from this
    run is 96.18% (679/706).

Earlier verification:

  • uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py tests/test_resume_remote.py tests/test_acp_client_resume.py tests/test_node_client.py tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py tests/test_subagent_manager.py tests/test_rpc_bootstrap.py tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py tests/test_subagent_acp.py tests/test_scope_canon.py tests/test_l4_entrances.py
    -- all passed (605 passed). Covered: the record written owner-only, kept current
    per frame and per prompt, removed when the turn ends here and kept when
    this process leaves; a dispatch left at shutdown neither finished nor
    announced by the manager; an endpoint ended while leaving leaves the
    agent with the daemon (real daemon); a task picked up from the frame
    last read, with the watchers hearing each frame; a pick-up the machine
    refuses or cannot carry; an answer awaited for a request another
    process sent, ids carried on, no session/cancel while leaving; the
    reply read off replayed frames with the other sessions' frames
    ignored, a turn cut short marked partial, an unreachable machine
    keeping the record, a run the machine no longer has, a prompt never
    sent, an agent that ended while this computer was away, an answer
    waited out, a turn that said nothing, a pick-up beside a live
    connection closing its own; the manager-side conclusion: record
    finished and announced, an error announced and the record failed, an
    unreachable machine keeping the record, an agent off the table, a
    failure inside one pick-up not stopping the next, a run with no
    conversation, the once-per-process scheduling; a cli run recorded
    while it runs and left at shutdown, picked up whole on the real daemon
    behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a
    machine that no longer has it, a link down, this computer leaving), a
    cancel that is not a leave ending the command with no record left; the
    TUI's stop saying the agents stay before the cancel and its start
    scheduling the pick-up; the record annotated with the dispatch's
    origin and summary when the manager reads it as left, the announce
    quoting the task as authored and naming instance, node and tool call,
    with and without the manager's fields; the record outliving a refused
    announce and a failed conclusion, claim released; a cli record read
    without an ACP session, an older record read with defaults; the record
    moving with the frames before the prompt and not past it, a dispatch's
    record forgotten by the manager's id with its claim, the manager
    dispatching with reported_by_caller and forgetting the record after
    an announce that was taken and annotating it after one that was
    refused; on the real daemon behind the stand-in ssh, an acp reply and
    a cli output returned with the record kept and the host "stopped"
    before the announce, the next start reading the same reply again
    whole, and a dispatch without the keyword leaving no record. End to end on the real
    daemon behind the stand-in ssh: a turn dispatched, this process
    "stopped" the way the gateway stops, the agent still running on the
    daemon, then a fresh pick-up reading its answer, finishing the record
    and announcing it, with the agent's process the pool's connection
    again.

  • uv run pytest on this tree, at an earlier revision of this change -- 29270 passed, 121 skipped, 3 failed:
    test_rpc_files.py::test_a_host_without_libreoffice_says_so,
    test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs
    and test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence,
    which fail the same way on main on this machine; diff coverage of this
    change against its base: 97.45% (153 of 157 executable changed lines),
    and 94.67% for the four changes against main; lint-imports 10 kept,
    0 broken; ty clean.

  • Real machine (a registered Linux machine with Claude Code signed in,
    node reinstalled with this tree): process A dispatched a 25 s task,
    saw the prompt out after 2.4 s and the record keep up with the frames
    (6, then 10), then stopped the way the gateway stops 5.4 s after the
    dispatch, leaving the record and the agent running there; process B,
    started fresh, found the record and the running task, picked it up
    in 2.4 s, announced the agent's reply to the conversation with the
    machine note, finished the record, removed the run's record, held the
    agent's process as the pool's connection, and closing the pool ended
    the agent on the machine. This revision, the cli lane with the code
    before and after, the node reinstalled for each: a cli entry
    sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat' was dispatched
    and the host stopped the gateway's way 2.7 s later. Before: no
    record, the command left running on the machine, and the next start
    found nothing to pick up. After: the record kept with the task as
    authored and the manager's origin, the command running on, and the
    next start picked it up in 16.8 s (the sleep), announcing done and
    hi with the instance, node and tool call and the machine note. The
    same with the link killed before the stop: recorded, left, picked up.
    With the machine's daemon ended between the stop and the pick-up, it
    ended its groups on its way out, the task's record there said exit
    -9, and the pick-up announced that as the run's error in 0.1 s,
    leaving nothing on the machine.

  • scripts/check_source_language.py, scripts/check_commit_messages.py
    and scripts/check_large_files.py on the branch range, commitlint,
    ruff -- all clean.

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally

  • User-facing docs or screenshots are updated when needed

The English and Chinese agent-collaboration guides explain normal startup,
registered-machine setup and recovery, and the removed trial switch.

Risk

Configured machine entries are available by default; entries without a
machine remain local. A remote-looking file path is resolved against the
registry even without an environment variable; use an explicit local path
such as ./a:b for a local filename containing a colon. A formerly hidden
machine entry is now available for dispatch, and an old variable set to 0
no longer disables it. Stopping Raven no longer cancels a turn
running on a machine; it goes on there until the agent finishes or is
cancelled, and its answer arrives in the conversation after the next
start, as a sub-agent result that triggers a turn. One record per run in
the raven home, owner-only, removed when the run is concluded; a machine
that cannot be reached keeps it, so a run is reported at most once and
never lost quietly. The adopted connection answers requests with ids
past the ones the stopped process used, so a replayed frame resolves
nothing sent after the restart. Rollback is a plain revert; a record left
behind is then an inert file.

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

Related Issues

#920 and #922 (merged); #921 was carried by #922.

Handsome-wzw and others added 17 commits October 12, 2026 19:03
…ched ssh link

A registered machine gets this Raven's own code as a daemon (raven/node):
started by the relay the host's ssh runs there (`python -I -m raven.node
--attach --dir <node_dir>`) when none answers on its socket under the
node directory, and left on its own once idle. It answers file calls with
Raven's own tools fenced to the directories each call names (fs/read,
list, find, grep, stat, write, edit, pull), starts acp agents under task
ids of the host's in their own process groups in the machine user's
login environment, keeps each agent's frames in the task's record on
disk so a connection that goes leaves the agent running and the next one
takes it over from the frame the host last read (agent/attach), lists
the tasks it runs or remembers, and probes the machine and an agent's
command. It opens no network port; only --stdio ties the agents to the
connection. `raven ops connection install-node <id>` puts it on a
machine after one look and one question, and `--prune` removes older
installs.

The host keeps one link per machine (raven/node/client.py): the ssh line
runs the relay, a heartbeat watches the link, a lost link parks the
agents on it and is tried again on a backoff for as long as any waits,
and a link that comes back attaches every task again by its id, sending
what the host wrote meanwhile and any cancel asked in between. The
registry's package tells a remote path from a local one and lends the
tools the pool through one seam (raven/ops/remote_path.py); the machine
lane stays debug-only (RAVEN_DEBUG_MACHINES=1).

First of three changes on this design; the next runs acp sub-agents on
a machine through this daemon, the one after reads and writes files
there by address. Nothing dispatches to a machine yet.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…mon, add it from a chat

An acp entry that names a `machine` (and a `remoteCwd`, `~/raven-work`
by default) is started there by the machine's Raven daemon, over the
link the host keeps to it: its command runs in the machine user's login
environment with the entry's env on the link, never on a command line;
its session works in `<remoteCwd>/<instance or task id>`, made there for
each dispatch; its ACP frames travel on a channel of the link, under a
task id, so an ssh that goes mid-turn leaves it running and the turn goes
on once the link is back; a cancel ends its whole process tree. The
reply says where it ran, by address, and that what it wrote is there.
The files that come with a task are named to the agent by address
(remote.files_note); nothing is carried over. Raven's keys are not lent
to such an entry, and the lane stays debug-only: with it off the entry
is left off the table and every door says so in one sentence.

The roster names the machine after the entry's tags; the free probe
reads the link's state and the last test, never the machine; a test
runs the real handshake there and marks a refusal "needs attention"
without offering a command to run here. `subagents.add` takes `machine`
and `remote_cwd` for an acp preset -- from the page, and from a chat
through `raven_config`, which lists the registered machines while the
lane is on -- names the row `<preset>@<machine>`, refuses before anything
is spent when the lane is off, the id is not usable, the preset is not
acp or a key would be lent, and proves the entry with the same readiness
ping as a local add, run on the machine. The list rows carry `machine`;
the contract and the generated clients carry the fields.

Second of three changes on this design; the one after reads and writes
files on a machine by address.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…ress

The file tools take a path on a registered machine as it is written for
scp -- `user@host:/path` or `scp://user@host:port//path` -- and answer it
through the machine's daemon: read_file, list_dir, find and grep run
Raven's own tools there and bring back the answer; write_file and
edit_file write there with the digest of the last read as the expected
state, so a file changed since is refused and nothing is written;
deliver_files fetches one file by its address, whole and checked, under
a size cap. A path is read inside the directories Raven's agents work in
on that machine without asking, elsewhere once per machine and
directory for the session, writes asked apart from reads. What is shown
back is the address form, never a bare address; and the shell refuses
ssh, scp, sftp and rsync to a registered machine, naming the file tools.

Third of three changes on this design.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…when the machine has none

The install ran `python -m venv` and `python -m pip install` on the
machine, against the repository's rule that uv is the only Python package
manager. Everything Python now goes through uv: `uv venv` of the machine's
own Python 3.12 or of one uv keeps inside the node directory, `uv pip
install` for the pinned packages and the optional search package. A
machine without uv gets one under `<node_dir>/.uv` first, from uv's own
installer fetched with the machine's curl or wget and told to touch
nothing else; the probe looks for curl and wget, and for a uv an earlier
install put there; with no uv and no way to fetch one the install stops
and says so. The plan counts the room uv takes and says when it is fetched.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…s stay off the network

The reply of a run on a machine pointed at read_file, list_dir, find,
grep, write_file, edit_file and deliver_files for the files it left
there, which on this change still read a path as this computer's: it
now says only where the files are. Two tests of an add on a machine
reached for the machine over a real ssh when the capability record ran
after the stood-in ping, and waited out its connect timeout on CI: the
record is stood in for too. The page's fixture gate lists the `machine`
field of a sub-agent row as one its library does not send (every row
there runs on this computer), and a daemon the stand-in ssh starts in a
test leaves soon after the test does.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
… file under its whole path

A read's numbered lines, a search's matches and a diff's body came back
with every registered root rewritten as an address, so the model edited
against text the file does not hold: only the tool's own sentences --
its errors, its headers and footers, a write's account and a diff's two
header lines -- are shown that way now. A delivered file landed under
its bare name, so two files of one name from two directories landed on
each other: it lands under its whole path on the machine. A file
outside the directories this Raven's agents work in there was fetched
on the strength of a read some conversation had been allowed, which is
no evidence for this one: it is refused, and read there instead. The
remote run's reply names the file tools again, now that they take its
path.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…h argv goes as it is

The socket's fallback for a long node directory lay in the temp
directory itself, so the relay forwarded the host's lines to whatever
listened at that path, and a file there the user could not remove kept
the daemon from starting: the fallback now lies in a directory of the
user's own (0700, checked, never a link or another's), the relay
forwards only to a socket of the user's, and a daemon refused its
socket's place says so in one line and leaves. The task records -- the
agent's frames as they were -- were made with the default mode: tasks/,
each record and its three files are the owner's alone now, as the
host's journal is, and the installer makes the node directory so. A
page read, a digest, a chunk and a checked write ran on the daemon's
one loop, which also answers the heartbeat: they run in a thread now,
writers one at a time under a lock. The ssh argv went to the connection
as one shell-quoted line, which a Windows host's own grammar split into
twelve words: it goes as the argv, which no host reads again. A call on
its way to a node counts as use, so a link is not closed under it as
idle. A drive-relative or extended Windows path (C:file, \\?\C:\...)
no longer reads as a machine's. The daemon's tests skip where there is
no fcntl, the POSIX teardown tests lend a SIGKILL where the platform
has none, the login-shell test checks a directory of its own instead
of /nonexistent, a node is run with nothing importable beyond its
requirements, and remote.py says what does reach a model: a path on
the machine, written the way scp writes it.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…dows path form to this computer

The copy refusal compared a POSIX basename with scp, sftp and rsync, so
scp.exe, SCP.EXE and C:\Windows\System32\OpenSSH\scp.exe reached the
executor and went around the file tools' budgets and read check: the
program a word runs is read with the directory, the case and the suffix
taken off, and on a Windows host the words are read with their
backslashes kept, as the shell there reads them. A drive-relative or
extended Windows path (C:file, \\?\C:\...) was refused by read_file as
a machine nobody registered: it is this computer's, through the public
tools too.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…test sets its own mode

The record opener handed os.fdopen a splatted keyword map, which no
overload of it matches under the type check: the binary and the text
mode are two calls now. The test of another user's directory asserted
the mode mkdir leaves, which is the umask's (0775 under 0002, where it
failed): the test sets 0755 first and asserts it is left so.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…host reads as a drive

On a Windows host one letter before the colon is a drive, so `a:b.txt`
is this computer's there and the test that expects a refusal failed
(reviewed 2026-10-12); a two-letter name reads as a machine's everywhere.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…ed word reads bare on Windows

A fetched file landed under its path on the machine joined as this
computer's path, and a Linux file name may hold backslashes: on a
Windows host `..\` inside one name read as two steps up, and the file
landed above the workspace (reviewed 2026-10-12). The path is read as
POSIX now, each component written as one name here (the other
platform's separator, what Windows reserves and control characters as
their percent-codes), and a landing that would not resolve under the
machine's delivery directory is refused before anything is made. On a
Windows host the words of a command kept their quotes, so a quoted
destination or port went unmatched and the command reached the
executor: the quotes that shell takes off are taken off before the
match.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…of a Windows word stays in it

A fetched name's backslash was written as its percent-code and a
percent sign was not, so `a\b.txt` and `a%5Cb.txt` landed as one file
(reviewed 2026-10-12): the percent sign is written out too, and no two
names land as one. On a Windows host the words of a command were read
with the quotes kept and taken off afterwards, so a quoted piece next
to a bare one -- `"root@"host`, `"587"17` -- became two words and the
host or port went unmatched: the words are read with POSIX quoting,
pieces joined as that shell's argv has them, and only the backslash
kept as a path's own.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…equests, not as its last

The test of a link that comes back read the new link's last request as
the attach, and its heartbeat, every 10 ms there, can follow the attach
before the check: it now looks for the attach among the requests
(reproduced on Windows, 2026-10-12).

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
A review date in a source comment is the task's context, not the
code's: the five comments that carried one keep their reason and drop
the day.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
A review date in a source comment is the task's context, not the
code's: the four comments that carried one keep their reason and drop
the day.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…aths land as one name

Two paths on a machine landed under their own names here, and a
filesystem that reads two names as one -- `Report.txt` and `report.txt`
on Windows or macOS, `report.txt` and `report.txt.` on Windows -- put
the second file's bytes under the first's entry. A file now lands under
`pulled/<machine>/<key>/<name>`, the key a digest of its path on the
machine, so two paths never share a landing, and the name beside it
stays readable.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…er stops

A turn on a registered machine ended with this computer: stopping the
gateway cancelled it and closed the agent, and a hard kill left it
running there with nobody to read its answer. Now a run on a machine is
remembered here while it is in flight -- the machine, the task its
daemon runs the agent under, the session, the prompt awaited, the frames
read, the ids used, the agent's handshake -- in an owner-only file under
the raven home, kept current as frames arrive and removed when the turn
ends here. When this process stops, the agents on machines stay with
their daemons: no session/cancel, the connection's close ends the
endpoint only, and the manager leaves the dispatch's record open. At the
next start the runs are picked up: the task attached from the frame last
read, the connection adopted into the pool with ids carried on, the
prompt's answer awaited, the reply built as a turn ended here is, the
record finished, the registry and node claim closed, and the result
announced into the conversation that asked. A machine that cannot be
asked keeps the record for a later start; a run the machine no longer
has, or whose agent ended meanwhile, is announced as the error it is.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
@Handsome-wzw
Handsome-wzw requested a review from LivXue as a code owner October 13, 2026 03:37

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: cover the TUI host, preserve spawn context, and keep resumptions retryable until delivery completes.

I found three merge blockers in the restart/resume handoff.

Coverage: I read git diff github/main...HEAD, separated the head commit from its stacked machine-support prerequisites, followed the startup/shutdown hosts, node attach/replay, ACP request IDs and frame accounting, manager finalization and announcement callers, and relevant commit history. I also checked AGENTS.md, CONTEXT-MAP.md/CONTEXT.md terminology and architecture constraints, backward compatibility, dependency/schema changes, and the test diff for weakened coverage. git diff --check github/main...HEAD is clean.

Verification: after uv sync --extra dev, the change's stated 683-test set passed. uv run pytest -q completed with 29,002 passed, 144 skipped, and 2 failed: test_cli_theme.py::test_bold_accent_renders_styled_not_bare is on an untouched area, while test_subagent_acp_remote.py::test_the_free_probe_shows_the_link_s_state_once_a_dispatch_has_opened_it passed when rerun alone (the isolated rerun was 1 passed plus the same theme failure). None of these results weakens the blockers below.

Comment thread raven/cli/tui_commands.py
Comment thread raven/agent/subagent/resume_remote.py Outdated
Comment thread raven/agent/subagent/resume_remote.py Outdated
…ch the link

The agent's exit reaches the link a beat after the close that asked for
it, and on a slow runner the listing read the link before it did: the
test waits for the link to read idle, as the listing does.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: the three open restart/resume findings still require implementation changes.

The new commit only makes the remote probe test wait for the link's asynchronous agent-exit update. That wait preserves the final idle assertion and is sound, but it does not change or answer any of the three open implementation findings, so they remain blocking on this revision.

Coverage for this revision: I compared 482c704fc035..HEAD, rechecked the affected probe/link caller and the open review record, checked AGENTS.md and the existing CONTEXT architecture constraints, backward compatibility, and that the test was not weakened. git diff --check github/main...HEAD is clean. uv run pytest tests/test_subagent_acp_remote.py tests/test_remote_runs.py tests/test_remote_resume.py tests/test_resume_remote.py tests/test_acp_client_resume.py tests/test_node_client.py tests/test_subagent_manager.py tests/test_rpc_bootstrap.py -q passed: 364 tests.

@LivXue

LivXue commented Oct 13, 2026

Copy link
Copy Markdown
Member

Windows compatibility review: one new test portability finding was reproduced.

Scope: complete 79-file PR at 00c671d9e16c0e796459f557df9331671cd53776, API base 182dacac001e6697b3260da4400d2f9e36ba536c, merge base 829eb28e1db877dffdd3df4f835d707e1f049d3a. The isolated snapshot matched all 4,386 head-tree blobs. Rechecked the full scope, the 19-file delta from the previously reviewed #922 prerequisite, persistence/attach/replay and host lifecycle callers, and all discussions. The other 60 changed files retain the reviewed prerequisite blobs.

P2 - Separate POSIX permission bits from the portable record round-trip test. Reproduced on native Windows. tests/test_remote_runs.py:58 unconditionally requires file mode 0600 and directory mode 0700. The actual file reports st_mode=33206, whose permission bits are 0666, so the first test fails before its read-back assertions. Windows stat/chmod mode bits do not express POSIX owner/group permissions. Keep the persistence assertions on Windows and gate the POSIX-mode assertions appropriately; verify Windows ACLs separately if the owner-only requirement is intended there. Both this test and the record module are absent from the exact API base.

The remaining record tests passed. A separate native probe used the real Tracker/record/pending/forget path under a home containing spaces and non-ASCII characters, then started a fresh uv Python process: it recovered the same complete record, including prompt ID 17, next ID 18 and consumed-frame count 7, and removal left no pending record.

Existing functional findings remain tracked in gloryfromca's TUI lifecycle, spawn metadata, and retry-safe conclusion discussions. Those are not duplicated as new findings here.

Native Windows / Python 3.13.12, fresh run-owned environment from uv sync --frozen --extra dev --dev; UV_NO_SYNC=1, PYTHONUTF8=1, isolated user/Raven/temp directories, and imports checked against this snapshot. Exact commands from its root:

$R = 'C:\Users\livxu\AppData\Local\Temp\raven-windows-review\a0f78a02ffe84d9c83968056559a226b'
uv run pytest tests/test_remote_runs.py -n 0 -x --tb=short --basetemp "$R\checks\934-records\pytest" --junitxml "$R\checks\934-records\junit.xml"
# 1 failed: the permission-bit assertion.
uv run pytest tests/test_remote_runs.py -k 'not written_owner_only' -n 0 -x --tb=short --basetemp "$R\checks\934-record-behavior\pytest" --junitxml "$R\checks\934-record-behavior\junit.xml"
# 7 passed, 1 deselected.
uv run pytest tests/test_remote_resume.py tests/test_resume_remote.py tests/test_acp_client_resume.py -n 0 -x --tb=short --basetemp "$R\checks\934-resume\pytest" --junitxml "$R\checks\934-resume\junit.xml"
# 25 passed.
uv run pytest tests/test_node_client.py tests/test_acp_client_launch.py tests/test_acp_client_teardown.py -n 0 -x --tb=short --basetemp "$R\checks\934-node\pytest" --junitxml "$R\checks\934-node\junit.xml"
# 33 passed, 33 skipped.
uv run pytest tests/test_subagent_manager.py::test_a_dispatch_left_running_on_its_machine_when_the_host_stops_is_neither_finished_nor_announced -n 0 -x --tb=short --basetemp "$R\checks\934-manager\pytest" --junitxml "$R\checks\934-manager\junit.xml"
# 1 passed.
uv run pytest tests/test_ops_remote_path.py tests/test_deliver_files_tool.py tests/test_remote_file_tools.py tests/test_permissions_remote_read.py -n 0 -x --tb=short --basetemp "$R\checks\934-prerequisites\pytest" --junitxml "$R\checks\934-prerequisites\junit.xml"
# 165 passed, 3 skipped.
uv run pytest "$R\test_native_record_restart.py" -s -n 0 -x --tb=short --basetemp "$R\checks\934-native-record\pytest" --junitxml "$R\checks\934-native-record\junit.xml"
# 1 passed.

Total: 232 passed, 36 skipped, 1 failed. The skips require POSIX-shell fixtures. Recovery protocol tests use in-process endpoint/node substitutes; the cross-process probe verifies local persistence only. No live SSH/remote daemon, real host shutdown/restart, Windows ACL audit, or full repository suite was exercised.

Handsome-wzw and others added 3 commits October 13, 2026 15:25
…nly an acp one

A cli, openai or builtin row that named a machine lost the key on load
without a word and ran on this computer, since only the acp config
declared it. Now every kind carries `machine` and runs there its own way:

- cli: the command is started on the machine by its Raven node, in the
  run's directory there (`<remoteCwd>/<handle>`), and read back over the
  link. The prompt reaches it the three ways it does here -- the file is
  written there under the run's `.raven/`, stdin travels on the link as
  the new `agent/stdin` (protocol 4), which also closes it, since the
  node opens a stdin for every agent and a command that reads to the end
  would wait. MCP servers are not lent (the file there names none, the
  reply says so), the files that came with the task are named by
  address, this computer's shell, home and model binding stay here, and
  a cancel or timeout ends the process tree there.
- openai: each turn opens an ssh tunnel to the machine (`-N -L`, held for
  the turn) and posts through it, with `Host` as written and no proxy;
  http only, refused in one sentence before any ssh otherwise;
  `remoteCwd` is refused on write and dropped with a warning on load.
- builtin: declared so a row naming one is told loudly and cleared;
  Raven's own loop runs here and reaches a machine as the main loop does.

`agent_meta` reads the machine off every kind, so the roster tag, the
registry's lane gate and the dispatch agree; the free probe of a cli or
openai row on a machine reaches nothing and says to run a test, which
runs there; `subagents.add` takes any preset but a built-in one, with
`remote_cwd` refused for an openai one; `raven_config` lists the
machines for every kind.

With it, the review's three notes: a machine row's direct-chat
attachments are kept for the backend to name by address and the page's
bullets are spelled the same way, with the comment and docstring saying
so; `raven_config` says exec can look on the machine instead of saying
nothing reaches it; a failed run's announce no longer says its machine
directory was made. `AgentMeta.machine` now says what stays out of a
model's view (the way in) and what does not (a path, as scp writes it),
and the CONTEXT.md Machine, node, Roster and working-directory entries
describe every kind and no longer name the page's add or `deliver_files`.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
…files

# Conflicts:
#	tests/test_subagent_probe.py

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: the three existing lifecycle defects remain, and remote CLI shutdown leaves an untracked orphan.

The earlier three open threads are unchanged in this revision and still stand. The merge also enables CLI and OpenAI rows on registered machines; the remote CLI path adds the shutdown blocker marked inline.

This new blocker meets the post-fourth-round bar: this revision introduces it by enabling CLI machine rows, ordinary gateway/RPC shutdown reaches it whenever such a turn is active, and Raven offers no recovery because it records the turn as cancelled while leaving an untracked process on the machine.

Coverage: I read AGENTS.md/CLAUDE.md and the runtime context map, compared github/main...HEAD plus the delta from the previously reviewed head, traced shutdown/cancellation through the manager, CLI backend, node endpoint, run tracker, and resume scheduler, and checked callers, history, schemas/generated clients, backward compatibility, and whether tests were weakened. git diff --check github/main...HEAD is clean. uv run pytest tests/test_subagent_cli_remote.py tests/test_subagent_acp_remote.py tests/test_remote_runs.py tests/test_resume_remote.py tests/test_acp_client_resume.py tests/test_node_client.py tests/test_node_server.py tests/test_subagent_manager.py tests/test_rpc_bootstrap.py tests/test_rpc_subagents.py tests/test_cli_gateway_commands.py tests/test_cli_tui_commands.py -q passed: 787 tests. At review time GitHub CI had one Ubuntu unit shard failed and another pending; the failed job's log was not yet available while the workflow remained in progress. I also checked the existing Windows portability report without duplicating or re-grading it.

Comment thread raven/agent/subagent/backends/cli_agent.py
…p's index, stay in node_dir

Four notes of the review, each measured on this branch:

- A page read decoded and split the whole file, two C calls that hold
  the interpreter for as long as the file takes (7.5 s on 1 GiB), so a
  node's heartbeat went unanswered and a host's turn stood still.
  `read_file` now reads the file a chunk at a time, counts the line
  breaks `splitlines` knows as the bytes come, holds back what could
  begin a break at a chunk's edge, and decodes only the page's own
  lines; the file is never whole in memory. Same lines, same footer, and
  a byte that is not UTF-8 refuses the page it is on as it refused the
  file.
- In a worker thread a page read and the digest taken beside it could
  land inside a write of the same file and leave with the old page and
  the new digest, which a later write carrying that digest as `expect`
  would take as leave. The whole-file calls now go through one gate:
  readers together, a writer alone, a waiting writer before new readers.
- uv reads no pip configuration, so a machine that reaches only the
  mirror pip is configured with could not be installed. The probe reads
  that index the way pip does (`PIP_INDEX_URL`, then `index-url` in
  pip's files, user over system, install over global), the plan names
  it, and `uv pip install` gets it as `--default-index`.
- The uv fetched for a machine without one wrote a receipt under the
  home directory and was whichever version was current; it is now one
  pinned version, installed unmanaged, which writes no receipt and
  changes no PATH. And `node_dir` is made owner-only only when the
  install makes it: one that was there is the owner's.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: the four open lifecycle defects still have to be fixed.

The new node/filesystem hardening commit does not touch or invalidate any of those four threads, so they remain open and this revision cannot merge yet. I found no additional blocker in the six-file delta.

Coverage this round: the repository rules and runtime context; git diff github/main...HEAD and the complete delta from the previously reviewed head; the changed filesystem, node server, installer, and tests; chunk-boundary behavior, concurrent read/write ordering, installer shell quoting and package-index precedence; callers and history; backward compatibility; the current discussions; and whether tests were weakened. git diff --check github/main...HEAD is clean. uv run pytest tests/test_filesystem_tools.py tests/test_node_install.py tests/test_node_server.py tests/test_subagent_cli_remote.py tests/test_subagent_acp_remote.py tests/test_remote_runs.py tests/test_resume_remote.py tests/test_acp_client_resume.py tests/test_subagent_manager.py tests/test_rpc_bootstrap.py tests/test_rpc_subagents.py tests/test_cli_gateway_commands.py tests/test_cli_tui_commands.py -q passed: 809 tests. GitHub checks were still in progress; all completed checks were passing.

…in pip may keep inside it

pip takes an index with a sign-in in the URL (`https://user:password@host/simple`),
and the plan that names the index is printed before the confirmation, and under
`--yes`, so the password reached the terminal and any transcript of it. The plan
now shows the host and the path only, saying so when a sign-in was left out; uv
still gets the full URL, and a failed step's sentence, which quotes uv's own last
line, is shown the same way.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

Reviewed the exact delta from the prior clean head and reconciled it with the complete github/main...HEAD patch. The only production change moves PickedUp, UnreachableError, and PICKED_UP_NOTE into a Raven-independent leaf; remote_resume re-exports the identical objects for backward compatibility, while the CLI backend imports the leaf and no longer closes the reported module cycle. The previously reviewed lifecycle behavior is unchanged.

Covered the repository rules and canonical context, the four-file diff, callers and import history, public import compatibility, the cycle-budget architecture constraint, and whether tests were weakened. git diff --check passes; the import-cycle guard plus ACP/CLI pickup and resume suites pass 75/75. The direct compatibility check also confirms all three old exports are object-identical to the new leaf exports. No finding remains.

Merge main with EverMind-AI#922 squashed in (b43d78b). The five files that
conflicted -- the acp and cli backends, the node client and the two
stand-in-machine test files -- already carry main's side byte for byte,
since EverMind-AI#922's commits were brought in before; the conflicts are this
branch's own changes beside them, kept whole: the run's record and
pick-up on the acp and cli lanes, the frame count a channel hands over,
the transcript parse moved into a helper the pick-up shares, and the
tests of a run left and picked up. Nothing of main is dropped: what
these files lose against main are lines this branch rewrote on purpose.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
@Handsome-wzw

Copy link
Copy Markdown
Contributor Author

Merged main (b43d78b, #922 squashed in) into this branch as a plain merge commit; no rebase, no force. The five files that conflicted (the acp and cli backends, the node client, the two stand-in-machine test files) carry main's side byte for byte already, since #922's commits were brought in before: the conflicts were this branch's own changes beside them, kept whole, and what these files lose against main are only lines this branch rewrote on purpose (the transcript parse moved into a helper the pick-up shares). The diff is now this change's own. #921 is closed: its content is on main through #922.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

Reviewed the exact merge from the prior clean head after refreshing github/main to b43d78bf0, then re-read the resulting 26-file github/main...HEAD feature diff. The merge tree is byte-identical to the prior clean tree. Its five conflict resolutions correctly retain the feature-side ACP/CLI run tracking, frame handoff count, shared transcript parsing, and pickup tests beside the mainline files-by-address behavior; no mainline behavior is silently dropped.

Covered the repository rules and canonical context, the diff and both-parent history, conflict callers and callees, serialized/backward compatibility, import-cycle and node-transport architecture constraints, and whether tests were weakened. Both git diff --check scopes pass. The import-cycle, ACP/CLI resume, durable-record, node-client, and stand-in-machine suites pass 218/218 at this head. GitHub also reports the exact-head Windows ACP, TUI, lint, repository, and packaging checks green; remaining unit shards were still running when reviewed. No finding remains, and every thread I opened is resolved.

Handsome-wzw and others added 2 commits October 14, 2026 18:07
Retain the reviewed recovery changes while taking main's session undo,
loopback policy and interface wording updates.

Co-authored-by: Codex <noreply@openai.com>
…n, and the turn replays whole

The backend removed the run's record on its machine the moment the
prompt returned, before the manager had announced the reply: a host
killed in between had no way back to the run, and its reply was lost.
And the record's frame count had moved past the answer by then, so a
record left behind would have resumed after it and waited forever.

Now the manager dispatches with `reported_by_caller=True`, the backend
leaves the record behind with the reply, and the manager removes it
once the announce was taken (`remote_runs.forget_dispatch`) -- or, when
the draining scheduler refused the announce, annotates it with the
dispatch's origin and summary for the next start to report. A caller
that takes the reply as its own result (a graph node, a probe, a direct
chat) dispatches without the keyword, and the record goes as before.
The record's frame count stops at the prompt: the turn's own frames are
never checkpointed, so a pick-up replays the turn whole, the chunks
before the answer as much as the answer, and the reply it reports is
complete. The cli lane keeps its record the same way; its pick-up read
the whole output from the start already.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Handsome-wzw added a commit to Handsome-wzw/Raven that referenced this pull request Oct 14, 2026
…ng change

The record of a run on a machine now outlives the reply until the manager's
announce was taken, and the turn is replayed whole on a pick-up (EverMind-AI#934's
branch, 62c5c64); the CONTEXT entry carries both that and the lending.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Handsome-wzw and others added 2 commits October 14, 2026 18:16
Remove the default-off machine gate from dispatch, node access, file
routing, preset addition and configuration messages. Registered machines
work without an environment variable, including when an old launch still
sets the removed variable to zero. Existing SSH, registry and file
permission checks continue to apply.

Exercise the previous missing-agent recovery failure with the real
registry, and cover default dispatch, file routing and permission checks.
Document the former trial switch and the normal startup behavior.

Co-authored-by: Codex <noreply@openai.com>
Combine the machine gate removal with the independently published change
that retains a run until its result is announced and replays the full turn.

Co-authored-by: Codex <noreply@openai.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: preserve the optional backend contract through forwarding wrappers.

Reviewed the new reply-to-announce durability delta and the complete main-to-head change. I covered AGENTS.md/CLAUDE.md and the Runtime vocabulary, the diff and its callers/history, backward compatibility, test weakening, and the backend/manager architecture contract. The record lifetime and whole-turn replay paths are supported by the focused process tests, but the new keyword breaks an existing forwarding-wrapper path, so this revision cannot merge yet.

Verification: the focused remote-run/manager/resume/ACP/CLI/routing suite passed 390/390. The unchanged direct-chat compatibility test fails deterministically at this head, and exact-head CI unit shard 4/4 reports the same failure; shard 3/4 also hit its existing process-timeout/leak case, while that isolated test passed locally. git diff --check passed.

Patch-risk assessment for the immutable github/main...HEAD diff (218b366129bf7161145c849e84989a1af4abff336056565a8bfcfe6cf2dfbf0f) validates as revise: impact high (persistent restart state), regression likelihood high (reproduced contract break), protection partial, recovery managed, confidence high.

Comment thread raven/agent/subagent/manager.py
Handsome-wzw and others added 2 commits October 14, 2026 18:27
… fake behind it declares

Its `**kw` told the manager it takes every keyword, so the manager handed
it the record keyword a run on a machine takes (`reported_by_caller`),
which the fake behind it does not declare, and the queued spawn failed
instead of completing.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
The queued-spawn test wraps the backend with a keyword-forwarding gate.
Match the backend call's result ownership keyword in its fake so the test
reaches the queue completion assertion instead of a signature error.

Co-authored-by: Codex <noreply@openai.com>
@Handsome-wzw

Copy link
Copy Markdown
Contributor Author

Two commits since the last look, both from the review of #947 (which carries this branch):

  • 62c5c64: a run's record outlives its reply until the announce was taken, and a pick-up replays the turn whole. The backends removed the record the moment the prompt or the command returned, before the manager announced, so a host killed in between lost the reply; and the record's frame count had moved past the answer, so a record left behind would have resumed after it. Now the manager dispatches with reported_by_caller=True and removes the record after an announce that was taken (remote_runs.forget_dispatch), or annotates and keeps it after one the draining scheduler refused; Tracker.received stops at the prompt. Tested on the real daemon behind the stand-in ssh and on a registered machine with the code before and after (the before: "nothing was announced: the reply is lost"; with the old record forced to stay, a pick-up that waited out its deadline). Details in the thread on feat(*): lend the host's MCP servers to an acp agent on a machine through its node #947.
  • b184d82: a test stand-in that forwarded every keyword to a fake that declares fewer (the direct chat's gating double), which the new keyword made visible.

The body above carries both.

Handsome-wzw and others added 2 commits October 14, 2026 18:29
Keep the published wrapper fix for the queued direct-chat test unchanged
while adding machine availability without the trial switch.

Co-authored-by: Codex <noreply@openai.com>
… the prompt

Two docstrings still said the daemon replays only what this computer never
read; the record's count now stops at the prompt, so the turn is read again
from its first frame.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The new commit correctly confines the previously failing keyword forwarding to the test-only gating wrapper; it does not weaken the reply-record, whole-turn replay, or manager-announcement assertions. I rechecked the repository rules and Runtime terminology, the full main-to-head diff plus this one-file delta, callers and history, backward compatibility, test weakening, and the manager/backend architecture. The broader custom-route edge is not reachable through the repository's shipped routed implementations and does not meet the late-round blocking bar.

Verification: git diff --check passed; the formerly failing direct-chat test plus the focused routing, manager, record, resume, ACP, and CLI suite passed 391/391. Exact-head GitHub jobs were pending at review time. My prior thread is withdrawn and resolved, and no thread I opened remains outstanding.

The recovery docstrings describe replay from the prompt boundary. Keep
that wording together with the removal of the machine trial switch.

Co-authored-by: Codex <noreply@openai.com>
@Handsome-wzw

Copy link
Copy Markdown
Contributor Author

Removed the machine trial switch at the owner's request in a0babda. The final branch also retains the independently published result-retention fix from 62c5c64 and current main.

What the parameter used to do: RAVEN_DEBUG_MACHINES was a default-off feature gate for trying registered-machine support. It was not a verbose-logging option. Unless enabled, machine-backed agents were excluded from the dispatch table, node and installer access was refused, remote preset destinations were hidden, and the file tools treated remote-looking paths locally. The startup restore path still ran, so an omitted variable could make a pending run look like an agent that had been removed.

Now the variable and every production check are gone. Start Raven normally; a configured machine selects the destination, and an entry without it stays local. A stale RAVEN_DEBUG_MACHINES=0 is ignored. Registration validation, SSH authentication, file permission prompts and concurrency limits continue to apply. Listing entries and capability backfills still do not log into every machine.

Verification:

  • The default-behavior regression selection produced 8 failures before removal, with the enabled-value control passing; all 9 pass afterwards. The recovery tests construct a real AgentRegistry for both ACP and CLI, with the variable absent and with the old value 0.
  • The final combined tree passes 1799 related tests with strict idle accounting. Changed-line coverage for the whole PR is 92.92% (656/706); the removal commit's new executable lines are 2/2 covered. Python lint, types, imports, dependencies, scope gates and both RPC generators pass; five frontend fixture-contract tests pass.
  • On a registered Linux machine, separate dispatch and restart processes reproduce the original loss: start with the switch on, restart without it, and the outcome is an absent-agent error with the record removed. With the change, both an absent variable and old 0 recover the full result and read the output file by address. The absent-variable case also kills the relay before shutdown. Both were repeated after integrating the latest recovery fix.
  • With no variable set, 12 timing/boundary probes and three attachment/command/IPv6 checks pass. They include cancellation during a dead link and 11 concurrent launches allowing 8 and refusing 3. Windows command parsing is simulated, not a native Windows run. The dedicated test tasks, tunnels and remote node directories have been cleaned up.

Checked documentation against the code: CONTEXT.md, config and RPC descriptions, transport/path docstrings, both collaboration guides, the external design notes and this PR description. They now explain normal startup and what the removed parameter previously controlled. The existing CI environment-contamination investigation and the separate CLI-derived-session observation were not changed by this work.

Full-suite follow-up: 31345 passed, 230 skipped, 2 xfailed and 5 failed. Four failures reproduce in a separate current-main checkout (host color rendering, LibreOffice detection and two simulation cases). The fifth exposed a direct-chat test stub that did not accept the newly published reported_by_caller keyword; the published fix in b184d82 is retained unchanged: the wrapper forwards only keywords its fake backend declares. The full suite was not rerun for that test-only adjustment; the complete direct-chat/manager recheck reports 277 passed with strict idle accounting. Full-PR changed-line coverage is 96.18% (679/706).

The final integration also carries the published replay-docstring correction. The two Python files differ only in docstrings from the tested implementation; their executable syntax trees were compared and are identical.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The branch advanced during this review, so this stance covers the current aeaa24432942 tree rather than stopping at the assigned 2052eb3888e1 parent. I rechecked the new default-on registered-machine commit and the merge integrations against current main, along with the replay-docstring delta: removing the former trial switch leaves registry usability, SSH, permission, and local-path safeguards intact, and the updated replay wording matches the prompt-time frame cursor used by pickup.

Covered the repository rules and canonical runtime terms, the new diff and its callers/history, backward compatibility (including an old RAVEN_DEBUG_MACHINES=0), test changes for weakening, and the machine/recovery architecture. git diff --check github/main...HEAD passes. The focused exact-head run passed 1,182 tests with 4 expected skips for unavailable csh/tcsh; GitHub checks are green except the four unit shards still running at review time.

@0xKT

0xKT commented Oct 14, 2026

Copy link
Copy Markdown
Member

Not a blocker. Grading this PR A (mergeable). Two items for awareness before merge; neither blocks.

Severity of this one finding, not a verdict on the pull request. Any blocking findings from this pass are review threads on the changed files; GitHub renders those collapsed, as a file name with no text.

1. Defense-in-depth reduction: the kernel machine-lane gate is removed (by design)

This PR deletes MACHINE_LANE_ENV / machine_lane_enabled() from the kernel (home.py) and all 6 call sites. In the prior design (accepted in #920/#921) that env var was deliberately in the kernel so, in its own words, the model "must not be able to turn the lane on itself". After this PR the heavier machine integrations (persistent acp/cli sub-agents, remote file tools, remote resume) ride only the normal config/exec approval tiers.

I graded this A, not blocking, after checking three things against the merged tree:

  • Not default-on: a fresh install has an empty connection registry, so remote.machine() refuses any id and subagents.add refuses through the same seam; an entry with no machine runs locally. The model cannot add a host (the ops connection-add tool is off by default); an operator must register one first.
  • No new remote-exec path: exec(machine=<id>) already ran a 60s-capped command on an operator-registered machine in base, ungated by the env var (base shell.py:293-296 calls run_on_machine with no machine_lane_enabled() check; the gate at base:311-317 only changes the wording of a refusal message for a typed raw-ssh command). So prompt-injection -> remote-exec on a registered machine pre-dates this PR; this PR graduates the heavier integrations into that same already-reachable trust domain.
  • Highest-risk sub-capability stays with the user: lending credentials/MCP to a machine sub-agent (lend_key) is still touches_sensitive() == True, so in SMART mode it is kept with the user, not routed to the judge.

Flagging it because it is a conscious security-posture change (a kernel gate my own #920/#921 acceptances relied on) and it intersects the open #947 finding (host MCP lent to a machine acp agent). Your call at merge whether a maturing feature should keep a non-model gate.

2. Non-blocking: a failed remote run's error is lost on a draining-refused announce (manager.py:1956)

The CRUX-1 fix (62c5c64) makes the SUCCESS branch keep the run's machine record when _announce_result returns False during scheduler draining (announced = taken is not False, then the finally annotates instead of forgetting). The two ERROR branches (except SubagentActionAbortedError, except Exception) call _announce_result(..., 'error') but do not capture its return, so announced stays True (its init value at 1807) and the finally runs forget_dispatch(task_id) -- removing the record the backend deliberately kept, even when the error-announce was refused by draining.

Result: a remote sub-agent run that fails in the host-shutdown/draining window has its failure neither announced (draining refused it) nor recoverable by resume (record gone); only a status=failed spawn record persists on disk, never re-announced. Narrow (shutdown race) and the failure is still on disk, hence non-blocking. Mirroring the success branch's announced = taken is not False onto the error path would close it.

@gloryfromca

Copy link
Copy Markdown
Member

No blockers; suggestions only, and they are marked inline.

The new draining-error observation changes my prior clean stance to nonblocking. I confirmed that both exception branches discard a false return from _announce_result(), so announced remains true and the finally block forgets the remote run record even though the draining scheduler refused its error announcement. The focused success/refusal and resumed-announcement tests pass 2/2, but they do not exercise this error/refusal branch.

This remains a narrow shutdown race, and the failed spawn record is still durable locally, so it does not hold the merge. The deliberate removal of the machine trial gate does not otherwise change my stance.

@0xKT
0xKT merged commit aa02321 into EverMind-AI:main Oct 14, 2026
26 of 27 checks passed
Handsome-wzw added a commit to Handsome-wzw/Raven that referenced this pull request Oct 15, 2026
…nce, a dead-link cancel

Seven edges of a run on a registered machine, found by the review of EverMind-AI#934 and
by timing probes on the real machine:

- A Raven node killed outright left its agents running with nobody reading
  them, and the host read a bare exit -1 on the attach. The next daemon now
  brings every record left at `running` to `lost` at its start
  (`Daemon.recover`), ending the agent when its pid is still the process it
  was started as, and the task's exit carries a `reason` the host hands on in
  place of the code (`ChannelEndpoint.exited(rc, reason)`). On Linux the agent
  is also tied to the daemon's life (`PR_SET_PDEATHSIG`).
- A failed remote run whose error announce the draining scheduler refused
  lost the failure: the manager's two error branches now read whether the
  announce was taken, as the reply branch does, and the acp backend keeps the
  record of a prompt that went out and failed (the cli backend of a timed-out
  command) for the caller that reports it.
- A cli run picked up after a restart binds its instance to the session id in
  its output, as the dispatch's deferred commit would have (`_bind_picked_up`).
- A cancel while the machine's link is known dead no longer waits out the
  settle budget: the session is unsettled at once and the cancel rides the
  parked endpoint to the next link (`ChannelEndpoint.offline`).
- An openai row's tunnel checks its local port again just before ssh starts
  and gives one taken in between, or one ssh lost the race for, up for
  another; a listener that took the port no longer answers the readiness
  probe in ssh's place.
- A run record that is not JSON is set aside once under a name that says so,
  instead of being warned about at every start.
- A path written as remote that names no registered machine: already refused
  since the debug switch went; a regression test pins it for write_file and
  edit_file.

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Handsome-wzw added a commit that referenced this pull request Oct 15, 2026
…ough its node (#947)

## Summary

Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.

What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.

- The agent on the machine runs the node's own bridge as a stdio MCP
  server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
  --lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
  pumps around a connection to the daemon's socket, no MCP parsed, no
  credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
  holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
  task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
  one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
  travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
  daemon relays and keeps nothing. A stream ends with either end, the
  host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
  connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
  names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
  (`session_mcp_delivered`). The cli lane on a machine still
  withholds its servers (the next change); the openai lane has no tools.

Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Follow-up fix 05eaba6, after the c9de539 review:

- The nine tests the Windows review named now pin the host fact through
one
  fixture, `unix_socket_host` (`tests/conftest.py`), which sets
  `raven.mcp.bridge.unix_sockets_available` to true and re-points every
  `raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
  `asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
  merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
  the base: nothing this change adds or touches fails on a host without
  unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
  stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
  read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
  awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
  stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
  one of its MCP servers, so the later parts of

`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
  passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
  simulation cases, which fail the same way on main on this computer);
  diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
  agent's bridge reached a host-held server and 1 MiB and 8 MiB results
  crossed whole, bridge exit 0; nothing was left on the machine.

Follow-up fix c9de539, after the same-user token finding:

- The daemon no longer uses a token. `agent/launch` mints and writes
none,
  `mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
  identifies the bridge by the connecting process instead (`_peer_pid` +
  `Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
  because identity is the connection the kernel reports, not a readable
  secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
  B that names task A is refused (`the connection is a process of task
  'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
  unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
  process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
  or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
  machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
  test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
  `ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
  an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
  connecting-process identity admits the legitimate bridge end to end on
  real Linux; the cross-task refusal on Linux is the real-OS test above,
  which CI runs there.

Earlier review fixes (4dc5b3e, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de539), so the token-named tests and reproduction
below no
longer describe the current tree.


- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
  strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
  compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
  different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
  remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
  -- 97.94% (381/389 changed executable lines); this fix alone is 100%
  (40/40). Python lint, type, dependency
  and import checks pass; all 10 import contracts hold. The scope checks
  for commit messages, source language and file sizes pass. Both RPC
  generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
  before, the attacker's own task token plus the victim's public grant
  names returns the victim endpoint's marker; after, it returns no data
  and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
  is killed and an ordinary ping reattaches both tasks. A full chunk of
  262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
  With the host Unix socket APIs removed to simulate Windows, the remote
  agent still completes, receives no MCP, explains the host limitation,
  and the host MCP ledger records no extra call. This is not a native
  Windows run. Dedicated test tasks, host endpoints and bridge processes
  are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
  external design notes and this description were checked against the
  implementation. Main's no-debug-switch behavior and full-turn recovery
  documentation are retained.

Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).

Earlier author verification:

- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
  -- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
  and a stdio node naming none, the token written owner-only beside the
  record, a stream opened with the task's token and refused with another
  or for a task not running, refused with no host holding the task,
  refused past 16 per task, one per bridge connection, the pieces of a
  frame crossing both ways as they are and a stranger carrying nothing,
  and every way a stream ends (the bridge closing, the host closing, the
  bridge's connection ending, the host's link ending, the agent ending)
  with the other end told; the bridge as a process against a real daemon
  over its socket: a frame to the host and one back whole, a large frame
  each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
  a wrong token and a missing daemon each refused on stderr with exit 1;
  the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
  stdio server here and answers with its result, the stanza handed to it
  naming the node's bridge, the task, the dispatch and the server and
  nothing of the server's definition or socket, the ledger here showing
  one call, no bridge outliving the dispatch; the reply's note and the
  roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,

`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
  the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
  the node reinstalled with the code before and after):
  A28, before (the
  code of #934's head and its node): the turn closed with "MCP servers
  'probe' were not lent", Claude Code said no tool of that name was
  connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
  in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
  directory finds nothing of the server's command or of this computer's
  socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
  machine's process list, then cancelled: 3 s later no bridge there and
  no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
  all reached the server and came back; an 8 MiB answer: the server here
  produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
  Code itself closed the server on that answer and reported the call
  failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.

## Risk

Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.

The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

#934 (merged); #920 and #922 (merged).

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Handsome-wzw added a commit that referenced this pull request Oct 16, 2026
…nce, a dead-link cancel (#957)

## Summary

Seven edges of a run on a registered machine, left by the review of #934
and by timing probes on the real machine. Each is a fix the host or the
node makes on its own; nothing changes for an entry without `machine`.

- **A Raven node killed outright no longer leaves agents behind or a
bare `exit -1`.** The next daemon brings every record left at `running`
to `lost` at its start (`Daemon.recover`), ends the agent when its pid
is still the process it was started as (group leader, the recorded
command, and the start mark the launch recorded: Linux's `/proc` start
tick, elsewhere `ps -o lstart`), and writes why; a record without the
mark, or a pid whose mark differs, is left alone and recorded as gone.
`agent/exit` and the attach answer carry that `reason`; the host hands
it on in place of the exit code (`ChannelEndpoint.exited(rc, reason)`),
so a cli run says "the Raven node on this machine was restarted while
the agent ran, and the agent is gone" and a picked-up acp run says the
same. On Linux the agent is also tied to the daemon's life
(`PR_SET_PDEATHSIG`, a SIGTERM). Protocol number unchanged: the field is
optional and old nodes never send it.
- **A failed remote run's error survives a draining-refused announce.**
The manager's two error branches read whether the announce was taken, as
the reply branch does (`announced = taken is not False`), and the acp
backend keeps the record of a prompt that went out and failed, the cli
backend that of a timed-out command, for the caller that reports it. The
next start reads how the task ended off the machine and announces the
error.
- **A cli run picked up after a restart binds its instance to the
session id in its output** (`_bind_picked_up`), as the dispatch's
deferred commit would have; with no id in the output the reply carries
the same "not resumable" warning the dispatch would. A `provisioned` id
or a stateless run binds nothing.
- **A cancel while the machine's link is known dead returns at once.**
`_cancel_turn` reads `ChannelEndpoint.offline` (parked, or the link not
alive), marks the session unsettled and leaves the held `session/cancel`
for the next link, instead of waiting the settle budget for a settlement
that cannot arrive. In the gap before the heartbeat notices nothing
changes.
- **An openai row's tunnel gives up a taken local port for another.**
The port is checked again just before ssh starts (`_still_free`), and
ssh's own "Address already in use" / "Could not request local
forwarding" ends in another try (`_TUNNEL_TRIES` = 3) rather than in a
request sent to whoever took the port; each attempt reads only what ssh
wrote in its own run of the log.
- **A run record that is not JSON, or not UTF-8, is set aside once**
(`<name>.json.unreadable-<ts>`), named in the log that once, instead of
being warned about at every start; a record a later version wrote is
left alone as before. (The first revision read the record as text and
let a `UnicodeDecodeError` out of the whole scan, found by the Windows
review; the record is read as bytes now and decoded by `json.loads`, so
a decoding failure is set aside like a JSON one, and one bad record
never stops the others from being read.)
- **A path written as remote that names no registered machine** is
refused in one sentence for `write_file` and `edit_file` too, never
taken for a relative path here. This has been the case since the debug
switch went (#934); a regression test pins it.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Every fix has a test that failed on the old code first.

- `.venv/bin/python -m pytest -n 6 -q tests/test_node_server.py
tests/test_node_client.py tests/test_remote_runs.py
tests/test_subagent_manager.py tests/test_subagent_cli_remote.py
tests/test_remote_file_tools.py tests/test_subagent_acp_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_subagent_acp.py` - all green on macOS (the Linux-only pact
test skipped there); the last run of the four fast files 331 passed, cli
remote 52, remote file tools 24, the acp remote recovery tests 3
- Full suite: 31577 passed, 234 skipped, 2 xfailed in 6 min 14 s; the 3
failures are the ones `main` has on this machine (a host without
LibreOffice, two simulation records). Changed-line coverage: 98.18% of
165 executable changed lines (`scripts/coverage_gate.py diff --base-ref
origin/main`); the 3 uncovered lines are the Linux-only `prctl` import
block, which CI's Linux shards run.
- `ruff check` / `ruff format --check`, `ty check` on the changed
modules, `lint-imports` (10 contracts kept),
`scripts/check_commit_file.py`, `scripts/check_source_language.py`,
`scripts/check_large_files.py`: clean.
- Real machine (ravenx2, an isolated registry copy and Raven home per
tree; the node of each tree installed under its own `node_dir`; the same
probe script on this branch and on `main`):

| Probe | `main` | this branch |
|---|---|---|
| P1 `kill -9` the daemon under a running cli turn | 4.9 s after the
kill: `CLI agent 'x' exited -1`; the loop the daemon had started was
still running (1 left) | 5.1 s: `CLI agent 'x': the Raven node on this
machine was restarted while the agent ran, and the agent is gone`; 0
left; a new daemon pid |
| P2 an acp prompt that fails after it went out, `reported_by_caller` |
the prompt failed (`exit 5`); no record left | record kept; the next
start announced the error with `exit 5` and removed the record |
| P3 cancel while the link is known dead; the cancel on the next link |
the cancel returned after 2.35-2.46 s, held until the link was back (a 2
s backoff in the probe; up to the 5 s budget in production); it landed
on the next link (task frames 3->4) | 0.00 s; landed the same way (3->4
within 0.1 s of the restore) |
| P4 the chosen local tunnel port is taken by a listener | the tunnel
used the taken port; the request timed out against the listener that
held it | the next port; HTTP 200 through the machine |
| P6 a derived-id cli create left running, picked up by the next start |
picked up ok, no binding (`commits=[]`) | picked up ok, the handle bound
to `edge-123` |

After the merge of `main` (which brought #947's MCP streams, #953's
host-death sweep and #963's install fixes into the same files): the
related groups (node server/client/bridge, remote runs, resume, cli
remote, acp remote, manager, remote file tools, acp client, import-cycle
budget) 1006 passed; diff coverage 97.30% of 185 changed lines (the 5
uncovered are the Linux-only `prctl` lines, run by CI's Linux shards);
the real-machine probes re-run on the merged tree: 8/8 PASS (P1 5.2 s, 0
orphans; P2 record kept and the error announced; P3 0.00 s and the
cancel landed on the next link; P4 the next port, HTTP 200; P6 bound to
`edge-123`).

Docs checked against the code before pushing: CONTEXT.md's Raven node
entry (records of failures, set-aside records, the daemon's recovery and
the `reason`), the module docstrings of `remote.tunnel`,
`remote_runs.pending`, `Daemon.recover`, `_still_ours`, `_death_pact`,
`ChannelEndpoint.exited/offline`, `_bind_picked_up`,
`_track_on_machine`, the `agent/exit` comment in `protocol.py`, and the
CHANGELOG entry.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered - the daemon ends only a pid that is
still the process it started: group leader of its own group, running the
recorded command, and started when the record says (`born`, written at
the launch); a reused pid fails the last mark even for the same program
and is left alone, the record then saying the agent is gone. No new
network surface; the `reason` is the node's fixed sentence, never user
or agent text.
- [x] Backward compatibility considered - `agent/exit`'s `reason` is
optional, protocol number unchanged; a host on this branch reads an old
node's exits as before, an old host ignores the field. Records written
before this branch read unchanged.
- [x] Rollback path is clear for risky changes - a plain revert; nothing
is migrated.

## Related Issues

Follow-up to #934 (the two non-blocking notes of its review) and to the
timing probes run on #921/#922/#934.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
LivXue pushed a commit that referenced this pull request Oct 16, 2026
)

## Summary

The two code notes LivXue left on #957 about `_bind_picked_up` (the
pick-up of a cli create left running on a registered machine), and the
two comments the same review found stale.

- **A stateless run binds nothing at a pick-up.** The gate read `not
run.handle`, which never fires: every run carries a handle (`instance`
or `task_id`), and a stateless entry may declare `idSource: derived`
too, which the dispatch path reads nothing from and binds nothing with.
The pick-up now reads the record's `stateful`, the flag the dispatch
wrote, so a stateless derived run is picked up with no binding and no
"not resumable" warning, as `_attempt` leaves it.
- **A binding a dispatch made while the pick-up waited is kept.** The
pick-up committed without the handle's lock and without a lookup; a
dispatch to the same named instance during the window looked up, found
nothing, created and bound, and the pick-up then overwrote that with the
picked-up create's session, moving the instance back to a thread the
newer turn never saw. `_bind_picked_up` now takes `hold_handle` around a
lookup and the commit, as `run` -> `_run_stateful` does, and keeps any
binding it finds (with no warning: the instance is resumable through
it).
- **The binding is one atomic step, for both id sources, and the pick-up
waits for no turn** (the review's two notes on this PR's first commit).
`InstanceRegistry.bind_if_absent` binds a handle only where nothing is
bound, under the registry's own lock, and returns what the handle is
bound to after; the cli pick-up (`derived`) and
`resume_remote._conclude` (the minted id of the default `provisioned`
entry, which had the same race) both use it. The pick-up takes no
`hold_handle`: the dispatch holds that across its whole turn with no
timeout, and the pick-ups run one after another, so waiting there would
have held back every run picked up after it. The dispatch commits last
whichever way the two land, so the handle ends on its session in every
interleaving.
- **Two comments in `manager.py`** (`announced = True` and the
`finally`) said the kept record was a refused reply's only; since #957 a
refused failure announce keeps it too, and the comments say so now.
CHANGELOG's #957 entry gained the same clause.

Thread links:
#957 (comment)
(stateless gate),
#957 (comment)
(lock and lookup),
#957 (comment)
(comments). The third note on that PR, the stderr replay, is #977.

Follow-up commit, after the review of this PR:

- **A left turn that resumed the session its handle is bound to closes
its row.** `bind_if_absent` found the handle bound to that very id and
wrote nothing, so the `running` the manager put on the row when the turn
began stayed and the instance read as interrupted after a reply that was
announced (the merge base's `commit` rebuilt the row and dropped it). It
now rebuilds a row bound to this very id too, as a binding alone; a row
bound to another id is still that turn's.
- **The pick-up finishes its own record.** `_spawn_record` took the
latest running record of the agent and handle, so a newer turn to the
same named instance, which opens its record as its dispatch starts,
could have the left run's reply written into it and its node closed. The
record the manager named when it left the run (`node_id` in the run's
origin) is taken first; the latest running is the guess only for a run a
host killed outright left unannotated.

Thread links:
#979 (comment)
(the row),
#979 (comment)
(the record).

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Each fix has a test that failed first on the tree before it: the pick-up
binds the id its output names; a stateless derived run binds nothing and
warns of nothing; a newer binding is kept; the interleaving itself
against the real `InstanceRegistry` (the pick-up completes while a
dispatch still holds the handle, and the handle ends on the dispatch's
commit); `bind_if_absent` binds once and reads a status row or another
kind as absent; `_conclude` keeps a newer binding for a minted id. The
first three fail on `main`; the last three fail on this PR's first
commit.

- `.venv/bin/python -m pytest -n 6 -q <the 21 test files that touch the
cli backend, hold_handle, the instance registry, the remote run records
or the pick-up>` - 1842 passed; changed-line coverage 100% (23/23,
`scripts/coverage_gate.py diff --base-ref origin/main`). Full suite:
32047 passed, 174 skipped, 2 xfailed in 3 min 50 s; the 3 failures are
the ones `main` has on this machine (a host without LibreOffice, two
simulation records).
- `ruff check` / `ruff format --check`, `ty check` on the two changed
modules, `lint-imports` (10 contracts kept),
`scripts/check_commit_file.py`, `scripts/check_source_language.py`,
`scripts/check_large_files.py`, `pre-commit`: clean.
- Real machine (ravenx2, isolated registry copies and Raven homes, this
branch's node and `main`'s node under their own `node_dir`; the same
script on both trees):

| Probe | `main` | this branch |
|---|---|---|
| F3: a named derived create left running; the next start picks it up; a
dispatch to the same instance runs during the pick-up; then one more
turn | the pick-up overwrote the dispatch's binding with its own
session, and the next turn resumed the old session (`resumed
first-<pid1>`) | the dispatch's binding kept, and the next turn resumed
it (`resumed first-<pid2>`); the pick-up's own session left unbound |
| F2: a stateless derived create left running, picked up by the next
start | picked up ok, and bound the handle to `s-<pid>` | picked up ok,
nothing bound, no warning |
| F1: a provisioned (default) named create left running; a dispatch to
the same instance during the pick-up; then one more turn | `_conclude`
bound the minted id over the dispatch's, and the next turn resumed the
minted session | the dispatch's session kept, and the next turn resumed
it; the minted id left unbound |
| F2T: a dispatch holding the handle for ~25 s while the left run's
command had already ended (this PR's first commit vs this head) | the
pick-up's announce came at +25.4 s, only when the newer turn released
the handle at +25.4 s | the pick-up announced at +5.0 s (as soon as its
command's output was read) while the newer turn ended at +25.4 s; the
handle ended on the newer turn's session |

Docs checked against the code before pushing: the `_bind_picked_up`,
`bind_if_absent` and `_track_on_machine` docstrings, the comment in
`_conclude`, the two `manager.py` comments, CONTEXT.md's Raven node
entry (unchanged: it describes the pick-up without the binding detail),
and the CHANGELOG entry.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

- Follow-up commit 1f4fa74: the three tests above fail on fc57b79
(status `running` kept; the newer
record finished) and pass now; `tests/test_resume_remote.py`,
`tests/test_subagent_cli_remote.py`,
`tests/test_subagent_acp_remote.py`, `tests/test_subagent_manager.py`,
`tests/test_remote_runs.py`,
`tests/test_remote_resume.py`: 372 passed. Full suite on the merged
tree: 32061 passed, 173 skipped,
3 failed (the same 3 that fail on main on this computer: LibreOffice and
two simulation cases). Diff
coverage 93.18% (41/44). ruff, ty, lint-imports and the three range
checks on `origin/main..HEAD` clean.
  main merged in first (#964, #970, #980), no conflicts.

## Risk

- [x] Security impact considered - none: the pick-up binds strictly less
than before (never for a stateless run, never over a binding already
made), through one step under the registry's own lock.
- [x] Backward compatibility considered - records written before this
change carry `stateful` already (#934); no format change.
- [x] Rollback path is clear for risky changes - a plain revert.

## Related Issues

Follow-up to #957 (the three nonblocking notes of its review).

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants