Background work should not lock the composer — and "send during a background phase" needs a real answer (#528 part B)

[edspencer]

Follow-up to #528. Part A (#662) made Stop work during a chat's background phase. This is the other half, and it is the one users hit daily.

The problem

When a turn launches background work — a dev server, a Monitor, a run_in_background sub-agent — herdctl's reaper deliberately holds the session open (decideReap keepAlives while backgroundTasks > 0). That is correct and intended. But Paddock renders that stretch as streaming: true, and streaming gates the composer:

• ChatPane.tsx:1023 — Enter routes to queue, not send
• ChatPane.tsx:1374 — placeholder becomes "Queue a message to send next…"
• ChatPane.tsx:1420 — the button is Stop, not Send
• ChatPane.tsx:1450 — hint reads "Enter to queue"

So a chat with a healthy long-running dev server is unusable for conversation until that server exits. Since #662, Stop works — but Stop means end the session, which kills the very work you wanted. The user's only options are "queue and wait" or "destroy it". Neither is right.

Note the codebase already knows this is wrong. useSubagentActivity.ts:123-127:

"The robust fix is server-side — chat:active should not report running:false while background sub-agents are still in flight."

…which is the same seam from the other direction.

The easy part (do not mistake this for the fix)

The client cannot currently tell a background turn from a live model turn. ChatActiveMessage (ws-protocol.ts:538) carries {projectSlug, sessionId, jobId, running, startedAt?} and nothing else. Ironically #662 removed the last accidental tell by minting a synthetic jobId where there used to be null.

Adding the signal is mechanical, and startedAt (#787) is the exact precedent to copy:

1. Turn.background (session-hub.ts:63) → ActiveInfo.background (:114) → activeFrame (ws.ts:269) → ChatActiveMessage.background?: boolean — optional on the wire, so an older client still parses the frame.
2. Set it in makeBackgroundTurnSink's ensureTurn (ws-turn.ts:303) — the single place a background hub turn is created.
3. Reconnect and replay come free: activeInfo (ws.ts:523, subscribe) and runningSessions (ws.ts:1463, on-connect snapshot) both build ActiveInfo from Turn. No buffered-frame change needed, because the state lives on chat:active, not on content frames.

Client-side, split streaming into streaming (live model turn — keeps every gate above) and backgroundRunning (drives a strip, gates nothing).

Presentation is already solved too. RunningSubagents.tsx is a docked strip above the composer that reports live background work without touching the composer at all — the exact shape needed. FleetReadout (#787) would also get a background label for free.

The actual hard part

Unlocking the composer exposes two real defects. Neither is hypothetical.

1. Starting a foreground turn silently evicts the background turn

ws.ts:781 calls hub.startTurn(slug, socket, sessionId) → session-hub.ts:348:

const prev = this.bySession.get(sessionId);
if (prev && prev !== turn) this.evict(prev);   // ← the background turn

evict only drops the map entry; it leaves prev.running === true. Consequences:

• hub.sessionForJob(bgJobId) returns null, so a Stop of the background job can no longer be tagged reason: "stop" (queue hand-back breaks). The reap itself still works, since herdctl.cancel routes via backgroundTurns.
• Worse: when the background stream later ends, onDone (ws-turn.ts:377) emits chat:complete for that same sessionId and calls turn.end() → chat:active{running:false} — which clears streaming on the user's brand-new, still-running turn. The composer unlocks mid-turn and the UI lies.

The hub needs to either hold two turns per session, or the background turn must be deliberately ended/detached when a foreground turn starts.

2. A send during the background phase stalls for up to five minutes

chat:send → chatSession → openChatSession({resume, manageLifecycle: true}). herdctl's #403 collision guard (job-control.js:346) sees the session is still live and calls deferResumeUntilReaped, which waits on whenSessionReaped up to DEFAULT_RESUME_DEFER_TIMEOUT_MS — 5 minutes (job-control.js:25). Paddock never passes resumeDeferTimeoutMs.

So: it is safe (no self-interrupt) while blocked — but it is invisible and unstoppable. hub.startTurn fires chat:active{running:true} immediately, while onJobCreated only fires after openChatSession returns. The user sees "working…" with a Stop that takes the deferred-cancel path and never fires, for up to five minutes, with zero frames. After the ceiling it resumes anyway and genuinely collides — the [Request interrupted by user] class. Background work that outlives 5 minutes is the normal case, not the edge.

Proposed solution

Send into the live session instead of resuming it.

During a background phase the session is already open and idle. RuntimeSession.send(text) is exactly the primitive for this — "send a user turn into the session". Routing chat:send there rather than through openChatSession(resume:):

• Sidesteps blocker 2 entirely. No second claude process, so the Break up oversized source files (>1000 LOC): ws.ts, ChatPane.tsx, routes.ts, and 5 more #403 guard never engages and there is nothing to wait for.
• Sidesteps blocker 1 too. The reply arrives on the same stream the background sink is already consuming, so there is no second hub turn to collide with the first — no eviction, no premature running:false.
• Needs no @herdctl/core change. send() is already on the interface.

The one thing standing in the way is that Paddock throws the handle away: chatSession does liveSessions.delete(turnId) in its finally (herdctl.ts:1393), and consumeBackgroundTurns receives only the iterator, not the session. So the work is: retain the RuntimeSession for the background phase (a natural sibling to the backgroundTurns map #662 added), and route chat:send to it when one exists.

Known gaps in this proposal — not yet designed

Being explicit about what I have not worked out, since these are where it could still fall down:

• Per-turn model override. ensureAgentModel re-registers the keeper agent before a turn; session.send() bypasses that, so changing the model mid-background-phase would not take effect. May need setModel() on the session (which exists) instead.
• Attachments, injected MCP servers, provenance are all wired on the openChatSession path. Which of them a send()-path turn still needs is unresolved.
• Rendering. The user's turn would arrive through makeBackgroundTurnSink, which currently groups the whole stretch into one hub turn and filters sidechain steps. A user turn inside a "background" turn needs a UI decision.
• Whether streaming should flip true for that turn (it is a live model turn now, on a session that is also holding background work) — probably yes, with backgroundRunning staying true independently.

Rejected alternatives

Just add the flag and unlock the composer. Actively broken — both blockers above fire immediately. The flag is necessary but nowhere near sufficient.

Auto-reap the session on send (make "send" an implicit Stop-then-send). Simple, no herdctl involvement, resolves both blockers. Rejected: it makes sending a message destructive. The user types a question and their dev server dies. The whole point of this issue is that background work is legitimate.

Pass a short resumeDeferTimeoutMs and surface "waiting for background work" as a first-class UI state. Honest, and a genuine improvement over the current silent five-minute stall. Rejected as the primary fix because the user still cannot talk to their agent, which is the actual complaint — and shortening the ceiling makes a real collision more likely, not less. Worth keeping as a fallback for any path that must still resume.

Wire recovery.limboTimeoutMs as a backstop. That lever was never implemented and is being deleted in #805; #528 removed its rationale. Not applicable.

Scope note

Parts 1 (the background flag) and 2 (the two blockers) are separable. Landing the flag alone is still worth it — it lets FleetReadout and the sub-agent bar distinguish background work, and it is a prerequisite for everything else — as long as the composer stays locked until the blockers are fixed. Shipping the flag and the unlock together without addressing them would be a regression.