fix(server): make Stop work while a chat runs background work (#528)

[edspencer]

Closes #528.

Symptom

A chat sits with the spinner and Stop showing forever. Stop does nothing — no error, no frame, no log line. The composer silently queues anything you type instead of sending it, so the chat is fully inert. Reloading doesn't help: the state is server-authoritative and replays as running. Only restarting the server clears it.

Root cause

Two independent things had to be wrong at once, and both were.

1. The background-phase turn had no cancellable identity.

Once a session-mode turn's primary result lands, the session can stay open — the reaper holds it while the turn's background work runs — and autonomous re-invocation turns keep arriving on the same stream. Paddock renders that stretch through makeBackgroundTurnSink as one hub turn, shown to the user as running, with a Stop button.

The sink opened its turn with hub.startTurn(...) and never called setJobId. That method was being called at only two of the five turn-start sites, and this was one of the three that missed. So every frame and every chat:active carried jobId: null; the client's deferred cancel (#196) waits for a jobId that never arrives; clicking Stop put nothing on the wire at all. That's why it failed silently rather than erroring.

2. There was nothing for it to route to.

HerdctlService.cancel knew two kinds of id — a live turn in liveSessions (→ session.interrupt()) and a batch job (→ cancelJob). The primary turn's liveSessions entry is deleted the moment it returns, so a background-phase id matched neither and fell through to cancelJob(<synthetic uuid>) → JobNotFoundError → false, which the WS layer discards.

interrupt() would have been the wrong primitive even if it could fire: it targets an in-flight model turn, and this session is idle, holding background work.

The fix

• The sink mints a synthetic job id and publishes it via setJobId the moment the turn opens — exactly as the foreground path does via onJobCreated — and registers it against the session the stream belongs to.
• cancel gains a middle branch: a registered background-phase id routes to fleet.reapChatSession() (herdctl#441, released in @herdctl/core 5.31.0). Ending the session ends the stream, which drives the existing consumeBackgroundTurns → onDone → chat:complete + turn.end() unwind. The UI unlocks through the path that already works.
• The registration is dropped when the stream ends, so a late Stop can't reap a session that has since taken that id.

Why the usage-limit case matters

This is easiest to hit on a subscription usage limit, which is how it was reported: sub-agents die on the shared quota, the parent's re-invocation turn dies too — without firing a Stop hook — and the reaper's awaitingTasks flag (already cleared by that turn's activity) means no later background_tasks_changed can do anything. Nothing left can reap the session. herdctl#441 covers that state machine in detail.

It also covers #528's originally reported trigger: a model-authored until loop whose sentinel never arrives, so the background task set never drains.

Deliberately not in scope

#528 also proposes (B) rendering the background phase as a distinct non-blocking state so healthy long-lived background work never locks the composer, and wiring up recovery.limboTimeoutMs. Both are worth doing and both are bigger than this. Flagging one thing found along the way: limboTimeoutMs is parsed, defaulted, and exposed as editable in the Settings UI, but read by nothing — zero consumers in packages/server/src. Anyone who set it expecting it to catch this got nothing. Worth its own issue.

Note that Stop here means end the session, so it does kill healthy in-flight background work too. That's the right semantic for a button labelled Stop, but it's the reason (B) is the real fix for the non-wedged case.

Tests

• herdctl-cancel-routing.test.ts (7) — all three cancel branches, including that a live turn still outranks a reap, and that an unregistered id falls back to cancelJob rather than reaping a reused session.
• background-turn-cancellable.test.ts (7) — the sink publishes a jobId, registers it against the right session, registers once per stretch, unregisters on stream end (and only once), and opens nothing for a sidechain-only stretch. 5 of these 7 fail without the fix — verified by stashing ws-turn.ts and re-running.

The full chain is covered by composition rather than one end-to-end test (an SDK-session background phase isn't practical to drive in-process): force-reap → close() is covered in herdctl#441, stream-end → onDone by the existing resume-drain.test.ts, and onDone → chat:complete here.

Re-verified against the published @herdctl/core@5.31.0 (not a local build): typecheck green, server 1608 passed, web 918 passed, plus the E2E suite.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying paddock with    Cloudflare Pages

Latest commit:	bd8c1e3
Status:	✅  Deploy successful!
Preview URL:	https://2d29d7f5.paddock-7u2.pages.dev
Branch Preview URL:	https://fix-528-stop-during-backgrou.paddock-7u2.pages.dev

View logs