Problem
Each agent is now a persistent chat session, so a session's transcript only grows over time. But the runtime sends the entire prior transcript on every LLM call with no compaction, trimming, or selection.
priorChatMessages in src/execution/chat-task.ts (~L591) is explicit about this:
We replay the full ordered transcript of every prior turn in the same chat session — user/assistant text plus the assistant tool_calls and role:"tool" results
There is no token budgeting, summarization, or relevance filtering anywhere in the context-assembly path (priorChatMessages → runLoop workingMessages). For a long-lived agent chat this means:
- Unbounded cost/latency growth — every turn re-sends every past turn, including verbose tool transcripts (e.g. full
read_skill bodies, large tool results).
- Hard context-window ceiling — eventually the session exceeds the provider's context window and turns start failing instead of degrading gracefully.
- Noise crowds out signal — stale tool output from dozens of turns ago competes for attention with the current task.
What we want
Real context management for persistent chats, covering two related capabilities:
-
Auto-compaction. When a session's transcript crosses a token threshold, summarize older turns into a compact form and persist it, so subsequent turns replay summary + recent turns instead of the full history. Needs to:
- Preserve durable facts the model relies on (created issue IDs, decisions, identity/state changes) — the same things the current comment calls out as the reason for replaying structured tool results.
- Be durable + crash-safe (fits the existing chatMessages persistence model and partial-turn pairing logic).
- Be deterministic enough to test (injectable threshold + summarizer, no wall-clock dependence).
-
Per-call context selection. Decide which prior content to include in the next call rather than always "all of it." Candidate strategies to evaluate:
- Token-budgeted recency window (drop/elide oldest turns first).
- Relevance-ranked inclusion (reuse the existing memory recall/rerank machinery to pull in only the prior turns/tool results relevant to the current message).
- Tool-transcript elision — keep the assistant's intent + a short result digest, drop multi-KB raw tool bodies once they're old.
Scope / open questions
- Threshold + budget: per-provider context window aware? Configurable per instance/agent?
- Where compaction runs: inline before a turn, or a background job after a turn lands?
- Interaction with per-agent memory isolation (ADR
agent-memory-isolation.md) and semantic recall — compaction summaries vs. retained memories shouldn't double-store or conflict.
- Must preserve the defensive tool_calls/tool-result pairing invariant in
priorChatMessages so replay can never produce a provider 400.
- Likely warrants a new ADR for the persistent-chat context model.
Pointers
src/execution/chat-task.ts — priorChatMessages (history replay) and runLoop (workingMessages assembly).src/execution/effective-context.ts — per-agent provider/toolset resolution; natural place to surface a context budget.- Memory recall/rerank pipeline — reusable for relevance-ranked selection.
Problem
Each agent is now a persistent chat session, so a session's transcript only grows over time. But the runtime sends the entire prior transcript on every LLM call with no compaction, trimming, or selection.
priorChatMessagesinsrc/execution/chat-task.ts(~L591) is explicit about this:There is no token budgeting, summarization, or relevance filtering anywhere in the context-assembly path (
priorChatMessages→runLoopworkingMessages). For a long-lived agent chat this means:read_skillbodies, large tool results).What we want
Real context management for persistent chats, covering two related capabilities:
Auto-compaction. When a session's transcript crosses a token threshold, summarize older turns into a compact form and persist it, so subsequent turns replay
summary + recent turnsinstead of the full history. Needs to:Per-call context selection. Decide which prior content to include in the next call rather than always "all of it." Candidate strategies to evaluate:
Scope / open questions
agent-memory-isolation.md) and semantic recall — compaction summaries vs. retained memories shouldn't double-store or conflict.priorChatMessagesso replay can never produce a provider 400.Pointers
src/execution/chat-task.ts—priorChatMessages(history replay) andrunLoop(workingMessagesassembly).src/execution/effective-context.ts— per-agent provider/toolset resolution; natural place to surface a context budget.