feat: cache-stable memory snapshot as first message of history

[caos30]

Problem

The system prompt sits before the message history in the request payload, so injecting memory blocks into the system prompt breaks the provider prompt cache in two ways:

• the ms-precision timestamps in <memory_metadata> change every request (<memory_metadata> breaks KV-cache / prompt-cache by injecting ms-precision timestamp every turn #24)
• any memory edit during a session rewrites the system prompt and invalidates the cache for the entire history that follows it — including the multi-million-token sessions where cache matters most (System prompt injection busts Anthropic prompt cache on every turn #8)

This plugin encourages frequent memory updates, which makes the problem worse the more it is used as intended.

Approach

Inject the memory blocks as a synthetic first message of the conversation (experimental.chat.messages.transform) instead of the system prompt, and freeze them per session:

• Once rendered, the exact same bytes are re-injected on every request. The system prompt and history stay a stable cache prefix for the whole session, even while memory is being edited.
• In-session edits remain visible to the model as natural amendments in the history — a coherent timeline: the model still sees what a block said when the session started, and then sees it being updated. (Re-injecting the latest state every turn, as the current system-prompt approach effectively does, rewrites the past the model already narrated.)
• A versioned HTML-comment marker (<!-- opencode-agent-memory:snapshot:1 -->) lets the plugin recognise its own injection and never clobber messages injected by other plugins.
• Snapshots persist to ~/.local/share/opencode/agent-memory/snapshots/<sessionID>.xml (atomic writes, 30-day purge) so they survive opencode/VM restarts byte-identically — a restart does not bust the cache either. In-memory Map is only an L1 cache.
• session.compacted invalidates the snapshot so post-compaction requests render a fresh one.
• The optional journal note stays in the system prompt (static text, cache-safe).

Validation

Real 8-turn session (4 user / 4 agent, 80 API requests, ~10.9M tokens) against a DeepSeek usage panel:

Cache hit	Cache miss	Hit/input
9,476,480	126,870	98.2%

The cache prefix never broke across: memory block edits mid-session, AGENTS.md edits and project skill edits. Before this change the same workload measured ~10% cache hit.

Unit tests cover freeze/re-use, simulated restart (new store instance, same disk → no re-render), invalidation and purge. bun test: 32 pass. tsc --noEmit: clean.

Refs #8, #24. This is a larger but more complete alternative to #25 (which fixes injection placement only); if this lands, #25 can be closed.

[assert-app]

Review on Assert →

2 clusters identified

[caos30]

My DeepSeek test today: $0.13 USD for almost 11 million tokens! There are two bars because it took me about half an hour to implement a fix in the app I’m working on. As you can see, each bar represents roughly 5 million tokens. Total cache hit: 98,2% !! wowww 🤩

Most of the cache misses are in the first bar, since it includes loading the system context, skills, tools, and memory blocks. 😊

Honestly, I can now experiment with models more intensively without needing a fixed subscription plan with token quotas.