fix: preserve provider prompt cache (date-only timestamps + append at end) - #25
Open
caos30 wants to merge 3 commits into
Open
fix: preserve provider prompt cache (date-only timestamps + append at end)#25caos30 wants to merge 3 commits into
caos30 wants to merge 3 commits into
Conversation
The ms-precision ISO timestamps (`now.toISOString()`, per-block `lastModified`) made the injected XML differ on every request, so the provider KV/prompt cache prefix was invalidated each turn even when no memory block changed. Truncate both to YYYY-MM-DD granularity. Fixes joshuadavidthomas#24, refs joshuadavidthomas#8
Splicing the memory XML at position 1 placed volatile content before nearly the entire cacheable prefix (system prompt, tools, history), so any memory edit busted the whole provider prompt cache. Append at the end instead: edits now only invalidate the small trailing segment. Refs joshuadavidthomas#8
|
2 clusters identified |
Author
|
I’ve been using this OpenCode plugin for almost three months, and I’m in love with it and with how it helps my agent truly learn about me, my environment, and my projects in every session. Thanks for such great work! I submitted this PR because my GLM-5.3 agent and I discovered that my sessions had almost zero cache hits. It was able to diagnose and fix the issue, so I think this is important for other users as well. I usually use “token plans” with hourly or weekly quotas, but when I use PAYG providers such as DeepSeek, Mistral, or NeuralWatt, my purchased credits disappear incredibly quickly! 😔 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix: preserve provider prompt cache (date-only timestamps + append at end)
Problem
The system-prompt injection busts the provider KV/prompt cache prefix on every request, even when no memory block changes. Two independent causes:
renderMemoryMetadata()embedsnow.toISOString()and per-blocklastModifiedat millisecond granularity, so the injected XML differs on every single request. (<memory_metadata> breaks KV-cache / prompt-cache by injecting ms-precision timestamp every turn #24)output.system.splice(1, 0, xml)places the memory XML before nearly the entire cacheable prefix (rest of system prompt, tools, conversation history). Any memory edit invalidates the whole cache, not just the memory segment. (System prompt injection busts Anthropic prompt cache on every turn #8)Users on Anthropic/OpenAI/DeepSeek-style billing see cache hit ratios near zero with this plugin active (measured ~10% where ~90% is expected on long sessions).
Fix
YYYY-MM-DD— the XML is now byte-stable across requests within a day, and across the common case ofmemory_set/memory_replaceedits that don't change the newest-block date.Both changes preserve existing behaviour: same XML content, same blocks, same journal handling (journal note still appended last).
Note: PR #20 addresses cause 1 only; this PR covers both. Happy to rebase/split if preferred.
Testing
bun test: 27 pass / 0 fail<memory_metadata>: no sub-day timestamp present.Fixes #24. Refs #8.