Find what breaks LLM prompt caching in your project.
audit-prompt-caching helps Codex, Claude Code, and compatible agents trace
cache misses through your request code, provider settings, and routing.
Start with code and configuration. Get findings with evidence and a verification step; add exported payloads or telemetry when available.
Read more about cache and routing design on notevskii.tech and follow engineering notes on Telegram.
Get started · See an example · What it audits · Commands and scenarios
From the project you want to audit, install the skill for your agent with the skills CLI (requires Node.js/npx):
npx skills add https://github.com/sernote/audit-prompt-caching --skill audit-prompt-cachingStart a new agent session in that project and ask:
Use the audit-prompt-caching skill
to audit this project's LLM calls.
Start with code and configuration.
Show findings, evidence, and checks.
Explain when no change is justified
or what evidence is still missing.
Add your symptom and relevant paths. In Codex, you can also invoke the skill as
$audit-prompt-caching. The first-project guide explains
how to check one conclusion and share optional feedback.
Install without Node.js, or choose a custom directory
Use the bundled Bash installer from a local checkout:
git clone --depth 1 https://github.com/sernote/audit-prompt-caching.git
cd audit-prompt-caching
bash install.sh --source-dir . --agent codexUse --agent claude for Claude Code, --agent both for both agents, or
--dir path/to/skills for a custom skills directory. Existing installations
are preserved unless you pass --force to replace them. See
bash install.sh --help for all options.
The local audit helpers need Python 3.10+ and use only the standard library. The installer itself requires Bash; the command above also uses Git.
In the bundled example, a changing timestamp and support-ticket context precede the shared task instructions. Moving that context later preserves all the information and gives repeated requests a longer identical beginning.
The change in block order:
- Request context (changes)
Task instructions (shared)
+ Request context (changes)| Compared request pair | Common UTF-8 prefix |
|---|---|
| Before reordering | 43 bytes |
| After reordering | 254 bytes |
These are measured local text comparisons from a small synthetic example. They show prefix stability; provider eligibility, cache hits, latency and savings still require their own checks.
Reproduce the comparison with Python — no API key needed
Clone this repository, or use your existing checkout. Run from its root:
git clone --depth 1 https://github.com/sernote/audit-prompt-caching.git
cd audit-prompt-caching
python3 audit-prompt-caching/scripts/prefix_stability_check.py --json \
examples/first-audit/before-a.txt examples/first-audit/before-b.txt
python3 audit-prompt-caching/scripts/prefix_stability_check.py --json \
examples/first-audit/after-a.txt examples/first-audit/after-b.txtBoth comparisons intentionally return exit status 1: the complete requests
still differ. Read stable_prefix_bytes in each result. The
walkthrough includes the exact output and
shows how to compare two renders from your own application.
An audit connects its conclusion to the active request path and a verification step. It can produce:
- A finding with evidence: where the prefix changes, usage is lost or miscounted, or routing/cache state needs attention; what to change and how to check the result.
- A supported no-change result: the layout is already appropriate, the workload rarely repeats, or output generation and tools dominate the cost.
- A specific missing measurement: the request, usage field, route or worker observation needed to resolve the question.
For a longer handoff, the report format
keeps applicability, evidence, prefix stability, accounting, routing, economics
and isolation separate. Unproven dimensions remain unknown.
| Your question | What the audit inspects |
|---|---|
| Why do repeated prompts miss? | Request builders, early variable content, serialization and the first prefix divergence |
| Why did an agent become more expensive? | Tool/MCP registries, schemas, history, compaction and per-step configuration changes |
| Can I trust these cached-token numbers? | Actual provider/API path, raw usage fields, wrapper accounting and missing telemetry |
| Why is a warm request still slow? | Prefill vs. decode, client timings, route selection, queue/load signals and retries |
| Why did reuse drop after deployment or scaling? | vLLM/SGLang versions, effective configuration, replica locality, KV pressure, retention and offload |
| Is caching worth changing here? | Prefix length and repeat cadence, input/output costs, cache read/write prices and tenant boundaries |
The skill covers code reviews, agent loops, provider migrations and self-hosted deployment audits. It can also guide an investigation without a repository. See the scenario prompts for concrete starting points.
Provider-specific checks live in selectively loaded references:
- APIs and gateways: OpenAI, Azure OpenAI, Anthropic, Amazon Bedrock, Gemini, OpenRouter.
- Agents and frameworks: tools, MCP and agent loops, Vercel AI SDK, Mastra.
- Self-hosted inference: vLLM, SGLang, routing evidence.
More provider references
DeepSeek · Qwen · Moonshot · MiniMax · Mistral · xAI · YandexGPT · Z.AI · Tencent · Xiaomi
The agent verifies current official documentation before exact claims about model support, cache controls, TTL, pricing or usage fields. Bundled references are a starting point for that check.
| Start here | What it demonstrates |
|---|---|
| Prefix-layout walkthrough | Reorder a rendered prompt and inspect its first divergence |
| Recorded vllm-router observations | A session-ID routing input and HTTP success that does not guarantee a complete stream; real router, synthetic worker, no GPU |
| Usage and ROI examples | Run the helpers on synthetic records and explicit sample prices |
| Script reference | Locate calls, lint payloads, compare prefixes, analyze usage and render a report |
| Routing capture guide | Collect evidence for a self-hosted routing investigation |
The scripts run locally with Python's standard library. The agent uses them as needed; a first audit does not require running every helper.
The routing analyzer is experimental. It consumes a normalized JSONL export and has no bundled native-router-log adapter. End-to-end capture and analysis still need validation on a real deployment. Request/attempt joins and measured worker/client outcomes are necessary before drawing routing conclusions.
Use the skill alongside your existing telemetry: request payloads, provider usage, billing exports, traces and per-replica metrics. It does not capture live traffic by itself. The usage guide describes useful inputs and how the helpers preserve uncertain accounting.
A gateway response-cache hit, a provider prompt-cache read and self-hosted KV reuse are different observations. A stable local prefix alone establishes none of them. Each finding should name the cache layer and the evidence behind it.
Tried it on a project? Share an audit result: what helped, what was unclear, or what evidence was missing. Feedback is optional; keep credentials and private project contents out of public issues.
CONTRIBUTING.md covers tests, package validation and change guidelines. The validation notes distinguish script tests, trigger-dataset checks and agent behavior evaluations.
MIT.