Stamp the effective cache policy onto outgoing requests so prefix protection reaches the wire

[senamakel]

Summary

RunPolicy::cache.protect_prompt_prefix was inert on the wire. The agent loop resolved the effective cache policy (request-level, else the harness RunPolicy::cache) and used it to decide whether to call apply_prompt_cache_breakpoints — but that function, and every provider adapter, read request.cache_policy alone, which the loop never sets. So a host that protected the prefix the documented way (on the run policy) got:

• no prompt_cache_key routing hint in provider_options, ever;
• no cache_control markers from the Anthropic adapter, ever;
• a PromptCacheGuardMiddleware that kept reporting the prefix as protected.

wave2_cache_layout only covered the request-level path (with_cache_policy), which is why this passed.

The loop now stamps the effective policy onto the outgoing request (a clone — the original is what the response-cache key was derived from) before injecting the key and dispatching, so both readers see it. Stamping also runs when the run policy does not protect but a middleware declared cacheable segments: with the companion tinyinference change an adapter treats declared segments as the opt-in, and the run policy must be able to veto. The common path (no protection, no declared prefix) still pays for no clone.

Also pins vendor/tinyinference to tinyhumansai/tinyinference#10, which supplies ModelRequest::wants_prompt_cache_breakpoints, a complete native Anthropic adapter (tools, streaming, three-marker placement), OpenRouter cache_control, and DeepSeek's prompt_cache_hit_tokens. That PR should merge first; this one's submodule pointer then needs no change (the branch commit is on the upstream repo).

API Or Behavior Changes

• No public API change. Behavior: with protect_prompt_prefix on the run policy and a declared cacheable prefix, outgoing requests now carry cache_policy: Some(<effective>) and a derived prompt_cache_key in provider_options; with it off, they carry cache_policy: Some(<effective, protect=false>) when segments are declared, so adapters cannot opt in behind the host's back. The [cache] debug line per model call reports protect_prompt_prefix, whether a key was injected, and the cacheable-segment count.
• PROMPT_CACHE_KEY_OPTION is unchanged; adapters that have no routing-key concept (Anthropic) consume and drop it.

Tests

• cargo fmt --check
• cargo clippy --all-targets -- -D warnings
• cargo clippy --all-targets --all-features -- -D warnings
• cargo build --all-targets
• cargo build --all-targets --all-features
• cargo test — all green
• cargo test --all-features — all green

New: wave2_cache_layout::run_policy_breakpoints drives a real AgentHarness with a request-recording model and a middleware that declares the system prompt as cacheable (the way a host that assembles its own messages does), and pins both directions — a protecting run policy reaches the provider as a stamped policy plus a tap-… prompt_cache_key; an unprotecting one is stamped as the veto and injects nothing.

Documentation

docs/modules/harness/cache.md gains "Where protect_prompt_prefix is read", describing the stamp, the two readers, and the regression.

Summary by CodeRabbit

• Bug Fixes

  • Prompt-cache protection policies are now correctly applied to outgoing model requests.
  • Prompt-cache breakpoints are injected when prefix protection is enabled, while explicitly disabled policies prevent breakpoint injection.
  • Provider adapters now receive the effective cache policy and can emit cache-control markers consistently.

• Documentation

  • Updated cache documentation to explain policy resolution and breakpoint behavior.

• Tests

  • Added coverage for protected and unprotected run-policy scenarios.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-11T19:42:43.149386Z	ee67228	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 203675b7-d74e-4d77-9acf-8632684ad1b0

📥 Commits

Reviewing files that changed from the base of the PR and between 3536708 and ee67228.

📒 Files selected for processing (4)

• crates/tinyagents-harness/src/agent_loop/model_call.rs
• crates/tinyagents-integration-tests/tests/wave2_cache_layout.rs
• docs/modules/harness/cache.md
• vendor/tinyinference

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The harness now stamps inherited cache policy onto outgoing model requests when required. Breakpoint injection and provider adapters read the stamped policy. Integration tests cover protected and unprotected run policies. Documentation describes the policy flow.

Changes

Prompt cache policy propagation

Layer / File(s)	Summary
Stamp effective cache policy crates/tinyagents-harness/src/agent_loop/model_call.rs	The model call path stamps inherited cache policy onto a cloned request when prefix protection or cacheable segments require it.
Validate run-policy breakpoint behavior crates/tinyagents-integration-tests/tests/wave2_cache_layout.rs, docs/modules/harness/cache.md	Integration tests verify breakpoint injection for protected policies and veto behavior for unprotected policies. The cache documentation describes the stamped policy and its readers.

Vendor reference update

Layer / File(s)	Summary
Update vendor reference vendor/tinyinference	The vendored tinyinference reference changes to a new commit.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant AgentHarness
  participant ModelCall
  participant BreakpointLogic
  participant ProviderAdapter
  AgentHarness->>ModelCall: provide run-level CachePolicy
  ModelCall->>ModelCall: stamp effective policy on request clone
  ModelCall->>BreakpointLogic: process stamped request
  BreakpointLogic->>BreakpointLogic: inject prompt_cache_key when protection is enabled
  ModelCall->>ProviderAdapter: send stamped request
  ProviderAdapter->>ProviderAdapter: evaluate wants_prompt_cache_breakpoints()

Loading

Merge Risk: ⚪ Minimal · up to ee672

The updated request policy flow preserves protected and unprotected prompt-cache behavior without an identified merge-blocking risk.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files. (2 skipped: 2…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: stamping the effective cache policy onto outgoing requests so prefix protection reaches provider adapters.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files. (2 skipped: 2 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

---

_{A rabbit reads each line,
The patch grows clear beneath the moon,
Small changes hop in place,
Tests guard the garden path,
Reviews bloom before the dawn.}

_{Comment @coderabbitai help to get the list of available commands.}

[tinysweeper]

tinysweeper found nothing blocking. Approving.

             $0.0043 · 32,418 in / 2,997 out · 3,142 cached (10%) · deepseek/deepseek-v4-flash, z-ai/glm-5.2
critique:    $0.0011 · 12,259 in / 577 out   · 0 cached (0%)      · deepseek/deepseek-v4-flash
security:    $0.0017 · 7,379 in  / 501 out   · 3,142 cached (43%) · deepseek/deepseek-v4-flash, z-ai/glm-5.2
tests:       $0.0007 · 5,108 in  / 1,532 out · 0 cached (0%)      · deepseek/deepseek-v4-flash
description: $0.0005 · 5,916 in  / 63 out    · 0 cached (0%)      · deepseek/deepseek-v4-flash