Extend the contract gate to composio_execute, MCP registry, and workflows - #4861
Extend the contract gate to composio_execute, MCP registry, and workflows#4861yh928 wants to merge 10 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds ChangesContract gate
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Model
participant ContractGateMiddleware
participant ContractResolver
participant LateBoundTool
Model->>ContractGateMiddleware: issue gated tool call
ContractGateMiddleware->>ContractResolver: fetch full contract
ContractResolver-->>ContractGateMiddleware: return contract or lookup result
ContractGateMiddleware-->>Model: return contract in tool error
Model->>ContractGateMiddleware: retry call with marker present
ContractGateMiddleware->>LateBoundTool: execute tool
LateBoundTool-->>Model: return tool result
Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6236939f1f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/openhuman/tinyagents/contract_gate.rs (1)
174-184: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winAvoid bulk allocation of tool message texts.
Collecting
m.text()for all tool messages into an intermediateVec<String>forces a peak memory allocation proportional to the total size of all tool messages in the transcript (which can reach megabytes).You can avoid this O(N) memory spike by streaming the iterator directly into
collect_present_keysusingAsRef<str>, evaluating and dropping each message's text one at a time.♻️ Proposed refactor to stream texts without an intermediate `Vec`
Update
refresh_presentto remove the.collect():- fn refresh_present(&self, messages: &[TaMessage]) { - let tool_texts: Vec<String> = messages - .iter() - .filter(|m| matches!(m, TaMessage::Tool(_))) - .map(|m| m.text()) - .collect(); - let keys = collect_present_keys(tool_texts.iter().map(String::as_str)); - if let Ok(mut set) = self.present.lock() { - *set = keys; - } - } + fn refresh_present(&self, messages: &[TaMessage]) { + let keys = collect_present_keys( + messages + .iter() + .filter(|m| matches!(m, TaMessage::Tool(_))) + .map(|m| m.text()), + ); + if let Ok(mut set) = self.present.lock() { + *set = keys; + } + }Then generalize the helper's signature further down the file so it accepts the owned
Stringiterator:-fn collect_present_keys<'a>(texts: impl Iterator<Item = &'a str>) -> HashSet<String> { +fn collect_present_keys(texts: impl Iterator<Item = impl AsRef<str>>) -> HashSet<String> { let mut set = HashSet::new(); for text in texts { - if let Some(rest) = text.strip_prefix(MARKER_OPEN) { + if let Some(rest) = text.as_ref().strip_prefix(MARKER_OPEN) { if let Some(j) = rest.find(']') { set.insert(rest[..j].to_string()); } } } set }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/tinyagents/contract_gate.rs` around lines 174 - 184, Update refresh_present to pass the filtered tool-message text iterator directly to collect_present_keys, removing the intermediate Vec<String> allocation. Generalize collect_present_keys to accept an iterator of items implementing AsRef<str>, while preserving key collection behavior and allowing each generated text value to be dropped after processing.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/openhuman/tinyagents/contract_gate.rs`:
- Around line 478-480: Add a verbose, grep-friendly diagnostic log in the
ContractResult::Unavailable arm of the fetch_contract match before delegating to
next.run(ctx, state, call). Include enough context to identify the target and
transient contract-fetch failure, while preserving the existing execution flow.
---
Nitpick comments:
In `@src/openhuman/tinyagents/contract_gate.rs`:
- Around line 174-184: Update refresh_present to pass the filtered tool-message
text iterator directly to collect_present_keys, removing the intermediate
Vec<String> allocation. Generalize collect_present_keys to accept an iterator of
items implementing AsRef<str>, while preserving key collection behavior and
allowing each generated text value to be dropped after processing.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a6d9c2ae-35b5-4406-854f-586e018ae5f7
📒 Files selected for processing (3)
src/openhuman/tinyagents/contract_gate.rssrc/openhuman/tinyagents/contract_gate_tests.rssrc/openhuman/tinyagents/mod.rs
6236939 to
62d3b81
Compare
62d3b81 to
fad71b4
Compare
fad71b4 to
ef89c16
Compare
YellowSnnowmann
left a comment
There was a problem hiding this comment.
Solid, well-engineered change — the design directly addresses #4853, and deriving contract-presence from a transcript marker (rather than mutable turn state) is genuinely nice: it's correct-by-construction across summarization, microcompact, hard-trim, and durable sub-agent resume, which is strictly better than the "reset seen-state on compaction" the issue asked for. The anti-spoof prefix check and the "all same-batch calls gated" reasoning are right and are tested.
I'm requesting changes on verification gaps, not on the design. Four things I'd want resolved before approving:
1. The core behavior is untested. All 22 tests are pure logic. The network resolvers and — more importantly — the real harness wiring (before_model refreshes presence, then wrap_tool gates, in the actual middleware order from assemble_turn_harness) have no integration coverage. The gated_call_delivers_the_contract_then_the_retry_passes test manually simulates the rescan; it doesn't prove the middleware ordering actually produces that sequence. The top acceptance criterion — "repro gone" — has no automated proof. Please add at least one integration test exercising deliver → retry through assemble_turn_harness.
2. The inventory-count assertions may not run. The updated 16 / 5 counts in tests.rs sit under a comment stating they're "NOT compiled by cargo check --lib … verify/adjust the exact numbers when the test suite actually compiles." Please confirm that block actually executes in CI (link a passing run), or the counts are unverified.
3. UPPER_SNAKE false-positive is a hard-failure mode. looks_like_composio_slug gates on name shape alone. Any non-Composio tool ever named in UPPER_SNAKE_CASE would resolve to Composio(name) → toolkit_from_slug returns None → NotFound → the tool never executes and the model gets a misleading "not a valid Composio action slug" message. This relies on an unenforced lowercase-tool convention. Please add a guard (e.g. check the name against the actual registered Composio action set) or at minimum a comment documenting the invariant.
4. Contracts are delivered as tool errors — confirm they don't trip an error breaker. The NotFound path's own comment references "the repeated-tool-failure breaker." Since Found deliveries are also TaToolResult::error, a turn touching several gated targets emits multiple errors before any retry lands. Please confirm the Found deliveries don't count toward that breaker (or exempt them).
None of these are design objections — CI green + CodeRabbit clean is necessary but doesn't cover any of the above. Happy to re-review once addressed.
🤖 Council Merge Gate — APPROVE (merge blocked by branch protection)The model council review gate returned APPROVE (session However, Action needed (human): resolve the failing required checks / obtain the required review, then merge. A maintainer with admin rights may also merge explicitly. The PR is left OPEN. Posted automatically by the |
Review notes on #4861 (contract gate for #4853) — non-blocking commentReviewed the diff and read Does it address the root cause?Mechanically, yes. The gate short-circuits the first call per late-bound target each turn and hands back the full contract as a tool error, so the retry runs with the real schema in context — the same lever the issue's own diagnostic ("tell it to read the contract first") confirmed works. Verified:
Gaps / risks worth weighing
Test qualityGood — 22 pure-logic tests with real assertions: target extraction for every kind (+ missing-arg pass-through), the core deliver→retry transition, same-batch gating, marker presence across resume/compaction/anti-spoof, the Composio/MCP resolvers (found + not-found + valid-list + empty-server), list capping, and contract rendering. One caveat: the async Bottom lineSolid engineering and a faithful implementation of the issue's chosen design; merge-worthy on mechanism and quality. It does not, on its own, prove the repro is eliminated — that hinges on a live Gmail run and on Composio's real schema carrying the quoting guidance. Suggest a live smoke of the Gmail search repro before/at merge, and noting the deferred |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 69d383f403
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex flagged (tinyhumansai#4861 review) that a delivered contract, returned as an ordinary tool error, flows through the same output pipeline as any tool result — payload summarization, TokenJuice compaction, the 16 KiB result cap. Any of those can rewrite or truncate the body, and because the `[contract-gate:]` presence marker rides in that same body, the next turn could treat a partial contract as "present" and execute with an incomplete schema (or lose the marker and re-deliver forever). Rather than exempt deliveries from that pipeline, embed a fingerprint: the marker now carries an XXH3-64 digest of the exact bytes that follow it — `[contract-gate:<digest>:<key>[,<key>...]]`. On rescan `before_model` re-hashes those bytes and credits the key(s) ONLY when the digest still matches. A summarized / capped / rewritten body fails the check, so the gate re-delivers the full contract (fail-safe) instead of trusting a mutated one. XXH3 is fast and stable across processes (a resumed sub-agent's byte-identical history still verifies); non-cryptographic — it guards accidental mutation, not a forged collision. - new `xxhash-rust` dep (xxh3 feature) - `payload_digest` / `build_marker` assemble+verify the digest; the marker-to-body concatenation is owned by `deliver_body` and `prefix_with_present_marker` so the hashed bytes always equal the post-marker bytes - `present_marker(keys) -> Option<String>` becomes `prefix_with_present_marker(keys, body) -> String`; the three discovery pre-crediting call sites updated (mcp helper now returns keys, not a marker) - `collect_present_keys` parses `<digest>:<keys>` and drops the keys unless the recomputed digest matches - tests rebuilt on the real delivery path, plus new coverage: a truncated body and a rewritten body both fail the digest gate, and a digest-less marker is ignored Addresses the "preserve contract bodies through tool-output rewriting" review point on tinyhumansai#4861. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012BLvp9tiJ2bFUrcY821FY1
…contract leak) Codex flagged (tinyhumansai#4861 review) that the gate's `fetch_workflow` resolved a workflow contract straight from the global registry, while the real `run_workflow` / `describe_workflow` tools enforce the active profile's `skill_allowlist`. A model guessing a scoped-out `workflow_id` would receive that workflow's input contract from the gate — leaking metadata the profile hides. Thread the allowlist to the gate the same identity-from-the-tool way `composio_actions` already flows: `RunWorkflowTool` self-reports its `skill_allowlist` via a new `Tool::workflow_gate_allowlist` trait method, and `assemble_turn_harness` collects it into `ContractGateMiddleware`. `decide` then passes a scoped-out workflow straight through (no contract rendered) so the real tool owns the canonical "not available to the active agent profile" rejection — the gate never reveals such a workflow exists. Checked the other two gated surfaces: Composio (`composio_integrations`) and MCP-registry (`allowed_mcp_servers`) have NO equivalent leak, because their executing tools do not enforce that per-profile scope in the first place (the gate mirrors the absent enforcement, exposing nothing the model couldn't already reach by calling the tool). Only workflows enforce an allowlist the gate bypassed. - new default-None `Tool::workflow_gate_allowlist`; `RunWorkflowTool` overrides it - gate stores `workflow_allowlist`, checks it in `decide` via `workflow_allowed` - tests: scoped-out workflow not gated; allowed workflow still gated; a None allowlist gates every workflow Addresses the "honor workflow profile allowlists before returning contracts" review point on tinyhumansai#4861. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012BLvp9tiJ2bFUrcY821FY1
Reviewer tinyhumansai#1 on tinyhumansai#4861 asked for a test through the real `assemble_turn_harness` middleware ordering (not just unit-level `decide()`), exercising the whole deliver→retry so the transcript-derived presence check is validated end to end. - cfg(test)-only contract-source seam (`test_support::InjectedContracts`): `fetch_contract` first consults an injected map keyed by `GateTarget::key()`, so an integration test resolves a deterministic contract without reaching a live Composio / workflow / MCP source. The middleware is built inside `assemble_turn_harness`, so the test never holds it — the seam is global, serial-locked, and RAII-cleared. String-valued (delivered as `ContractResult::Found`) so the private `ContractResult` keeps its visibility. The whole module is `#[cfg(test)]` and absent from production builds. - integration test in `tinyagents::tests`: a mock provider issues the same gated `composio_execute` call twice (gated first attempt, then retry) with a recording stub tool, driven through `run_turn_via_tinyagents_shared` → `assemble_turn_harness`. The single assertion `executed == 1` proves BOTH halves: the gate blocked the first attempt (else 2) AND the delivered contract round-tripped through `before_model` so the retry passed (else 0, looping). Also asserts the delivery precedes the execution in the transcript. `openhuman::tinyagents` suite green (152, incl. the `adapter_inventory` middleware-count assertions — the seam adds no middleware); fmt clean. Addresses reviewer integration-test point tinyhumansai#1 on tinyhumansai#4861. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy
Codex flagged (tinyhumansai#4861 review) that a delivered contract, returned as an ordinary tool error, flows through the same output pipeline as any tool result — payload summarization, TokenJuice compaction, the 16 KiB result cap. Any of those can rewrite or truncate the body, and because the `[contract-gate:]` presence marker rides in that same body, the next turn could treat a partial contract as "present" and execute with an incomplete schema (or lose the marker and re-deliver forever). Rather than exempt deliveries from that pipeline, embed a fingerprint: the marker now carries an XXH3-64 digest of the exact bytes that follow it — `[contract-gate:<digest>:<key>[,<key>...]]`. On rescan `before_model` re-hashes those bytes and credits the key(s) ONLY when the digest still matches. A summarized / capped / rewritten body fails the check, so the gate re-delivers the full contract (fail-safe) instead of trusting a mutated one. XXH3 is fast and stable across processes (a resumed sub-agent's byte-identical history still verifies); non-cryptographic — it guards accidental mutation, not a forged collision. - new `xxhash-rust` dep (xxh3 feature) - `payload_digest` / `build_marker` assemble+verify the digest; the marker-to-body concatenation is owned by `deliver_body` and `prefix_with_present_marker` so the hashed bytes always equal the post-marker bytes - `present_marker(keys) -> Option<String>` becomes `prefix_with_present_marker(keys, body) -> String`; the three discovery pre-crediting call sites updated (mcp helper now returns keys, not a marker) - `collect_present_keys` parses `<digest>:<keys>` and drops the keys unless the recomputed digest matches - tests rebuilt on the real delivery path, plus new coverage: a truncated body and a rewritten body both fail the digest gate, and a digest-less marker is ignored Addresses the "preserve contract bodies through tool-output rewriting" review point on tinyhumansai#4861. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012BLvp9tiJ2bFUrcY821FY1
98515dd to
51aec00
Compare
…contract leak) Codex flagged (tinyhumansai#4861 review) that the gate's `fetch_workflow` resolved a workflow contract straight from the global registry, while the real `run_workflow` / `describe_workflow` tools enforce the active profile's `skill_allowlist`. A model guessing a scoped-out `workflow_id` would receive that workflow's input contract from the gate — leaking metadata the profile hides. Thread the allowlist to the gate the same identity-from-the-tool way `composio_actions` already flows: `RunWorkflowTool` self-reports its `skill_allowlist` via a new `Tool::workflow_gate_allowlist` trait method, and `assemble_turn_harness` collects it into `ContractGateMiddleware`. `decide` then passes a scoped-out workflow straight through (no contract rendered) so the real tool owns the canonical "not available to the active agent profile" rejection — the gate never reveals such a workflow exists. Checked the other two gated surfaces: Composio (`composio_integrations`) and MCP-registry (`allowed_mcp_servers`) have NO equivalent leak, because their executing tools do not enforce that per-profile scope in the first place (the gate mirrors the absent enforcement, exposing nothing the model couldn't already reach by calling the tool). Only workflows enforce an allowlist the gate bypassed. - new default-None `Tool::workflow_gate_allowlist`; `RunWorkflowTool` overrides it - gate stores `workflow_allowlist`, checks it in `decide` via `workflow_allowed` - tests: scoped-out workflow not gated; allowed workflow still gated; a None allowlist gates every workflow Addresses the "honor workflow profile allowlists before returning contracts" review point on tinyhumansai#4861. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012BLvp9tiJ2bFUrcY821FY1
8ef5eb1 to
aa16823
Compare
|
Pushed Rust Quality was red for a lockfile, not for code.
The two codex threads are both already addressed on this head, which is why they show as outdated:
Tests: |
|
Pushed What was red and why it surfaced now. The gap, closed. Both text-mode serializers wrapped every result in Carried producer to renderer: The presence rescan now reads host-written rows (tool and user) rather than tool rows alone: a text-mode turn has no tool role to send, so an unframed delivery lands in a user turn, and scanning tool rows would have credited nothing after a durable rebuild. Assistant rows stay excluded — the model must not be able to credit its own echo of the marker. Tests: the marker test now asserts both serializers deliver the payload unchanged; added |
fe2a3c8 to
b9f71e7
Compare
|
@YellowSnnowmann — thanks for the review; it was the one that named verification rather than design, and point 1 was right. Pushed 1. The core behavior is untested — fixed. 2. Inventory-count assertions — obsolete. Upstream #5143 (provider cutover + lifecycle consolidation) removed 3. UPPER_SNAKE false-positive — retired by the redesign. 4. Deliveries vs the error breaker — confirmed exempt, and it is load-bearing. Also since your review: the branch closed the unframed-delivery gap it had documented with a deliberately failing test — |
009fe97 to
1c14743
Compare
The per-action gate (tinyhumansai#4853 / tinyhumansai#4995) lives in `composio/contract_gate.rs`, but its own module doc already flags the next step: generalising it to the `composio_execute` dispatcher, the MCP bridges, and the Workflow dispatcher. Those domains cannot reach into `composio::` without inverting the layering, so move the module to `openhuman/tools/contract_gate.rs` — the aggregation layer that already depends on every domain — before extending it. Pure relocation: no behaviour change, no logic edited. Only the module declaration, the `ComposioActionTool` import path, and two doc links move. Issue tinyhumansai#4853
The gate merged in tinyhumansai#4995 (and hardened in tinyhumansai#5119/tinyhumansai#5132/tinyhumansai#5154) fronts exactly one late-bound surface: the per-action `ComposioActionTool`. Three more share its shape — the model is handed an envelope whose real contract lives behind a separate discovery tool it usually has not called: composio_execute {tool, arguments} -> live toolkit catalog mcp_registry_tool_call {server_id, tool_name, args} -> server input_schema run_workflow {workflow_id, inputs} -> declared [[inputs]] On each of those the model composes the call before the schema is in context and guesses — the same failure tinyhumansai#4853 opened on, one layer up. Extend the existing gate rather than adding a second mechanism. `GateTarget` names what to resolve a contract for; a neutral `GatedContract` normalises the three sources so the merged `args_satisfy_contract` validator and one formatter serve them all. Every property the gate already had applies unchanged to the new kinds: surface-once per instance, validate-then-pass (tinyhumansai#5119) so a well-formed first call executes directly, and the process-wide auto-proceed safety net (tinyhumansai#5119) so re-delegation cannot loop. Kind-specific details: - Keys are namespaced per kind, so a workflow and an MCP tool sharing a name never credit each other's gate. Composio slugs stay case-folded; MCP pairs and workflow ids address exact identities and are not. - The `connection_id` exemption is Composio's alone — nothing injects extra keys onto an MCP or workflow call, so an unknown key there is a real guess. - Each surfaced contract names the call to re-issue. "Call the action again" tells a model nothing after a blocked `run_workflow`. - A workflow's `[[inputs]]` block is synthesised into a JSON Schema so the shared validator treats it like any other target. - `run_workflow` consults the gate AFTER its profile-allowlist check, so a scoped-out workflow's contract is never rendered. - Every lookup sits behind its domain's Cargo feature; a compiled-out domain yields no contract and the gate proceeds. The kind-agnostic half (target identity, validation, rendering) compiles in every configuration, and its tests run in the gates-off smoke lane. Verified: contract_gate 16/16 (3 of them also under --no-default-features), composio 367, mcp_registry 150, tools 807, skills 187, run_workflow 11; default clippy --all-features clean; gates-off check + scoped gates-off test run clean; fmt clean. Issue tinyhumansai#4853
…paction The gate tracked "already surfaced" as tool state. That answer is only useful while the contract is still in front of the model, and it often is not: summarisation, microcompact tool-body blanking, the hard trim, and result-size caps can each drop or rewrite a delivered contract mid-turn. The gate would keep counting it as seen, and the model would call the tool against a schema it can no longer read — the exact failure the gate exists to prevent. Derive presence from the transcript instead. Each delivery leads with a `[contract-gate:<slug list>]` marker and the hash of its payload is recorded; before every model call the tool messages are rescanned, each marker's payload re-hashed, and its slugs credited only on an exact match. Correct across every context-management path by construction: a contract that survives byte-for-byte (including in a resumed sub-agent's history) still hashes equal and is not re-delivered; one summarised, blanked, truncated, or whitespace-collapsed no longer does, so it is re-delivered once. Recognition is a fixed-prefix compare at byte 0, not a substring search — one `strip_prefix` per tool message, and a model echoing the marker mid-text cannot spoof presence. Only tool-role messages are scanned. The marker carries slugs only, so it stays short and whitespace-free while the integrity check rides on the payload hash. **A hash miss skips that message; it never rejects the target.** The scan accumulates across the whole transcript, so a lookalike — a tool echoing the syntax, a stale copy since rewritten — contributes nothing and a genuine copy elsewhere still credits. Short-circuiting on the first miss would let one lookalike make a contract permanently un-creditable and re-deliver it on every call, forever. Two supporting pieces this needs to hold: - **Deliveries reach the transcript verbatim.** `HandoffMiddleware` (sub-agents) collapses whitespace runs to one space, which flattens the delivered schema's indentation, and `ToolOutputMiddleware` summarises/tabulates/truncates. Either makes the re-hash miss. Both now skip a delivery. The check is the hash, not the bare marker, so a coincidental lookalike is still cleaned normally. - **Discovery pre-crediting.** `describe_workflow`, `mcp_registry_list_tools`, and `composio_list_tools` (full-JSON rendering only — never the thin `prefer_markdown` listing, which would credit a contract the model never saw) lead their output with one marker packing every target they fully described, so a model that already read the real schema pays no re-delivery. Also addresses the review finding on the auto-proceed counter: it was a process-lifetime global, so once a slug crossed the threshold in any turn, every later turn auto-proceeded for it permanently — and in a multi-tenant process one user's count suppressed the gate for everyone. It is now scoped to the top-level turn (sub-agent runs deliberately inherit rather than re-scope, since a fresh count per spawn is the very loop the net detects). That also makes the auto-proceed test idempotent within one test binary. Verified: contract_gate 20/20 (7 of them also under --no-default-features), tinyagents 126, mcp_registry 150, skills 187, run_workflow 11; clippy --all-features clean; gates-off check + scoped gates-off run clean; fmt clean; feature-gate-smoke allowlist unchanged. Issue tinyhumansai#4853
A presence check that answers "absent" costs a re-delivery, so the mechanism is only safe if it is guaranteed to eventually answer "present". Two properties give that, and both were load-bearing but only one was written down. 1. A hash miss skips that message; it never rejects the target. Already documented and covered. 2. **Delivery overwrites the recorded hash.** This was only an implicit consequence of `HashMap::insert`. Every delivery records the hash of the exact bytes it emits, so the newest delivery is always creditable: the next rescan finds it intact, credits the slug, and the retry runs. Termination does not depend on any earlier copy surviving. Keeping the first hash instead would be the bug — a second delivery whose contract text differs (the provider republished the schema, or a discovery listing recorded a different rendering first) would hash to a value never recorded, could never be credited, and the gate would re-deliver it on every call forever. The cost of overwriting is that a superseded copy still in the transcript stops matching, which is harmless: the fresh delivery beside it credits instead. Documents both on `record_delivered` and together in the module doc, and pins the overwrite with `a_re_delivery_is_immediately_creditable_even_when_the_ contract_changed`. No behaviour change. contract_gate 21/21 (8 gates-off); clippy + fmt clean. Issue tinyhumansai#4853
This test is red on purpose. It marks a gap the rest of this PR does not
close, so a green CI cannot imply the feature works on every path.
The gate credits a delivered contract by finding its `[contract-gate:…]`
marker at byte 0 of a tool message and re-hashing the payload after it.
`to_provider_messages` breaks both halves: it wraps each result in
`<tool_result id="…">` and prefixes the batch with a `[Tool results]`
banner, emitted as a *user* message:
"[Tool results]\n<tool_result id=\"call-1\">\n[contract-gate:…]\n\n…"
The session and sub-agent paths build their history through that renderer
(`session/turn/core.rs:1199`, `subagent_runner/ops/graph.rs:388`), so
presence is never credited there. The gate itself still behaves — the
per-instance seen-set and the auto-proceed net bound it, so there is no
loop and no regression against upstream — but the compaction safety added
in `fc41b064` is inert on those paths: a contract summarised out of
context is not re-delivered, because it was never counted as present.
The fix is a `trusted_verbatim` flag carried on the result, with a flagged
delivery emitted as its own unframed turn. The flag has to survive
`tinyagents::harness::message::ToolMessage`, which today carries only
`tool_call_id` + `content`, so it needs an upstream crate change first.
That work is in flight; this test turns green when the flag lands.
Verified failing for the stated reason, not by accident — the assertion
prints the actual rendered framing.
Issue tinyhumansai#4853
The gate's `xxhash-rust` dependency reached the root `Cargo.lock` but not `app/src-tauri/Cargo.lock`, which resolves `openhuman_core` by path. CI runs the shell crate with `--locked`, so the first cargo invocation there failed with "cannot update the lock file" before any formatting or lint ran — the whole Rust Quality lane, and the PR CI Gate behind it. The diff is the one dependency and nothing else. Also scopes `fresh_gates_eventually_auto_proceed` inside `with_turn`. The counter it asserts on lives in the turn's task-local state, and a test that skips the scope reads the process-wide fallback instead — so a re-run in the same binary (a nextest retry, or any future caller) would start at the threshold, auto-proceed on gate 1, and fail the first assertion for a reason unrelated to the behaviour under test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy
Closes the gap the branch's own failing test documented. `trusted_verbatim` landed upstream on `tinyagents::harness::message::ToolMessage`, so the flag the test was waiting for now exists and is carried end to end. The gap: both text-mode serializers wrapped every result in `<tool_result id>` and prefixed the batch with a `[Tool results]` banner, so a delivered contract reached the model mid-message with a payload carrying whatever else the round returned. The gate credits a contract only while its payload still hashes to the value recorded at delivery, so on the session and sub-agent paths — which rebuild history through these serializers every turn — presence was never credited and the compaction-safety was inert: a contract summarised out of context was not re-delivered, because it had never counted as present. Producer to renderer: - `ToolResult::trusted_verbatim` + `mark_trusted_verbatim` (opt-in; an unmarked result serializes to exactly the JSON it did before the field existed). - `contract_gate::surface_result` — one constructor for all four gated sites, so neither property a delivery needs (error result, verbatim) can be forgotten at a call site where forgetting is invisible. - The tinyagents adapter carries the flag across the crate boundary; it is the only conversion, so a flag dropped there is dropped everywhere. - `ToolResultMessage::trusted_verbatim` carries it on the persisted shape too, because the serializers run again every turn from durable history. - `split_verbatim_results` gives a marked result its own turn in both serializers, closing the batch before it opens. An unmarked round renders byte-for-byte as before. The presence rescan now reads host-written rows (tool + user), not tool rows alone: a text-mode turn has no tool role to send, so an unframed delivery lands in a user turn and scanning tool rows would credit nothing after a rebuild. Assistant rows stay excluded — the model must not credit its own echo. Tests: the marker test asserts both serializers deliver the payload unchanged; new coverage for the per-result split (order and framing of the rest preserved) and for an unmarked round rendering exactly as it always did. contract_gate 22, dispatcher 77, message_convert 8, middleware 51, composio 763, skills::types 12. clippy clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy
`ToolResult` gained `trusted_verbatim`, and seven integration targets build the struct literally. The lib suite compiled without them, so the break only surfaced in the coverage lane, which builds every target. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy
Every other test here calls `consult` directly, which covers the decision logic but not the wiring: that `ContractGatePresenceMiddleware::before_model` runs after the context middlewares and rebuilds presence from the transcript the model is about to read, that the tool's own consult sits inside the tool call, and that those two orderings compose into deliver-then-retry. A test that rescans by hand cannot fail if the middleware is registered in the wrong place, or not at all. This drives a turn through `assemble_turn_harness` (via the channel graph, its thinnest caller) with a scripted model that calls the same gated tool twice, and asserts the action ran exactly once — not on the first call, not skipped on the retry — with the contract in the transcript the second model call reads. The tool stands in for a per-action Composio tool: same consult, same delivery, no network. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy
1c14743 to
0d12fcb
Compare
The gate's payload fingerprint pulled in `xxhash-rust`, and the kernel floor guard rejected it: profile `flows` went to 313 packages against a limit of 312, 286 crate names against 285. That guard exists precisely so a dependency does not arrive unnoticed, and one added for a hash the tree can already compute is the case it is meant to catch. `sha2` is an unconditional dependency. SHA-256 truncated to its leading 8 bytes gives the same 64-bit, cross-process-reproducible fingerprint. It runs once per gate delivery on a few hundred bytes, so the cost against XXH3 is unmeasurable. Drops the dependency from `Cargo.toml` and restores both lockfiles, which makes the earlier Tauri-lockfile fix unnecessary. contract_gate 22 tests pass; lib checks clean under --all-features. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy
Maintainer review — two things stand between this and merge, and neither is smallChecked against current 1. A standing
|
| file | what main did since your base |
your diff |
|---|---|---|
agent/tinyagents/middleware.rs |
−4511 lines → split into middleware_part_01..05.rs |
+90 |
agent/tinyagents/mod.rs |
−2331 lines → split | +48/−20 |
integrations/composio/tools.rs |
−1591 lines → split | +58/−4 |
flows/tinyflows/caps/ops.rs |
−1906 lines → split | +1 |
agent/dispatcher.rs |
+289/−443 rewrite | +68/−18 |
agent/tests.rs |
deleted | +5 |
skills/types.rs, mcp/registry/tools.rs, tools/ops.rs, agent/dispatcher_tests.rs, agent/harness/session/turn_tests.rs, flows/ops_tests.rs, .github/workflows/ci-lite.yml |
all moved or rewritten | — |
On top of that the crate was renamed (tinyagents → tinyagents_harness / tinyagents_graph), TinyPlace was removed from core, and your commit 1 — the "pure relocation" of composio/contract_gate.rs → tools/contract_gate.rs — now has to be replayed against a composio/contract_gate.rs that has itself been hardened upstream since you branched (#5119 / #5132 / #5154 landed after your base). So the relocation is no longer a rename; it is a rename plus a merge of three months of divergence in the file being renamed.
I stopped short of doing this rebase deliberately. Re-homing 2723 lines across a restructured tree means making judgement calls about where each hunk belongs and which upstream hardening supersedes which of your lines — and getting one of those wrong in a security-adjacent gate is worse than leaving the PR conflicted. That is your call to make, not mine to guess at.
The design still looks right, and the problem is still real
Worth saying, because "needs a big rebase" reads like a verdict on the work and it isn't. The premise holds on current main: composio/contract_gate.rs is still the only contract gate, and it still fronts exactly one late-bound surface. composio_execute, mcp_registry_tool_call and run_workflow still hand the model an envelope whose real schema lives behind a discovery tool it usually hasn't called. Extending the existing gate rather than adding a second mechanism is the right shape, and deriving contract-presence from a transcript marker rather than mutable turn state — which is what makes it survive summarization, microcompact, hard-trim and durable sub-agent resume — is the part I'd most want kept intact through any rebase.
Suggested sequencing
- Get @YellowSnnowmann to re-review against your 2026-07-31 replies first. If any of the four still stands, you'd rather know before paying for the rebase.
- Then rebase, ideally commit-by-commit: land the relocation against current
composio/contract_gate.rsas its own PR (it is reviewable as a rename + a small merge), and put the three-surface extension on top. A 2723-line rebase reviewed as one diff is where mistakes hide. - Consider marking this a draft (
gh pr ready --undo) while that is in flight — a title prefix alone does not stop the auto-merger here.
CI shows all-green, but that run is from 2026-08-05 against a base that no longer exists; it is not evidence about the current tree.
I have not pushed anything to your branch, and I am not approving or closing anything.
|
I've gone through this against current Short versionThe gate half re-homes cleanly. The delivery half no longer lives in this repository, and without it the gate would loop forever on text-mode providers. So a rebase of this PR alone would produce something that passes CI and is broken in production for a subset of users — which is worse than leaving it open. Where the 13 conflicts actually goMost are re-homing, exactly as you described, and I mapped every one:
That part is a day's careful work, not a blocker. The boundary: the serializers moved to
|
|
Upstream PR opened for the blocker: tinyhumansai/tinyagents#145 That teaches Once it merges, this PR reduces to the re-homing I mapped above: the gate + the four middleware guards + the four gated call sites, with no serializer changes. I'll rebase to that then rather than force-pushing something that is knowingly broken on text dialects now. @YellowSnnowmann — separately, and whenever convenient: this still carries your |
Review — not mergeable yet, and one of the blockers is a correctness gap rather than a rebaseReviewed against current The design is right and I'd like to see it land. What follows is what stands between it and merge. 1. It does not merge — 13 conflicting files, and most are re-homing, not conflictsThe two that matter most are not textual:
2. Blocking: the delivery half no longer has a file in this repository
// src/openhuman/agent/dispatcher.rs:289
fn dispatch_format_results(dialect: &dyn ToolDialect, results: &[ToolExecutionResult]) -> ConversationMessage {
from_transcript_entry(dialect.format_results(&to_outcomes(results)))
}The serializer is in vendored // crates/tinyagents-harness/src/tool_calling/dialect/text.rs
pub fn format_results(results: &[ToolOutcome]) -> TranscriptEntry {
for result in results {
writeln!(content, "<tool_result name=\"{}\" status=\"{}\">\n{}\n</tool_result>", …,
neutralize_protocol_tags(&result.output))
}
TranscriptEntry::Chat(DialectMessage::user(format!("{TOOL_RESULTS_PREFIX}{content}")))
}
let Some(rest) = text.strip_prefix(MARKER_OPEN) else { return }; // marker at byte 0
…
if delivered.get(slug_list) != Some(&payload_hash(payload)) { return } // payload byte-exactThe Consequence, stated precisely. On 3. Blocking: the upstream dependency is still open, and it is not the whole fixtinyhumansai/tinyagents#145 is OPEN, unmerged, with no approving review. It is also a breaking change ( Even once it lands, the host side is still short a hop. // src/openhuman/agent/dispatcher.rs:158
.map(|result| ToolOutcome {
name: result.name.clone(),
output: result.output.clone(),
success: result.success,
tool_call_id: result.tool_call_id.clone(),
})So the rebase is not "drop the serializer changes and consume the crate" — it is that, plus carrying verbatim through 4. Blocking:
|
Summary
The contract gate merged in #4995 (hardened in #5119 / #5132 / #5154) fronts exactly one late-bound surface: the per-action
ComposioActionTool. Three more surfaces have the same shape — the model is handed an envelope whose real contract lives behind a separate discovery tool it usually has not called:composio_execute{tool, arguments}—argumentsis a bare objectmcp_registry_tool_call{server_id, tool_name, arguments}input_schemarun_workflow{workflow_id, inputs}[[inputs]]On each, the model composes the call before the schema is in context and guesses — the same failure #4853 was opened on, one layer up.
This PR extends the existing gate to those three, rather than introducing a second mechanism.
What changed
Commit 1 — pure relocation.
composio/contract_gate.rs→tools/contract_gate.rs.mcp_registry/andagent/tools/cannot reach intocomposio::without inverting the layering, and the module's own doc already flagged generalisation as the next step. No logic edited; renders as a rename.Commit 2 — the extension.
GateTargetnames what to resolve a contract for; a neutralGatedContractnormalises the three sources so the mergedargs_satisfy_contractvalidator and a single formatter serve all of them.Every property the gate already had carries over unchanged to the new kinds:
Kind-specific details worth review attention:
connection_idexemption is Composio's alone — nothing injects extra keys onto an MCP or workflow call, so an unknown key there is a real guess.run_workflow.[[inputs]]block is synthesised into a JSON Schema so the shared validator treats it like any other target.run_workflowconsults the gate after its profile-allowlist check, so a scoped-out workflow's contract is never rendered.History note
This PR previously proposed a separate
tinyagents::contract_gatemiddleware with transcript-derived presence, and removed #4995's tool-layer gate to avoid double-gating. That is fully withdrawn. #5132's validate-then-pass and #5154's auto-proceed net solve the loop and redundancy concerns that motivated it, so the merged design is now the one being extended — and none of #4995 / #5119 / #5132 / #5154 is reverted.Test plan
openhuman::tools::contract_gate— 16/16 (9 pre-existing Composio tests preserved; new coverage for MCP resolution, workflow surfacing/validate-then-pass/no-config, key namespacing, per-kind retry hints, injected-arg scoping)--no-default-features --features tokenjuice-treesitter(gates-off lane)cargo clippy --lib --all-featuresclean;cargo fmt --checkcleancargo checkcleanLocal validation
The full husky pre-push hook runs clean in this environment (
HOOK_EXIT=0):format:check->lint->compile->rust:clippy(root-D warningsandapp/src-tauri) ->lint:commands-tokens.An earlier revision of this description said the hook could not run here and that the
push used
--no-verify. That is no longer true and the claim is withdrawn: the missingpieces were environment-side (uninitialised Tauri submodules, absent
ripgrep, and theLinux GTK/WebKit dev packages CONTRIBUTING.md lists as prerequisites), and they have
since been installed.
Closes #5038