Composio contract fixture encodes the assumption under test — the flows required-arg gate is green in CI and inert in production

[M3gA-Mind]

Summary

seeded_gmail_send_contract() fabricates the Composio tool contract that the flows required-argument gate is tested against. The gate is therefore green in CI and inert in production, and no lane can detect the difference.

Found while researching #6154, where the gate's inertness is the reason a GMAIL_SEND_EMAIL node with no recipient reached run-time.

The fixture

src/openhuman/flows/builder_tools_tests.rs:39:

fn seeded_gmail_send_contract() -> ToolContract {
    ToolContract {
        slug: "GMAIL_SEND_EMAIL".to_string(),
        ...
        required_args: vec!["to".to_string(), "body".to_string()],
        input_schema: Some(json!({ "type": "object", "required": ["to", "body"] })),

Every flows test seeds this into the process-global catalog cache. The required-arg gate then works perfectly — against a contract the test itself wrote.

Why this is more than a weak test

Nothing anywhere asserts that the real Composio catalog declares a recipient required. The gate's entire behaviour depends on a property of an external system that is never checked against that system.

There is a tell that the fixture was written from the OpenHuman side rather than from a real catalog response: it uses to, which is OpenHuman's own alias, not Composio's recipient_email. A fixture derived from a real response would carry the provider's field name.

So the test does not encode "Composio requires a recipient". It encodes "we believe Composio requires a recipient, spelled the way we spell it" — and then verifies our code against our own belief. That is a fixture encoding the assumption under test, which is a test that cannot fail for the reason it exists.

Consequence

Green in CI, inert in production, and no lane could have caught it — not the unit lane, not coverage, not e2e. #6154 is what that looks like from the outside: a required-arg gate that exists, is tested, and does not fire.

Blast radius

Other tests seeded from the same cache and pinning current behaviour:

• src/openhuman/flows/builder_tools_tests_part_02_tests.rs:193 — dry_run_catches_unwired_required_composio_arg
• src/openhuman/flows/builder_tools_tests_part_03_tests.rs:177
• src/openhuman/flows/ops_tests_part_12_tests.rs:95 — builds a GMAIL_SEND_EMAIL node with args: {} and asserts only on connection status

The third is worth noting: it would newly fail if an argument check were added at that layer, so this fixture problem also shapes what a fix to #6154 is allowed to do.

Fix shape (not proposed as urgent)

The generalisable question is how a fixture for an external contract earns trust. Options, roughly in increasing cost:

1. Derive the fixture from a recorded real catalog response, checked in, with the provider's own field names — so the alias mapping is exercised rather than assumed.
2. A single contract test that fetches the live catalog for the handful of slugs prepare_execute_arguments hard-codes rules for (GMAIL_SEND_EMAIL, GMAIL_ADD_LABEL_TO_EMAIL, GOOGLECALENDAR_*, NOTION_FETCH_DATA) and asserts our beliefs still hold. Network-gated, run on a schedule rather than per PR.
3. Assert the narrower, checkable thing: that the fixture's required args survive OpenHuman's own alias normalisation, which at least stops the fixture and the production alias table drifting apart.

Option 2 is the only one that would actually have caught #6154. Whether that is worth a scheduled network-touching lane is a judgement call — filing this so the choice is explicit rather than implicit.

Severity

P3. No user-visible defect on its own. It is filed because it is the reason a P2/P3 defect shipped undetected, and because the same pattern will hide the next one — any gate whose fixture is written from our side of the boundary has this property.

Related

• Context: Flow validation: GMAIL_SEND_EMAIL runs without 'to' field — missing pre-flight argument check #6154 (the defect this fixture allowed through)

[tinysweeper]

Composio contract fixture encodes assumption under test, making required-arg gate green in CI and inert in production

Labelled priority: p3.