m12-pr17: action-gate contract pin (tests/ai/test_action_gate.py)#35
Open
BalaShankar9 wants to merge 1 commit into
Open
m12-pr17: action-gate contract pin (tests/ai/test_action_gate.py)#35BalaShankar9 wants to merge 1 commit into
BalaShankar9 wants to merge 1 commit into
Conversation
Adds 21 tests pinning cross-cutting invariants of the "action gate"
(Dispatcher.invoke per blueprint section 6.2 — the single chokepoint
that decides whether a model-proposed tool call may execute).
Existing tests in ai_engine/tests/registry/test_dispatcher.py already
cover mechanism (grant flow, capability tokens, sandbox routing, schema
validation). This file pins the cross-cutting CONTRACTS that hold
across every code path — the boundary itself.
Invariants pinned (8 groups, 21 tests)
1. Status enum is closed: every emitted status in
{ok, denied, invalid_input, invalid_output, timeout, error}.
Drives every branch in one test, asserts no surprise statuses leak.
2. Audit row count = 1 per invoke (parametrized across all 6 branches).
Catches double-write or silent-skip regressions.
3. Resolver never called on a deny (grant deny, invalid input,
capability missing, unknown tool). Side-effect-freedom on the
deny path is the whole point of the gate.
4. duration_ms >= 0 on every audit row (no monotonic-clock skew).
5. error_message truncated to 500 chars (DB column cap).
6. input_hash deterministic across identical args, distinct for
different args, order-independent (json sort_keys contract).
7. org_id / user_id propagate to audit row across ok/denied/
invalid_input/error branches.
8. Sink failure never masks tool result OR original tool exception
type — operators rely on exception type for incident response.
Why "action gate" vs "dispatcher": semantic. The blueprint calls this
the action gate; the file is named after the boundary, not the class.
Anyone searching for "where do we enforce that a model cannot directly
take an action" will find this file first.
Files
- backend/tests/ai/__init__.py (new, empty)
- backend/tests/ai/test_action_gate.py (new, 21 tests)
- context/TESTING_CONTEXT.md (Shipped m12-pr17 section)
- context/CHANGELOG_INTELLIGENCE.md (m12-pr17 row)
Validation
- pytest tests/ai/test_action_gate.py -v -> 21 passed in 0.16s
- No other tests touched; no production code touched.
Stacked on PR #34 (m12-pr16). Base branch: m12-pr16-test-partition-health.
There was a problem hiding this comment.
Pull request overview
Adds a new contract-style test suite that pins cross-cutting invariants of the “action gate” (ai_engine.registry.Dispatcher.invoke) and updates the project’s testing/changelog context docs to reflect the shipped work.
Changes:
- Introduces
backend/tests/ai/test_action_gate.pywith 21 tests covering status closure, audit row guarantees, deny-path side-effect freedom, hashing determinism, propagation of org/user IDs, and sink best-effort behavior. - Adds
backend/tests/ai/__init__.pyto formalize the test package. - Updates
/contextdocumentation (TESTING_CONTEXT.md,CHANGELOG_INTELLIGENCE.md) to record m12-pr17 as shipped.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| context/TESTING_CONTEXT.md | Documents the new “action gate” contract test coverage under Shipped (m12-pr17). |
| context/CHANGELOG_INTELLIGENCE.md | Adds the m12-pr17 ledger entry and links m12-pr16 to PR #34. |
| backend/tests/ai/init.py | Adds an empty package marker for backend.tests.ai. |
| backend/tests/ai/test_action_gate.py | New contract test suite pinning Dispatcher.invoke invariants across success/deny/error branches. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+36
to
+41
| # The dispatcher lives in ai_engine/, which is a sibling of backend/. | ||
| # Tests under backend/ run with backend/ on sys.path; add the repo root | ||
| # so `ai_engine` is importable. | ||
| REPO_ROOT = Path(__file__).resolve().parents[3] | ||
| if str(REPO_ROOT) not in sys.path: | ||
| sys.path.insert(0, str(REPO_ROOT)) |
Comment on lines
+180
to
+181
| # We exercised every allowed status except possibly invalid_output's | ||
| # parent — sanity check that we covered most of the enum. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
m12-pr17: action-gate contract pin (tests/ai/test_action_gate.py)
Adds 21 tests pinning cross-cutting invariants of the "action gate"
(Dispatcher.invoke per blueprint section 6.2 — the single chokepoint
that decides whether a model-proposed tool call may execute).
Existing tests in ai_engine/tests/registry/test_dispatcher.py already
cover mechanism (grant flow, capability tokens, sandbox routing, schema
validation). This file pins the cross-cutting CONTRACTS that hold
across every code path — the boundary itself.
Invariants pinned (8 groups, 21 tests)
{ok, denied, invalid_input, invalid_output, timeout, error}.
Drives every branch in one test, asserts no surprise statuses leak.
Catches double-write or silent-skip regressions.
capability missing, unknown tool). Side-effect-freedom on the
deny path is the whole point of the gate.
different args, order-independent (json sort_keys contract).
invalid_input/error branches.
type — operators rely on exception type for incident response.
Why "action gate" vs "dispatcher": semantic. The blueprint calls this
the action gate; the file is named after the boundary, not the class.
Anyone searching for "where do we enforce that a model cannot directly
take an action" will find this file first.
Files
Validation
Stacked on PR #34 (m12-pr16). Base branch: m12-pr16-test-partition-health.