Skip to content

m12-pr17: action-gate contract pin (tests/ai/test_action_gate.py)#35

Open
BalaShankar9 wants to merge 1 commit into
m12-pr16-test-partition-healthfrom
m12-pr17-test-action-gate
Open

m12-pr17: action-gate contract pin (tests/ai/test_action_gate.py)#35
BalaShankar9 wants to merge 1 commit into
m12-pr16-test-partition-healthfrom
m12-pr17-test-action-gate

Conversation

@BalaShankar9

Copy link
Copy Markdown
Owner

m12-pr17: action-gate contract pin (tests/ai/test_action_gate.py)

Adds 21 tests pinning cross-cutting invariants of the "action gate"
(Dispatcher.invoke per blueprint section 6.2 — the single chokepoint
that decides whether a model-proposed tool call may execute).

Existing tests in ai_engine/tests/registry/test_dispatcher.py already
cover mechanism (grant flow, capability tokens, sandbox routing, schema
validation). This file pins the cross-cutting CONTRACTS that hold
across every code path — the boundary itself.

Invariants pinned (8 groups, 21 tests)

  1. Status enum is closed: every emitted status in
    {ok, denied, invalid_input, invalid_output, timeout, error}.
    Drives every branch in one test, asserts no surprise statuses leak.
  2. Audit row count = 1 per invoke (parametrized across all 6 branches).
    Catches double-write or silent-skip regressions.
  3. Resolver never called on a deny (grant deny, invalid input,
    capability missing, unknown tool). Side-effect-freedom on the
    deny path is the whole point of the gate.
  4. duration_ms >= 0 on every audit row (no monotonic-clock skew).
  5. error_message truncated to 500 chars (DB column cap).
  6. input_hash deterministic across identical args, distinct for
    different args, order-independent (json sort_keys contract).
  7. org_id / user_id propagate to audit row across ok/denied/
    invalid_input/error branches.
  8. Sink failure never masks tool result OR original tool exception
    type — operators rely on exception type for incident response.

Why "action gate" vs "dispatcher": semantic. The blueprint calls this
the action gate; the file is named after the boundary, not the class.
Anyone searching for "where do we enforce that a model cannot directly
take an action" will find this file first.

Files

  • backend/tests/ai/init.py (new, empty)
  • backend/tests/ai/test_action_gate.py (new, 21 tests)
  • context/TESTING_CONTEXT.md (Shipped m12-pr17 section)
  • context/CHANGELOG_INTELLIGENCE.md (m12-pr17 row)

Validation

  • pytest tests/ai/test_action_gate.py -v -> 21 passed in 0.16s
  • No other tests touched; no production code touched.

Stacked on PR #34 (m12-pr16). Base branch: m12-pr16-test-partition-health.

Adds 21 tests pinning cross-cutting invariants of the "action gate"
(Dispatcher.invoke per blueprint section 6.2 — the single chokepoint
that decides whether a model-proposed tool call may execute).

Existing tests in ai_engine/tests/registry/test_dispatcher.py already
cover mechanism (grant flow, capability tokens, sandbox routing, schema
validation). This file pins the cross-cutting CONTRACTS that hold
across every code path — the boundary itself.

Invariants pinned (8 groups, 21 tests)
1. Status enum is closed: every emitted status in
   {ok, denied, invalid_input, invalid_output, timeout, error}.
   Drives every branch in one test, asserts no surprise statuses leak.
2. Audit row count = 1 per invoke (parametrized across all 6 branches).
   Catches double-write or silent-skip regressions.
3. Resolver never called on a deny (grant deny, invalid input,
   capability missing, unknown tool). Side-effect-freedom on the
   deny path is the whole point of the gate.
4. duration_ms >= 0 on every audit row (no monotonic-clock skew).
5. error_message truncated to 500 chars (DB column cap).
6. input_hash deterministic across identical args, distinct for
   different args, order-independent (json sort_keys contract).
7. org_id / user_id propagate to audit row across ok/denied/
   invalid_input/error branches.
8. Sink failure never masks tool result OR original tool exception
   type — operators rely on exception type for incident response.

Why "action gate" vs "dispatcher": semantic. The blueprint calls this
the action gate; the file is named after the boundary, not the class.
Anyone searching for "where do we enforce that a model cannot directly
take an action" will find this file first.

Files
- backend/tests/ai/__init__.py (new, empty)
- backend/tests/ai/test_action_gate.py (new, 21 tests)
- context/TESTING_CONTEXT.md (Shipped m12-pr17 section)
- context/CHANGELOG_INTELLIGENCE.md (m12-pr17 row)

Validation
- pytest tests/ai/test_action_gate.py -v -> 21 passed in 0.16s
- No other tests touched; no production code touched.

Stacked on PR #34 (m12-pr16). Base branch: m12-pr16-test-partition-health.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new contract-style test suite that pins cross-cutting invariants of the “action gate” (ai_engine.registry.Dispatcher.invoke) and updates the project’s testing/changelog context docs to reflect the shipped work.

Changes:

  • Introduces backend/tests/ai/test_action_gate.py with 21 tests covering status closure, audit row guarantees, deny-path side-effect freedom, hashing determinism, propagation of org/user IDs, and sink best-effort behavior.
  • Adds backend/tests/ai/__init__.py to formalize the test package.
  • Updates /context documentation (TESTING_CONTEXT.md, CHANGELOG_INTELLIGENCE.md) to record m12-pr17 as shipped.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File Description
context/TESTING_CONTEXT.md Documents the new “action gate” contract test coverage under Shipped (m12-pr17).
context/CHANGELOG_INTELLIGENCE.md Adds the m12-pr17 ledger entry and links m12-pr16 to PR #34.
backend/tests/ai/init.py Adds an empty package marker for backend.tests.ai.
backend/tests/ai/test_action_gate.py New contract test suite pinning Dispatcher.invoke invariants across success/deny/error branches.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +36 to +41
# The dispatcher lives in ai_engine/, which is a sibling of backend/.
# Tests under backend/ run with backend/ on sys.path; add the repo root
# so `ai_engine` is importable.
REPO_ROOT = Path(__file__).resolve().parents[3]
if str(REPO_ROOT) not in sys.path:
sys.path.insert(0, str(REPO_ROOT))
Comment on lines +180 to +181
# We exercised every allowed status except possibly invalid_output's
# parent — sanity check that we covered most of the enum.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants