m12-pr08 P0-4: per-org daily cost cap + cascade breaker

[BalaShankar9]

P0-4 from the World Class Architecture Blueprint. Wires the spend telemetry shipped in #25 into a hard cascade breaker that refuses LLM calls before any provider tier is touched once an org has spent past its configured daily $ cap.

What ships

ai_engine/cost_breaker.py — new module:

• OrgDailyCostCapExceeded exception (tenant_id, spent_cents, cap_cents).
• set_tenant_id / reset_tenant_id / get_tenant_id / async tenant_scope ContextVar helpers — single canonical place to bind tenant identity for downstream LLM calls.
• cap_cents_for(tenant_id) reads ORG_DAILY_COST_CAP_USD (global) and ORG_DAILY_COST_CAP_OVERRIDES (per-tenant JSON map). 0 / unset / parse error all mean "cap disabled" (fail-open).
• enforce_org_daily_cost_cap() — async; queries CostAttributionService.get_org_cost_today_cents (FRESH base-table read, not the 60s MV) and raises when spent_cents >= cap_cents. Emits cost.cap.tripped on the bound emitter (best-effort). Fail-open on cost-service errors.

ai_engine/client.py — 5 hooks:

• New _enforce_cost_cap() helper.
• Called from complete(), complete_json(), stream_completion(), and chat() immediately after _check_budget() and before any cascade work. One chokepoint per public LLM entrypoint covers all 9 production call sites.
• _record_invocation now auto-pulls tenant_id from the ContextVar when callers don't pass one — same scope drives both attribution writes and cap enforcement.

Tests — backend/tests/unit/test_cost_breaker.py, 20 tests:

• ContextVar lifecycle (default, set/reset, async scope).
• Env parsing (global, per-tenant overrides, 0 disables, invalid JSON, invalid string).
• Per-tenant override > global; 0 in override disables.
• Enforcement: no tenant = no DB call, no cap = no DB call, under = pass, over = raise, exact-cap = raise, ContextVar tenant resolves, fail-open on DB error.
• Event: cost.cap.tripped emitted on trip, emit_event=False suppresses.
• Exception message format.

Result: 30/30 green with test_cost_attribution.py (no regression on #25).

Blueprint: P0-4 flipped TODO → SHIPPED.

Why this shape

• Breaker fires BEFORE the cascade loop, not inside it — a tripped cap costs zero LLM tokens.
• Tenant flows via ContextVar so call sites don't need to thread an org_id parameter through every chain / agent / tool layer. Caller sets tenant_scope("org-X") once at request entry; the breaker AND the recorder both pick it up.
• Fail-open everywhere: bad env, JSON parse error, missing override, cost service down, event-emitter exception — nothing in the cap machinery blocks a legitimate call when the cap subsystem itself is unhealthy. A telemetry outage must not become an availability outage.
• Cap source is env-only (no new feature flag) because the value is per-deployment, not per-feature-flip. Per-tenant overrides live in a JSON env to keep the blast radius of a typo small.

Stack

... -> #21 -> #22 -> #23 -> #24 -> #25 (m12-pr07-cost-attribution) -> #THIS

Files: 4 changed, +487 / -1.

Deployment notes

Cap is OFF until an operator sets ORG_DAILY_COST_CAP_USD (e.g. 50 for $50/day). To enable per-tenant:

ORG_DAILY_COST_CAP_OVERRIDES='{"<tenant-uuid>": 200, "<vip-uuid>": 0}'

(0 = disabled for that tenant; positive = override the global cap.)

For tenant scoping to work, request entrypoints (HTTP handlers, worker job pickup, SSE start) must wrap their LLM-touching code with async with tenant_scope(org_id): .... Wiring the entrypoints is intentionally out-of-scope — separate PR per entrypoint family.

[Copilot]

Pull request overview

Implements blueprint P0-4 by introducing a per-tenant daily LLM spend cap (“cascade breaker”) enforced at the entry of AIClient public LLM methods, driven by tenant identity propagated via a ContextVar and backed by CostAttributionService’s “today” spend read.

Changes:

• Added ai_engine.cost_breaker with tenant ContextVar helpers, env-driven cap parsing (global + per-tenant overrides), and async enforcement that raises OrgDailyCostCapExceeded.
• Wired enforcement into AIClient entrypoints and made invocation recording default tenant_id from the ContextVar when not explicitly provided.
• Added unit tests for breaker behavior and updated the architecture blueprint status for P0-4.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
docs/architecture/WORLD_CLASS_ARCHITECTURE_BLUEPRINT.md	Marks P0-4 as shipped and documents the cap/breaker behavior.
backend/tests/unit/test_cost_breaker.py	New unit tests covering ContextVar lifecycle, env parsing, enforcement, and event emission.
ai_engine/cost_breaker.py	New module implementing cap parsing + enforcement and the OrgDailyCostCapExceeded exception.
ai_engine/client.py	Enforces the cap on LLM entrypoints and auto-attaches tenant_id to invocation records from the ContextVar.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.