From 21c65b75a5b8d17ab9789d3b7e2f124dd30631e9 Mon Sep 17 00:00:00 2001 From: Daniel Polo <106583643+danielPoloWork@users.noreply.github.com> Date: Thu, 9 Jul 2026 21:44:48 +0200 Subject: [PATCH] feat(os): model & effort routing policy as data + ADR-0017 (M16 16.1) Closes #252. New eighth OS spec orchestrator/os/routing/: work signals (tracker labels + sets-pattern/decision-heavy flags) escalate monotonically from a cheapest floor to a capability tier (fast/standard/frontier-reasoning) and an OS-neutral effort (low-max). Model names live only in the dated catalog (tiers-not-names); the tier call ships as a #224 worked-example decision surface registered under the examples gate. Advisory-first per RFC-0001: the human keeps final model authority. - os/routing/_schema.md + routing.yaml (auto-discovered by os-spec-completeness; os/** gate-coverage already applies) - eados_lint EXAMPLE_FILES += routing.yaml - ADR-0017 + index row; os/README spec table; AGENTS.md spec list; CHANGELOG [Unreleased] Co-Authored-By: Claude Fable 5 --- .../docs/adr/0017-model-effort-routing.md | 67 +++++++++++ .eados-core/docs/adr/README.md | 1 + .eados-core/orchestrator/os/README.md | 1 + .../orchestrator/os/routing/_schema.md | 68 +++++++++++ .../orchestrator/os/routing/routing.yaml | 108 ++++++++++++++++++ .eados-core/tools/eados_lint.py | 1 + AGENTS.md | 2 +- CHANGELOG.md | 15 +++ 8 files changed, 262 insertions(+), 1 deletion(-) create mode 100644 .eados-core/docs/adr/0017-model-effort-routing.md create mode 100644 .eados-core/orchestrator/os/routing/_schema.md create mode 100644 .eados-core/orchestrator/os/routing/routing.yaml diff --git a/.eados-core/docs/adr/0017-model-effort-routing.md b/.eados-core/docs/adr/0017-model-effort-routing.md new file mode 100644 index 0000000..25280ce --- /dev/null +++ b/.eados-core/docs/adr/0017-model-effort-routing.md @@ -0,0 +1,67 @@ +# ADR-0017: Model & effort routing — advisory-first, tiers-not-names + +## Status + +Accepted (2026-07-09) + +## Context + +EADOS had no notion of *which class of model* or *how much effort* a unit of work deserves: the +maintainer routed by hand (the M15 issue set was assessed manually on 2026-07-09, issue by issue). +Two facts constrain any mechanization: + +1. **Model names churn every few months.** A policy that says "use Fable 5" rots the day the + catalog moves on — exactly the trap ADR-0009 avoids for action pins and the risk model avoids + by keeping weights as data (#79). +2. **No host lets an agent switch its own top-level session model.** That is a human action (e.g. + `/model`). The only place "automatic" selection is real is *delegated* work, where a host may + support per-delegation model choice (e.g. an agent-spawn `model` parameter). + +## Decision + +1. **Advisory-first.** The OS *recommends* a route and states it at its read-points (Step-0 + triage, `/eados status`, the `.issues/` planning docs — wired in 16.3/#254). The human keeps + final model authority; advice never overrides an explicit human choice, and nothing in the + policy can express a top-level session model switch. Auto-application exists only for + host-delegated subagent work (16.4/#255) and falls back to advisory-only elsewhere. + +2. **Tiers, not model names.** The policy ([`os/routing/routing.yaml`](../../orchestrator/os/routing/routing.yaml)) + speaks capability tiers — `fast` / `standard` / `frontier-reasoning` — and concrete names live + *only* in its dated `catalog:` (today, Claude Code: Sonnet 5 / Opus 4.8 / Fable 5). Model churn + is a catalog edit with a refreshed `as_of:`; the rules and the tools never change for it. + +3. **OS-neutral effort vocabulary** — `low` | `medium` | `high` | `max` — with per-host + `effort_aliases` in the catalog (Claude Code: "ultracode" → `max`). Advice is emitted in the OS + vocabulary; adapters translate at the edge. + +4. **Resolution is monotonic escalation.** The recommendation starts at the cheapest floor + (`defaults`) and every matched rule raises it to at least its `min_tier` / `min_effort` — the + final answer is the max over floor and matches. Rules never lower a result, the same + only-add-never-relax principle as the workflow's domain overlays; resolution is deterministic + and order-independent (no model/LLM in the routing loop). Signals are what the tracker already + carries (`severity:*`, `adr`, `security`, …) plus two asserted flags: `sets-pattern` and + `decision-heavy`. + +5. **The judgment call ships as few-shot data.** The tier decision carries a worked-example + `examples:` block (verdicts = the tiers), registered under the `examples` gate — the M14 #224 + pattern, seeded from the maintainer-reviewed M15 assessment. + +## Consequences + +- `orchestrator/os/routing/` is the eighth OS spec: auto-discovered by `os-spec-completeness`, + parse-checked by `data-file-validity`, covered by the existing `os/**` gate-coverage entry; + the `examples` gate shape-checks its decision surface. Referential integrity (rules reference + declared tiers/efforts/flags; every host maps every tier) is documented as invariants and + enforced by the 16.2 evaluator's loud rejection. +- `tools/route_advice.py` (16.2/#253) can be a pure function over this file — fixture-testable + offline, identical advice at every surface. +- Recommending a *cheaper* route stays possible without a "lowering" rule: the floor is the + cheapest tier, so unlabeled small work (doc fixes, authoring) routes to `fast`/`low` by default. +- A future host (or a re-ranked market) is one catalog entry, not a schema change; a future tier + (e.g. a local model class) is an append to `tiers` plus rules that reference it. + +## References + +- Issue #252 (M16 16.1); milestone plan `.issues/M16-model-effort-routing-milestone.md` (#256); + RFC-0001 (human-in-the-loop); ADR-0009 (pin-rot precedent); risk-weights-as-data (#79); + worked-example decision surfaces (#224, M14); README model ranking (#122). diff --git a/.eados-core/docs/adr/README.md b/.eados-core/docs/adr/README.md index 57a52de..154ce92 100644 --- a/.eados-core/docs/adr/README.md +++ b/.eados-core/docs/adr/README.md @@ -25,3 +25,4 @@ is sequential and never reused. Template: the generated | [0014](0014-inbound-contribution-trust-model.md) | Inbound-contribution trust model (verify the change, never merge non-owner commits, adopt via co-author) | Accepted | | [0015](0015-web-domain-and-enterprise-posture.md) | Web domain (shipped) + enterprise as an orthogonal posture flag, not a domain | Accepted | | [0016](0016-authoring-language-model.md) | Authoring-language model: confirmed doc/comment-language defaults, non-English choices as recorded exceptions | Accepted | +| [0017](0017-model-effort-routing.md) | Model & effort routing: advisory-first, capability tiers not model names, dated per-host catalog | Accepted | diff --git a/.eados-core/orchestrator/os/README.md b/.eados-core/orchestrator/os/README.md index 8c3d575..2f49f2d 100644 --- a/.eados-core/orchestrator/os/README.md +++ b/.eados-core/orchestrator/os/README.md @@ -18,6 +18,7 @@ validates against a schema — never a special case in a tool. | **plan** | The roadmap-negotiation protocol: who proposes/sizes/reconciles, the T-shirt sizing scale, the `roadmap-covers-rfcs` gate | [`plan/_schema.md`](plan/_schema.md) · [`plan/plan.yaml`](plan/plan.yaml) (+ [`negotiation-protocol.md`](plan/negotiation-protocol.md)) | | **risk** | The audit risk model: security-surface globs, size buckets, levels, the per-domain mandatory-gate threshold | [`risk/_schema.md`](risk/_schema.md) · [`risk/risk.yaml`](risk/risk.yaml) | | **contribution** | The inbound-PR trust model: contributor tiers, the never-merge-non-owner-commits courtesy policy, the `contribution-review` gate, the escalation ladder | [`contribution/_schema.md`](contribution/_schema.md) · [`contribution/contribution.yaml`](contribution/contribution.yaml) | +| **routing** | Model & effort routing (advisory-first): work signals → capability tier + effort, the dated per-host model catalog (tiers-not-names, ADR-0017) | [`routing/_schema.md`](routing/_schema.md) · [`routing/routing.yaml`](routing/routing.yaml) | The **traceability graph** (requirement → RFC → milestone → PR → commit → release) and its lint are *described* here and in the RFC but are **built in M3/M4** — derived from the cross-links the diff --git a/.eados-core/orchestrator/os/routing/_schema.md b/.eados-core/orchestrator/os/routing/_schema.md new file mode 100644 index 0000000..96ee67b --- /dev/null +++ b/.eados-core/orchestrator/os/routing/_schema.md @@ -0,0 +1,68 @@ +# `routing.yaml` — schema + +The **model & effort routing policy** as data (M16 / ADR-0017). Given a unit of work's **signals** +— its tracker labels plus two derived flags — the policy recommends a **capability tier** and an +**effort level**. It is *advisory-first*: the OS states the recommendation at its read-points +(Step-0 triage, `/eados status`, the `.issues/` planning docs); the human keeps final model +authority, and auto-application exists only for host-delegated subagent work (RFC-0001 +human-in-the-loop). The evaluator is [`../../../tools/route_advice.py`](../../../tools/route_advice.py) +(M16 16.2; until it lands, the policy is read by the agent directly). + +**Tiers, not model names.** The rules speak capability tiers; concrete model names live *only* in +the dated `catalog:` below, so model churn is a catalog edit — never a policy or code change. +Precedent: risk-weights-as-data ([`../risk/_schema.md`](../risk/_schema.md)), the README model +ranking. + +`eados_lint` (`os-spec-completeness`) requires the instance to define every key below; the +`examples` gate (#224) shape-checks the worked-example decision surface. + +## Required structure + +```yaml +version: # integer schema version +tiers: # ordered capability tiers, cheapest -> most capable +efforts: # ordered effort levels, lowest -> highest (OS-neutral vocabulary) +flags: # derived-signal vocabulary beyond tracker labels: flag id -> meaning +defaults: # the floor when no rule matches + tier: # a `tiers` entry (the cheapest — rules only raise) + effort: # an `efforts` entry +rules: # monotonic escalation rules; each { id, when, min_tier, min_effort, why } +catalog: # the ONLY place concrete model names live + as_of: # "YYYY-MM-DD" the catalog was last verified against the market + hosts: # per-host entries { host, models: tier -> name, effort_aliases: alias -> effort } +examples: # worked-example decision surface (#224): verdicts = the tiers +``` + +## Resolution — monotonic escalation + +The recommendation starts at **`defaults`** (the cheapest floor) and every matched rule raises it +to at least the rule's `min_tier` / `min_effort` — the final answer is the **max** over the floor +and all matched rules, using the `tiers` / `efforts` order. A rule **matches** when *all* of its +`when` signals hold. Rules never lower a result — the same only-add-never-relax principle as the +workflow's domain overlays. Resolution is therefore deterministic and order-independent. + +**Signal vocabulary** (`when` entries): + +- `label:` — the unit of work carries that tracker label (e.g. `label:severity:high`, + `label:adr`, `label:security`). +- `flag:` — a derived flag from `flags:`, asserted by the caller (an evaluator heuristic or a + human judgment): `sets-pattern` (first of its class — the followers inherit a cheaper route) and + `decision-heavy` (the decision is the deliverable). + +## Effort vocabulary + +`efforts` is OS-neutral: `low` | `medium` | `high` | `max`. Hosts that use different words map +them in their catalog entry's `effort_aliases` (Claude Code: "ultracode" → `max`). Advice is +always *emitted* in the OS vocabulary; an adapter may translate on the way out. + +## Invariants + +- `defaults.tier` / `defaults.effort` and every rule's `min_tier` / `min_effort` are entries of + `tiers` / `efforts`. Every `flag:` signal references a `flags:` key. Every host's `models:` maps + **every** tier; every `effort_aliases` value is an `efforts` entry. (Shape is linted today; + referential integrity is enforced by the 16.2 evaluator's loud rejection.) +- `catalog.as_of` is refreshed whenever a mapping changes — a stale date is the review cue. +- The policy never contains a model name outside `catalog.hosts[].models`. +- Resolution is **deterministic** — same signals, same advice (no model/LLM in the loop). +- Advice never overrides a human's explicit model/effort choice, and no rule can force a + top-level session model switch — that action does not exist below the human (`AGENTS.md` §6). diff --git a/.eados-core/orchestrator/os/routing/routing.yaml b/.eados-core/orchestrator/os/routing/routing.yaml new file mode 100644 index 0000000..d05a699 --- /dev/null +++ b/.eados-core/orchestrator/os/routing/routing.yaml @@ -0,0 +1,108 @@ +# EADOS model & effort routing - reference instance. Schema: ./_schema.md (ADR-0017). +# Advisory-first: the OS recommends, the human keeps final model authority. The rules speak +# capability TIERS; concrete model names live ONLY in the dated catalog below, so model churn is +# a catalog edit - never a policy or code change. Evaluator: tools/route_advice.py (M16 16.2). +version: 1 + +# Ordered cheapest -> most capable / lowest -> highest. Resolution starts at the `defaults` floor +# and matched rules only ever RAISE (max over floor + matches) - the same only-add-never-relax +# principle as the workflow's domain overlays. Deterministic and order-independent. +tiers: [fast, standard, frontier-reasoning] +efforts: [low, medium, high, max] + +# Derived signals beyond tracker labels, asserted by an evaluator heuristic or a human judgment. +flags: + sets-pattern: "first of its class - fixes the template; the follower issues inherit a cheaper route" + decision-heavy: "the decision IS the deliverable (ADR / architecture call)" + +# The floor: absent any matching rule, work routes to the cheapest tier at the lowest effort +# (doc fixes, small authoring). Everything above is earned by a signal. +defaults: + tier: fast + effort: low + +# Signal vocabulary: "label:" or "flag:". A rule matches when ALL of +# its `when` signals hold; it raises the result to at least min_tier / min_effort. +rules: + - id: severity-medium + when: + - "label:severity:medium" + min_tier: standard + min_effort: medium + why: "a significant gap deserves substantive work - standard tier, real effort" + - id: severity-high + when: + - "label:severity:high" + min_tier: standard + min_effort: high + why: "core-guarantee work needs sustained effort; tier rises via flags, not severity alone" + - id: security-surface + when: + - "label:security" + min_tier: frontier-reasoning + min_effort: high + why: "on security posture the cost of a subtle miss dwarfs the routing saving" + - id: decision-heavy + when: + - "flag:decision-heavy" + min_tier: frontier-reasoning + min_effort: high + why: "when the decision is the deliverable, reasoning depth is the product" + - id: adr-is-decision-heavy + when: + - "label:adr" + min_tier: frontier-reasoning + min_effort: high + why: "an ADR-bearing issue is decision-heavy by definition" + - id: sets-pattern + when: + - "flag:sets-pattern" + min_tier: frontier-reasoning + min_effort: high + why: "the first of a class fixes the template every follower copies - amortized quality" + - id: foundational-decision + when: + - "flag:decision-heavy" + - "label:severity:high" + min_tier: frontier-reasoning + min_effort: max + why: "a wrong foundational call is the most expensive artifact the factory can produce" + +# The ONLY place model names live. `as_of` is refreshed whenever a mapping changes - a stale +# date is the review cue. Hosts with their own effort words map them via effort_aliases. +catalog: + as_of: "2026-07-09" + hosts: + - host: claude-code + models: + frontier-reasoning: "Fable 5" + standard: "Opus 4.8" + fast: "Sonnet 5" + effort_aliases: + ultracode: max + ultrathink: max + +# Worked-example decision surface (#224) - the judgment call is WHICH TIER a unit of work +# deserves. Shape-checked by the `examples` gate; drawn from the maintainer-reviewed M15 +# assessment of 2026-07-09 (.issues/M16-model-effort-routing-milestone.md, backfill table). +examples: + verdicts: [fast, standard, frontier-reasoning] + cases: + - input: "Doc drift - two playbooks omit a domain the code already ships (severity:low)" + verdict: fast + why: "mechanical prose fix against a known source of truth; no design, small blast radius" + - input: "Author a persona file plus a worked config example (severity:low)" + verdict: fast + why: "authoring to an established format; review catches style drift cheaply" + - input: "Third cross-cutting command doc, after the first fixed the template (severity:medium)" + verdict: standard + why: "substantive but pattern-following; the expensive thinking was spent on the first of the class" + - input: "Reconcile a defect backlog against in-tree fixes and build a regression index (severity:high)" + verdict: standard + why: "meticulous verification, not open-ended design - effort high, tier standard" + - input: "The taxonomy ADR every other issue in the milestone builds on (label:adr, severity:high)" + verdict: frontier-reasoning + why: "decision-heavy and foundational; a wrong call here is repaid with interest downstream" + - input: "First command of a new class - fixes the shape its siblings will copy (sets-pattern)" + verdict: frontier-reasoning + why: "sets-pattern - the marginal cost of the best model is amortized across every follower" diff --git a/.eados-core/tools/eados_lint.py b/.eados-core/tools/eados_lint.py index 4f26ee9..cd747d4 100644 --- a/.eados-core/tools/eados_lint.py +++ b/.eados-core/tools/eados_lint.py @@ -1155,6 +1155,7 @@ def check_gate_executability(fail): "orchestrator/questionnaire.yaml", # ask / default "learning/scope-examples.yaml", # apply / skip (companion to the lessons ledger) "orchestrator/triage.yaml", # answer / focused-change / five-step-loop (Step-0) + "orchestrator/os/routing/routing.yaml", # fast / standard / frontier-reasoning (tier calls) ) EXAMPLE_CASE_REQUIRED = ("input", "verdict", "why") diff --git a/AGENTS.md b/AGENTS.md index ed32300..905911e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -53,7 +53,7 @@ EADOS is a **phase-based agentic delivery operating system**: an opt-in pipeline its lifecycle. It is *declarative, gate-enforced, and human-gated* (not a runtime kernel); the design is [RFC-0001](.eados-core/docs/rfc/0001-eados-delivery-os.md), the phases are [`orchestrator/commands/`](.eados-core/orchestrator/commands/README.md), and the machine-readable -specs (`workflow`, `authority`, `git`, `rfc`, `plan`, `risk`, `contribution`) live under +specs (`workflow`, `authority`, `git`, `rfc`, `plan`, `risk`, `contribution`, `routing`) live under [`orchestrator/os/`](.eados-core/orchestrator/os/README.md). The **`scaffold` phase is the factory**: it reproduces the enterprise agent system of diff --git a/CHANGELOG.md b/CHANGELOG.md index ce9d9ca..1146784 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,21 @@ in the same PR. Releases follow Semantic Versioning; the latest is **v2.8.0**. ## [Unreleased] +### Added + +- **Model & effort routing policy — advisory-first, tiers-not-names (#252, M16 16.1, ADR-0017).** + A new eighth OS spec [`orchestrator/os/routing/`](.eados-core/orchestrator/os/routing/_schema.md) + turns "which model, how much effort?" from a manual per-issue judgment into policy-as-data: work + signals (tracker labels + the `sets-pattern` / `decision-heavy` flags) escalate monotonically + from a cheapest floor to a **capability tier** (`fast`/`standard`/`frontier-reasoning`) and an + OS-neutral **effort** (`low`–`max`, host aliases like "ultracode" → `max`). Concrete model names + live *only* in the dated `catalog:` (today: Sonnet 5 / Opus 4.8 / Fable 5 for Claude Code), so + model churn is a catalog edit — never a policy or code change. The tier call ships as a #224 + worked-example decision surface (registered under the `examples` gate); the spec is + auto-discovered by `os-spec-completeness` and covered by the existing `os/**` gate-coverage + entry. Advisory-first per RFC-0001: the human keeps final model authority; the evaluator and its + surfaces land with 16.2–16.4 (#253–#255). + ## [2.8.0] - 2026-07-08 ### Added