Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .eados-core/docs/adr/0017-model-effort-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# ADR-0017: Model & effort routing — advisory-first, tiers-not-names

## Status

Accepted (2026-07-09)

## Context

EADOS had no notion of *which class of model* or *how much effort* a unit of work deserves: the
maintainer routed by hand (the M15 issue set was assessed manually on 2026-07-09, issue by issue).
Two facts constrain any mechanization:

1. **Model names churn every few months.** A policy that says "use Fable 5" rots the day the
catalog moves on — exactly the trap ADR-0009 avoids for action pins and the risk model avoids
by keeping weights as data (#79).
2. **No host lets an agent switch its own top-level session model.** That is a human action (e.g.
`/model`). The only place "automatic" selection is real is *delegated* work, where a host may
support per-delegation model choice (e.g. an agent-spawn `model` parameter).

## Decision

1. **Advisory-first.** The OS *recommends* a route and states it at its read-points (Step-0
triage, `/eados status`, the `.issues/` planning docs — wired in 16.3/#254). The human keeps
final model authority; advice never overrides an explicit human choice, and nothing in the
policy can express a top-level session model switch. Auto-application exists only for
host-delegated subagent work (16.4/#255) and falls back to advisory-only elsewhere.

2. **Tiers, not model names.** The policy ([`os/routing/routing.yaml`](../../orchestrator/os/routing/routing.yaml))
speaks capability tiers — `fast` / `standard` / `frontier-reasoning` — and concrete names live
*only* in its dated `catalog:` (today, Claude Code: Sonnet 5 / Opus 4.8 / Fable 5). Model churn
is a catalog edit with a refreshed `as_of:`; the rules and the tools never change for it.

3. **OS-neutral effort vocabulary** — `low` | `medium` | `high` | `max` — with per-host
`effort_aliases` in the catalog (Claude Code: "ultracode" → `max`). Advice is emitted in the OS
vocabulary; adapters translate at the edge.

4. **Resolution is monotonic escalation.** The recommendation starts at the cheapest floor
(`defaults`) and every matched rule raises it to at least its `min_tier` / `min_effort` — the
final answer is the max over floor and matches. Rules never lower a result, the same
only-add-never-relax principle as the workflow's domain overlays; resolution is deterministic
and order-independent (no model/LLM in the routing loop). Signals are what the tracker already
carries (`severity:*`, `adr`, `security`, …) plus two asserted flags: `sets-pattern` and
`decision-heavy`.

5. **The judgment call ships as few-shot data.** The tier decision carries a worked-example
`examples:` block (verdicts = the tiers), registered under the `examples` gate — the M14 #224
pattern, seeded from the maintainer-reviewed M15 assessment.

## Consequences

- `orchestrator/os/routing/` is the eighth OS spec: auto-discovered by `os-spec-completeness`,
parse-checked by `data-file-validity`, covered by the existing `os/**` gate-coverage entry;
the `examples` gate shape-checks its decision surface. Referential integrity (rules reference
declared tiers/efforts/flags; every host maps every tier) is documented as invariants and
enforced by the 16.2 evaluator's loud rejection.
- `tools/route_advice.py` (16.2/#253) can be a pure function over this file — fixture-testable
offline, identical advice at every surface.
- Recommending a *cheaper* route stays possible without a "lowering" rule: the floor is the
cheapest tier, so unlabeled small work (doc fixes, authoring) routes to `fast`/`low` by default.
- A future host (or a re-ranked market) is one catalog entry, not a schema change; a future tier
(e.g. a local model class) is an append to `tiers` plus rules that reference it.

## References

- Issue #252 (M16 16.1); milestone plan `.issues/M16-model-effort-routing-milestone.md` (#256);
RFC-0001 (human-in-the-loop); ADR-0009 (pin-rot precedent); risk-weights-as-data (#79);
worked-example decision surfaces (#224, M14); README model ranking (#122).
1 change: 1 addition & 0 deletions .eados-core/docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ is sequential and never reused. Template: the generated
| [0014](0014-inbound-contribution-trust-model.md) | Inbound-contribution trust model (verify the change, never merge non-owner commits, adopt via co-author) | Accepted |
| [0015](0015-web-domain-and-enterprise-posture.md) | Web domain (shipped) + enterprise as an orthogonal posture flag, not a domain | Accepted |
| [0016](0016-authoring-language-model.md) | Authoring-language model: confirmed doc/comment-language defaults, non-English choices as recorded exceptions | Accepted |
| [0017](0017-model-effort-routing.md) | Model & effort routing: advisory-first, capability tiers not model names, dated per-host catalog | Accepted |
1 change: 1 addition & 0 deletions .eados-core/orchestrator/os/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ validates against a schema — never a special case in a tool.
| **plan** | The roadmap-negotiation protocol: who proposes/sizes/reconciles, the T-shirt sizing scale, the `roadmap-covers-rfcs` gate | [`plan/_schema.md`](plan/_schema.md) · [`plan/plan.yaml`](plan/plan.yaml) (+ [`negotiation-protocol.md`](plan/negotiation-protocol.md)) |
| **risk** | The audit risk model: security-surface globs, size buckets, levels, the per-domain mandatory-gate threshold | [`risk/_schema.md`](risk/_schema.md) · [`risk/risk.yaml`](risk/risk.yaml) |
| **contribution** | The inbound-PR trust model: contributor tiers, the never-merge-non-owner-commits courtesy policy, the `contribution-review` gate, the escalation ladder | [`contribution/_schema.md`](contribution/_schema.md) · [`contribution/contribution.yaml`](contribution/contribution.yaml) |
| **routing** | Model & effort routing (advisory-first): work signals → capability tier + effort, the dated per-host model catalog (tiers-not-names, ADR-0017) | [`routing/_schema.md`](routing/_schema.md) · [`routing/routing.yaml`](routing/routing.yaml) |

The **traceability graph** (requirement → RFC → milestone → PR → commit → release) and its lint
are *described* here and in the RFC but are **built in M3/M4** — derived from the cross-links the
Expand Down
68 changes: 68 additions & 0 deletions .eados-core/orchestrator/os/routing/_schema.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# `routing.yaml` — schema

The **model & effort routing policy** as data (M16 / ADR-0017). Given a unit of work's **signals**
— its tracker labels plus two derived flags — the policy recommends a **capability tier** and an
**effort level**. It is *advisory-first*: the OS states the recommendation at its read-points
(Step-0 triage, `/eados status`, the `.issues/` planning docs); the human keeps final model
authority, and auto-application exists only for host-delegated subagent work (RFC-0001
human-in-the-loop). The evaluator is [`../../../tools/route_advice.py`](../../../tools/route_advice.py)
(M16 16.2; until it lands, the policy is read by the agent directly).

**Tiers, not model names.** The rules speak capability tiers; concrete model names live *only* in
the dated `catalog:` below, so model churn is a catalog edit — never a policy or code change.
Precedent: risk-weights-as-data ([`../risk/_schema.md`](../risk/_schema.md)), the README model
ranking.

`eados_lint` (`os-spec-completeness`) requires the instance to define every key below; the
`examples` gate (#224) shape-checks the worked-example decision surface.

## Required structure

```yaml
version: # integer schema version
tiers: # ordered capability tiers, cheapest -> most capable
efforts: # ordered effort levels, lowest -> highest (OS-neutral vocabulary)
flags: # derived-signal vocabulary beyond tracker labels: flag id -> meaning
defaults: # the floor when no rule matches
tier: # a `tiers` entry (the cheapest — rules only raise)
effort: # an `efforts` entry
rules: # monotonic escalation rules; each { id, when, min_tier, min_effort, why }
catalog: # the ONLY place concrete model names live
as_of: # "YYYY-MM-DD" the catalog was last verified against the market
hosts: # per-host entries { host, models: tier -> name, effort_aliases: alias -> effort }
examples: # worked-example decision surface (#224): verdicts = the tiers
```

## Resolution — monotonic escalation

The recommendation starts at **`defaults`** (the cheapest floor) and every matched rule raises it
to at least the rule's `min_tier` / `min_effort` — the final answer is the **max** over the floor
and all matched rules, using the `tiers` / `efforts` order. A rule **matches** when *all* of its
`when` signals hold. Rules never lower a result — the same only-add-never-relax principle as the
workflow's domain overlays. Resolution is therefore deterministic and order-independent.

**Signal vocabulary** (`when` entries):

- `label:<name>` — the unit of work carries that tracker label (e.g. `label:severity:high`,
`label:adr`, `label:security`).
- `flag:<id>` — a derived flag from `flags:`, asserted by the caller (an evaluator heuristic or a
human judgment): `sets-pattern` (first of its class — the followers inherit a cheaper route) and
`decision-heavy` (the decision is the deliverable).

## Effort vocabulary

`efforts` is OS-neutral: `low` | `medium` | `high` | `max`. Hosts that use different words map
them in their catalog entry's `effort_aliases` (Claude Code: "ultracode" → `max`). Advice is
always *emitted* in the OS vocabulary; an adapter may translate on the way out.

## Invariants

- `defaults.tier` / `defaults.effort` and every rule's `min_tier` / `min_effort` are entries of
`tiers` / `efforts`. Every `flag:` signal references a `flags:` key. Every host's `models:` maps
**every** tier; every `effort_aliases` value is an `efforts` entry. (Shape is linted today;
referential integrity is enforced by the 16.2 evaluator's loud rejection.)
- `catalog.as_of` is refreshed whenever a mapping changes — a stale date is the review cue.
- The policy never contains a model name outside `catalog.hosts[].models`.
- Resolution is **deterministic** — same signals, same advice (no model/LLM in the loop).
- Advice never overrides a human's explicit model/effort choice, and no rule can force a
top-level session model switch — that action does not exist below the human (`AGENTS.md` §6).
108 changes: 108 additions & 0 deletions .eados-core/orchestrator/os/routing/routing.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# EADOS model & effort routing - reference instance. Schema: ./_schema.md (ADR-0017).
# Advisory-first: the OS recommends, the human keeps final model authority. The rules speak
# capability TIERS; concrete model names live ONLY in the dated catalog below, so model churn is
# a catalog edit - never a policy or code change. Evaluator: tools/route_advice.py (M16 16.2).
version: 1

# Ordered cheapest -> most capable / lowest -> highest. Resolution starts at the `defaults` floor
# and matched rules only ever RAISE (max over floor + matches) - the same only-add-never-relax
# principle as the workflow's domain overlays. Deterministic and order-independent.
tiers: [fast, standard, frontier-reasoning]
efforts: [low, medium, high, max]

# Derived signals beyond tracker labels, asserted by an evaluator heuristic or a human judgment.
flags:
sets-pattern: "first of its class - fixes the template; the follower issues inherit a cheaper route"
decision-heavy: "the decision IS the deliverable (ADR / architecture call)"

# The floor: absent any matching rule, work routes to the cheapest tier at the lowest effort
# (doc fixes, small authoring). Everything above is earned by a signal.
defaults:
tier: fast
effort: low

# Signal vocabulary: "label:<tracker label>" or "flag:<flags key>". A rule matches when ALL of
# its `when` signals hold; it raises the result to at least min_tier / min_effort.
rules:
- id: severity-medium
when:
- "label:severity:medium"
min_tier: standard
min_effort: medium
why: "a significant gap deserves substantive work - standard tier, real effort"
- id: severity-high
when:
- "label:severity:high"
min_tier: standard
min_effort: high
why: "core-guarantee work needs sustained effort; tier rises via flags, not severity alone"
- id: security-surface
when:
- "label:security"
min_tier: frontier-reasoning
min_effort: high
why: "on security posture the cost of a subtle miss dwarfs the routing saving"
- id: decision-heavy
when:
- "flag:decision-heavy"
min_tier: frontier-reasoning
min_effort: high
why: "when the decision is the deliverable, reasoning depth is the product"
- id: adr-is-decision-heavy
when:
- "label:adr"
min_tier: frontier-reasoning
min_effort: high
why: "an ADR-bearing issue is decision-heavy by definition"
- id: sets-pattern
when:
- "flag:sets-pattern"
min_tier: frontier-reasoning
min_effort: high
why: "the first of a class fixes the template every follower copies - amortized quality"
- id: foundational-decision
when:
- "flag:decision-heavy"
- "label:severity:high"
min_tier: frontier-reasoning
min_effort: max
why: "a wrong foundational call is the most expensive artifact the factory can produce"

# The ONLY place model names live. `as_of` is refreshed whenever a mapping changes - a stale
# date is the review cue. Hosts with their own effort words map them via effort_aliases.
catalog:
as_of: "2026-07-09"
hosts:
- host: claude-code
models:
frontier-reasoning: "Fable 5"
standard: "Opus 4.8"
fast: "Sonnet 5"
effort_aliases:
ultracode: max
ultrathink: max

# Worked-example decision surface (#224) - the judgment call is WHICH TIER a unit of work
# deserves. Shape-checked by the `examples` gate; drawn from the maintainer-reviewed M15
# assessment of 2026-07-09 (.issues/M16-model-effort-routing-milestone.md, backfill table).
examples:
verdicts: [fast, standard, frontier-reasoning]
cases:
- input: "Doc drift - two playbooks omit a domain the code already ships (severity:low)"
verdict: fast
why: "mechanical prose fix against a known source of truth; no design, small blast radius"
- input: "Author a persona file plus a worked config example (severity:low)"
verdict: fast
why: "authoring to an established format; review catches style drift cheaply"
- input: "Third cross-cutting command doc, after the first fixed the template (severity:medium)"
verdict: standard
why: "substantive but pattern-following; the expensive thinking was spent on the first of the class"
- input: "Reconcile a defect backlog against in-tree fixes and build a regression index (severity:high)"
verdict: standard
why: "meticulous verification, not open-ended design - effort high, tier standard"
- input: "The taxonomy ADR every other issue in the milestone builds on (label:adr, severity:high)"
verdict: frontier-reasoning
why: "decision-heavy and foundational; a wrong call here is repaid with interest downstream"
- input: "First command of a new class - fixes the shape its siblings will copy (sets-pattern)"
verdict: frontier-reasoning
why: "sets-pattern - the marginal cost of the best model is amortized across every follower"
1 change: 1 addition & 0 deletions .eados-core/tools/eados_lint.py
Original file line number Diff line number Diff line change
Expand Up @@ -1155,6 +1155,7 @@ def check_gate_executability(fail):
"orchestrator/questionnaire.yaml", # ask / default
"learning/scope-examples.yaml", # apply / skip (companion to the lessons ledger)
"orchestrator/triage.yaml", # answer / focused-change / five-step-loop (Step-0)
"orchestrator/os/routing/routing.yaml", # fast / standard / frontier-reasoning (tier calls)
)
EXAMPLE_CASE_REQUIRED = ("input", "verdict", "why")

Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ EADOS is a **phase-based agentic delivery operating system**: an opt-in pipeline
its lifecycle. It is *declarative, gate-enforced, and human-gated* (not a runtime kernel); the
design is [RFC-0001](.eados-core/docs/rfc/0001-eados-delivery-os.md), the phases are
[`orchestrator/commands/`](.eados-core/orchestrator/commands/README.md), and the machine-readable
specs (`workflow`, `authority`, `git`, `rfc`, `plan`, `risk`, `contribution`) live under
specs (`workflow`, `authority`, `git`, `rfc`, `plan`, `risk`, `contribution`, `routing`) live under
[`orchestrator/os/`](.eados-core/orchestrator/os/README.md).

The **`scaffold` phase is the factory**: it reproduces the enterprise agent system of
Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,21 @@ in the same PR. Releases follow Semantic Versioning; the latest is **v2.8.0**.

## [Unreleased]

### Added

- **Model & effort routing policy — advisory-first, tiers-not-names (#252, M16 16.1, ADR-0017).**
A new eighth OS spec [`orchestrator/os/routing/`](.eados-core/orchestrator/os/routing/_schema.md)
turns "which model, how much effort?" from a manual per-issue judgment into policy-as-data: work
signals (tracker labels + the `sets-pattern` / `decision-heavy` flags) escalate monotonically
from a cheapest floor to a **capability tier** (`fast`/`standard`/`frontier-reasoning`) and an
OS-neutral **effort** (`low`–`max`, host aliases like "ultracode" → `max`). Concrete model names
live *only* in the dated `catalog:` (today: Sonnet 5 / Opus 4.8 / Fable 5 for Claude Code), so
model churn is a catalog edit — never a policy or code change. The tier call ships as a #224
worked-example decision surface (registered under the `examples` gate); the spec is
auto-discovered by `os-spec-completeness` and covered by the existing `os/**` gate-coverage
entry. Advisory-first per RFC-0001: the human keeps final model authority; the evaluator and its
surfaces land with 16.2–16.4 (#253–#255).

## [2.8.0] - 2026-07-08

### Added
Expand Down