Decision plane, phase 1: one decide(goal, hints) in jarvisd, knowledge on the decision path, measured speed - #60
Merged
Merged
Conversation
…:8771 serve_decide.py --checkpoint <dir> lazy-imports torch (pointer_infer.py), answers in mode=pointer-lora with real probabilities and shape_concentration, and reports the checkpoint identity hash and its eval gate in /health. Without --checkpoint it stays mode=untrained. dottie-os owns :8770; this dev server defaults to :8771. decision_io gains answer_from_probabilities and checkpoint_identity (the hash a promotion stamp names). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
…traces and the training loop route_goal is the single routing policy for goal strings. MoMA-lite moves into dottie_loop.backends (its only implementation); the orchestrator MLP and System One over /decide are advisory unless gate_passed AND human-stamped, and only for an equal-or-cheaper tier. Default decisions are golden-tested against the pre-unification classifier. Each route appends a trace line (~/.dottie/traces, never under pytest by default); router_training packs real production traces into strict jev records, evaluates a checkpoint against the heuristic through evaluate_gates/promotion_decision, and router_artifacts holds the human promotion stamps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
… scout router scout harness route, harness run (runner step 1) and jarvisd route/plan/run now decide through dottie_loop.router.route_goal; decisions are unchanged (the MoMA-lite goldens pass through the real CLI). Route output keeps every existing key and adds authority, spec_tier and an advisory block. harness run appends its observed outcome to the route's trace. New: scout route (top-level alias) and the router plugin (status, pack, train, eval, promote). dottie-loop becomes a workspace dependency of scout-cli and jarvisd. Test suites pin traces and stamps into their throwaway homes; the state sweep watches ~/.dottie. Soft ruff debt 983 -> 970 as the classifier code left scout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
…nance sidecar Curation packs now carry only schema/id/state/questions/labels, validated by apps/jev-v0/decision_io.py when present; tier, source, consent and pair ids move to curated_pack_v2_provenance.jsonl. read_rows() and the pack smoke still read the old dottie-os-decision-schema-1.0.0 rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
… the real eval lib/orch_infer.py and lib/weights/champion_weights.json were never loaded by the API. The weights were labelled orch-mlp-v1-v5 with no eval behind them while every report names v4. The one weights file is ava-factory's and the one inference path is the router's MLP backend. lib/meta/eval_summary.json is re-vendored from ava-factory's eval_report.json (v4, gate_passed false); corpus_builder's hardcoded hill-climb numbers are labelled simulated and no longer overwrite it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
docs/ROUTER.md describes the router, its backends, authority (gate plus human stamp), traces, the training loop and port ownership. scout's README, INTEGRATION.md and docs/INSTALLER.md install with uv from the Dottie monorepo and say what the curl shim actually writes. .dottie/ is gitignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
The stamp store is declared in the manifest and gated with fs_write; the operator-named destinations (pack --out, train --out, the checkpoint's eval_summary.json, a DOTTIE_ROUTER_STAMPS override) use fs_write_arg. Keeps the fleet invariants in tests/test_policy.py (write:true declares paths, no new ungated write-capable plugin). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
The flywheel cycle's promoted path vendors the freshly trained weights through copy_artifacts._copy_weights and verifies their sha256; removing the function broke three flywheel_cycle tests in codeact-sandbox. Restore it, still skipped by the default main() so nothing is vendored unless the flywheel promotes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
…lder, decide) - context.py: ContextProvider protocol and build_context(goal, providers, budget) -> DecisionContext: bounded (per-provider/total items, text and byte caps), deterministic (sorted, digest over exactly the kept items), per-provider latency and timeout, fail soft. Private items (memories, goals, claims) carry text only under DOTTIE_TRACE_TEXT=1. Adapters: JarvisStateProvider (in-process, duck-typed on jarvisd.state.State), RunHistoryProvider, GraphifyProvider (off unless DOTTIE_CONTEXT_GRAPH). - run_history.py: HistoryIndex, an incremental (size, mtime) + byte-offset index over scout's timeline store with per-file partial aggregates, a reused merge when nothing changed, per-tier success/recovery/latency, and an optional persisted index for cold processes. - backends.system_one_state(goal, context): the ONE state builder for serve and pack; no context -> the pre-context state, so old traces rebuild unchanged. SystemOneBackend: /health cached per URL, one kept-alive http.client connection per thread (Nagle off), one /decide per route. - route_goal(context=...) hands the context to backends that take it and records per-backend latency; traces store the served context summary and state_sha256. - router_training.pack refuses stub-executor and untagged outcomes (counted in MANIFEST executors) and refuses a trace whose rebuilt state drifts. - decide.py: decide(goal, hints) = context -> route_goal -> decision record (latency breakdown, context ids/digest, System One view) with an LRU+TTL cache keyed by goal sha, context digest, backend config and hints. - execution.recovery_ladder: the harness ladder's one implementation. - bench_decide.py: the checked-in decision-latency bench. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
…efix pointer_infer.SharedPrefixScorer tokenizes each branch exactly as training did, finds the longest shared token prefix (never past an option or decide boundary), encodes it once with the KV cache and runs only each question's suffix on a copy of the cache: the same causal computation as encoding every branch whole, without re-encoding the state four times per request. The orchestration is torch-free (tokenize/encode/copy/score injected) and tested with a fake causal model; PointerPredictor.probabilities_many wires the torch pieces, and serve_decide.decide uses it when the predictor offers it. serve_decide speaks HTTP/1.1 with Nagle off so the router's System One backend keeps one connection alive; unread-body errors close the connection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
decide(goal, hints) builds the context from this daemon's store (FTS5 memory recall, open goals, active claims) plus scout's run history and the optional code graph, routes through dottie_loop.decide, and records the decision (context digest and ids, System One answers, latency breakdown, cache hit; the goal text only under DOTTIE_TRACE_TEXT=1) on the timeline as kind=decide. A per-daemon LRU+TTL decision cache is keyed by goal sha, context digest, backend config and hints. /api/route and harness.route stay compatible aliases (old fields plus decision); /api/plan decides the same way. route, plan and decide run in the threadpool instead of blocking the event loop. harness.run passes the same context providers to scout's runner. Tests pin the tool list, the record, the cache, the no-text rule and the threadpool dispatch; conftest points SCOUT_CHECKPOINT_BASE at tmp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
dottie_loop.env.ollama_host / ollama_env_values: OLLAMA_HOST first; the spellings clients grew (OLLAMA_BASE, OLLAMA_URL, DOTTIE_OLLAMA_URL) are accepted as deprecated aliases with a FutureWarning, once per alias per process. scout's llm, ollama, ava, agent and write clients and jarvisd's brain read it through that helper. ollama.candidate_bases now tries OLLAMA_HOST before the aliases (test updated). apps/dottie still reads DOTTIE_OLLAMA_URL; it is folded in Phase 2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
…d outcomes - Plugins register from a checked-in entry table (bigbang/core/plugin_entries.py, regenerated by python -m bigbang.core.plugin_loader --write-entries and held in sync by a test); a plugin module is imported only when its command runs. scout --help still lists every plugin, with the same help text, and imports none. A plugin missing from the table loads eagerly as before. Import failures stay silent on the command line; SCOUT_DEBUG_PLUGINS=1 prints the traceback and scout doctor imports and reports every plugin. SCOUT_EAGER_PLUGINS=1 restores the old behaviour. - bigbang.core.mcp_client imports the MCP SDK on first use. - scout route / harness route ask jarvisd POST /api/decide (JARVIS_URL, JARVIS_BEARER; short connect timeout) and fall back to the same dottie_loop.decide in-process with the run-history provider; the output says decided_by and carries the decision record. - harness run decides through dottie_loop.decide (context providers injectable, jarvisd passes its own), tags each timeline row with tier and executor kind, and tags the outcome executor: stub|real (only the MCP operator is real today). The one recovery ladder comes from dottie_loop.execution; the file-path import and inline replica are gone. - g_history_stats delegates to dottie_loop.run_history (same keys plus per_tier); SCOUT_CHECKPOINT_BASE overrides the base. - conftest: SCOUT_DECIDE_REMOTE=0 and the new env overrides are pinned so the suite never reaches a developer's jarvisd or real stores. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
dottie_loop.execution.recovery_ladder is the one implementation (a rung table with a fail-closed escalate default instead of an == chain). pipeline/recovery_ladder.py and pipeline/checkpoint_manager.py re-export it; bundles/ultra/recovery-ladder.js (imported by nothing) is removed, as is scout's inline replica (previous commit). tests/test_recovery_ladder_one_copy.py pins every (class, side effect, attempt) against the replaced function and checks each former copy now is the one. The three gate-audit baseline entries for the removed == chains (and two already-stale harness-api ones) are pruned; gate_audit --check reports 0 stale. The scout README and install.sh cited a HandoffEnvelope in a bundles/ultra/communication-pacing.js that never existed in this repo; they now point at the real handoff (jarvisd jarvis.send/inbox, scout comms). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
lib/production_routing.py carried its own MoMA-lite classifier, which had drifted from the router (it ignored the keyword overrides in classify_moma). scripts/vendor_router.py now copies the MoMA-lite section of packages/dottie-loop/dottie_loop/backends.py verbatim into lib/moma_lite.py (Vercel deploys this app without the monorepo), stamped with the section's sha256; --check fails when it is stale. production_routing keeps only the API's fail-closed RoutingRejected guards and response shape. tests/test_production_routing_parity.py runs the check and holds the API to the router's goldens. The dormant lib/heuristics.py and lib/vector_router.py (imported by nothing; the tests already forbade importing them) are deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
src/harness was imported by nothing (only docs cited it as src.harness) and its top-level name collided with packages/ava-open-harness's real harness package, which CI smoke-tests. apps/arxiviq/app/acd was a dormant ACD simulation that the app's own contract test forbade importing; the ESLint ignore for it goes too, and the contract test now asserts it stays deleted (node --test test/*.test.mjs: 41 pass). The docs that cited them are marked historical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
The checked-in decision-latency bench (dottie_loop/bench_decide.py) behind two commands. It measures p50/p95 for a cold scout route (fresh interpreter, forced in-process), in-process routing and decide with run-history context, a jarvisd served by uvicorn on a free loopback port (route alias, decide without and with context, decide with System One against a local untrained serve_decide), scout's run-history stats over 400 runs, and the System One backend round trip. It starts its own servers with temp state and writes nothing real. The router plugin manifest now declares its loopback-only network use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
PR #58 was squash-merged into main; its content equals this branch's base, so the 16 add/add and content conflicts all resolve to this branch's side. main's other changes (#57, #59) touch only apps/arxiviq and merge cleanly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
tests/test_recovery_ladder_one_copy.py keeps the replaced ladder verbatim as the reference the one implementation is checked against; its == chain ends in the terminal escalate rung (fail-closed). Accepted with that judgment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
… Phase 2 DAG docs/ARCHITECTURE.md: the four planes, the runtime topology, every service and port, the decide contract and record, which knowledge sources are on the decision path, the learning loop (one state builder, executor-tagged outcomes), the speed budget with the measured before/after bench, the rules, one copy of each thing, and the owner's 2026-09-23 retirement decisions (retire only apps/bluehenre; keep and integrate apps/dottie-rlm, apps/dottie and harness-api). docs/ECOSYSTEM.md keeps only what was aspirational vs true and the sections other documents cite. ROUTER.md, README, the jarvisd README and spec gain decide / harness.decide. project_dag.json: decision-plane-phase1 done (factory done), Phase 2 nodes decision-plane-real-executors, harness-api-jarvisd-proxy, retire-bluehenre, rlm-converge, agent-os-fold. HANDOFF.md: a new top block stamped at the merge commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
CI runners set FORCE_COLOR/GITHUB_ACTIONS and an 80-column terminal, so rich wrapped plugin names in ANSI codes and test_help_lists_every_plugin_ and_imports_none could not find ' a11y '. Pin a wide, colourless terminal for the probe subprocess and strip ANSI before matching. Reproduced with GITHUB_ACTIONS=true FORCE_COLOR=1 COLUMNS=80; passes under both that and a plain local run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
jcdavis131
marked this pull request as ready for review
September 23, 2026 22:49
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
jcdavis131
pushed a commit
that referenced
this pull request
Sep 24, 2026
The block named 07a2146, a claude/decision-plane commit that the squash merge left off main, so check_handoff_fresh.py --check failed lint-and-test on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2 (cherry picked from commit d9aaa9b)
jcdavis131
added a commit
that referenced
this pull request
Sep 24, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #58, which has been squash-merged.
origin/mainis merged in, not rebased; the 16 conflicts were files where main equals this branch's base, so each resolves to this branch's side. The normative description is nowdocs/ARCHITECTURE.md.What changed and why, by deliverable
1. Context builder (
packages/dottie-loop/dottie_loop/context.py)ContextProviderprotocol, plusbuild_context(goal, providers, budget). It returns aDecisionContextthat is:DOTTIE_TRACE_TEXT=1.JarvisStateProvider: runs in-process and only needs an object shaped likejarvisd.state.State. It pulls FTS5 memory recall, open goals and active claims.RunHistoryProvider: success, recovery and latency per tier, plus failure rate per role.GraphifyProvider: off unlessDOTTIE_CONTEXT_GRAPHis set.dottie_loop/run_history.pyreplaces the old full rescan ing_history_stats. It keeps an incremental index keyed by (size, mtime) and byte offset, stores partial totals per file, and reuses the merged result when nothing changed. It can persist the index so a cold process doesn't re-parse.g_history_statskeeps its keys and addsper_tier.SCOUT_CHECKPOINT_BASEoverrides the store location.2. One state builder
system_one_state(goal, context)is used by bothSystemOneBackendwhen serving andscout router packwhen training.state_sha256. The pack refuses any trace whose rebuilt state hashes differently.executor: stub|real. The pack refuses stub outcomes and untagged pre-tag outcomes, and counts both inMANIFEST.executors.3. Decision plane in jarvisd
dottie_loop.decide.decide: build context, callroute_goalwith the context passed to the backends, then write a decision record. The record has a latency breakdown (context, route, each backend, total), the context digest and ids, System One's answers, and whether the cache was hit. The goal text is included only on opt-in.POST /api/decideand MCPharness.decide, and recorded on the jarvisd timeline (kind=decide) and in the trace./api/routeandharness.routestay as compatible aliases: the old fields plusdecision.api_route,api_planandapi_decidenow run in the threadpool; a test checks this.4. Speed
scout --helplists the same 76 rows with the same text and imports no plugin.SCOUT_DEBUG_PLUGINS=1prints the traceback, andscout doctorimports every plugin and reports failures.bigbang.core.mcp_clientimportsmcponly on first use.scout routeandscout harness routetry jarvisd/api/decidefirst (JARVIS_URL/JARVIS_BEARER, 0.1 s connect timeout) and fall back to the same code in-process.SystemOneBackend:/healthis cached per URL, one keep-alivehttp.clientconnection per thread with Nagle off, and one/deciderequest per route carrying all four questions.SharedPrefixScorerencodes the shared state prefix once (KV cache) and runs only each question's suffix. The orchestration doesn't need torch; it is tested with a fake causal model.serve_decidenow speaks HTTP/1.1 with Nagle off.5. Bench:
dottie-loop bench decideandscout router bench, backed bydottie_loop/bench_decide.py.6. One copy of each thing
classify_moma's keyword overrides. It now serveslib/moma_lite.py, a verbatim copy vendored byscripts/vendor_router.py. Its--checkflag and a golden parity test catch drift. The dormantheuristics.pyandvector_router.pyare deleted.dottie_loop.execution.recovery_ladderis the one implementation. The twopipeline/modules re-export it. The inline copy in the runner andbundles/ultra/recovery-ladder.jsare removed. A test pins every input against the old function.OLLAMA_HOSTis the one variable.OLLAMA_BASE,OLLAMA_URLandDOTTIE_OLLAMA_URLstill work but warn that they are deprecated; this covers the scout clients and jarvisd's brain.src/harness(imported by nothing; its name collided with ava-open-harness) andapps/arxiviq/app/acd(imported by nothing).jarvis.send/jarvis.inbox.7. Docs
docs/ARCHITECTURE.mdis new and normative.docs/ECOSYSTEM.mdis now a pointer plus history.ROUTER.md, the README, the jarvisd README and spec, and the HANDOFF top block.Measured before/after
dottie-loop bench decide --n 100 --cold-n 20, run on a 4-CPU Linux container with Python 3.11.15. "Before" isfa04d85. System One rows use a local untrainedserve_decide, so they measure HTTP only and no model runs.route(fresh interpreter)scout --help(fresh interpreter)route_goaldecide+ run-history context/api/route/api/decide, no context/api/decide+ context/api/decide+ context + System OneThe budget targets were 10 ms for warm decide without System One and 250 ms for a cold scout route; both are met. The 150 ms System One on GPU target is not measured here because this box has no GPU or checkpoint.
Verification (local)
uvx ruff@0.15.22 check packages/dottie-loopis clean.uv lock --checkpasses.shell=Truesitesfactory checkSCOUT_CHECKPOINT_BASE,SCOUT_DECIDE_REMOTE=0and the trace dir so tests never write real~/.dottiedata or jarvisd DBs.Retirement and integration (owner decision, 2026-09-23)
apps/bluehenre: archive it in Phase 2, keep its history, and drop its CI job with it.apps/dottie-rlm: converge the three RLM implementations onto one.apps/dottie: fold its missions and research loop into the decision plane, and fix thedottiepackage-name collision./api/decide.apps/ava-factorystays as is (the frozen trainer mirror).Phase 2 (DAG nodes)
decision-plane-real-executors: today nearly every outcome isexecutor: stub, so no pack can be built from real labels. This is expected.harness-api-jarvisd-proxyretire-bluehenrerlm-convergeagent-os-foldBehaviour changes worth a look
deep_research, matching scout and jarvisd.ollama.candidate_basesnow checksOLLAMA_HOSTbefore the old names./api/routenow builds context, which costs about 2 ms more.scout --helplists plugins in sorted order.🤖 Generated with Claude Code
https://claude.ai/code/session_01WH1hPpf3u1xWecEhS6AHD2
Generated by Claude Code