Skip to content

spec-loop-runner.md

Latest commit

 

History

History
203 lines (179 loc) · 11.4 KB

spec-loop-runner.md

File metadata and controls

203 lines (179 loc) · 11.4 KB
Raw

title: Spec-loop runner status: experimental kind: feature mode: infra source: > tools/spec-loop/README.md, tools/spec-loop/loop.sh, tools/spec-loop/lib.sh, tools/spec-loop/tests/test_runner_fixtures.sh, tools/spec-loop/PROMPT_plan.md, tools/spec-loop/PROMPT_build.md, tools/spec-loop/PROMPT_update.md, tools/spec-loop/PROMPT_consolidate.md, and docs/spec-driven-development.md. acceptance:

  • The loop supports plan, build, update, and consolidate beats with one fresh agent context per iteration.
  • Build and update beats never commit to the integration base; they create reviewable local branches and stop before push or PR creation.
  • Open PRs and local work-item branches are passed into plan/build prompts as in-flight work so the loop does not duplicate already-built items.
  • The headless-agent harness contract is explicit for Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Kiro, and Grok Build.
  • The update marker tools/spec-loop/.last-sync is owned by the runner, not by the prompt.

Spec-loop runner

What it does

The spec-loop runner is the framework's local, review-first development loop. It runs a headless agent CLI against fixed prompts, one iteration at a time, to compare specs with the working tree, produce planned changes, back-fill specs after normal contributions, or consolidate a long implementation plan.

The loop is not an autonomous merge system. Its boundary is local branch creation: it may edit, validate, and commit locally, but it never pushes and never opens a pull request.

Where it lives

  • tools/spec-loop/loop.sh — Bash runner for the four beats.
  • tools/spec-loop/lib.sh — deterministic prompt assembly, harness command rendering, agent launch, and .last-sync marker helpers used by the runner and fixture tests.
  • tools/spec-loop/tests/test_runner_fixtures.sh — deterministic fixture tests for prompt assembly, harness command construction, and .last-sync marker helpers; executed by spec-validate.
  • tools/spec-loop/PROMPT_plan.md — gap-analysis prompt that rewrites IMPLEMENTATION_PLAN.md.
  • tools/spec-loop/PROMPT_build.md — implementation prompt for exactly one work item.
  • tools/spec-loop/PROMPT_update.md — spec back-fill prompt for functionality that landed outside the loop.
  • tools/spec-loop/PROMPT_consolidate.md — plan-size reduction prompt.
  • tools/spec-inventory/ — spec-inventory --brief supplies the compact repository inventory appended to every prompt, and spec-scope maps changed paths to the specs whose Where it lives sections name them.
  • tools/spec-loop/AGENTS.md — loop-specific operational context: repository map, validation commands, branch rules, hard limits, and commit rules.
  • tools/spec-loop/README.md — operator quickstart.
  • docs/spec-driven-development.md — full explanation of the loop's posture and lifecycle.

Behaviour & contract

  • Four beats, one mechanism. plan compares specs to code and updates the implementation plan without committing. build implements one uncovered work item. update compares code to specs and back-fills the spec directory. consolidate shrinks the implementation plan without dropping planned work.
  • One work item, one branch, one PR. Build work creates a bare <slug> branch off SPEC_LOOP_BASE and commits exactly one work item there. Update work creates a uniquely named sync-specs-<timestamp> branch. Consolidate commits only the plan file on the control branch.
  • No remote state changes. The runner and prompts forbid git push and gh pr create; each successful build/update beat prints the human-run push and gh pr create --web commands instead.
  • In-flight duplicate guard. Before each relevant iteration, the runner appends open PR context and local work-item branch context to the prompt. Plan/build treat both as in-flight work. This matters because a loop-built branch may be local-only and invisible to GitHub.
  • Control branch vs integration base. TOOLING_REF is captured before checkout. When the integration base does not carry the spec-loop files, the runner tells the agent to read prompts/specs/plan from the control branch with git show, while implementing product changes on the work branch.
  • Update marker ownership. tools/spec-loop/.last-sync records the base SHA last synced by the update beat. The runner reads it to append incremental-scope guidance, then amends or creates a marker commit after the agent finishes. Prompts must not instruct the agent to edit the marker.
  • Deterministic scope mapping. When .last-sync names a previous sync, the runner lists the paths changed since it (git diff --name-only <prev>..<BASE_HEAD> -- .claude/skills tools docs/modes.md), pipes them through spec-scope, and appends the specs it returns as likely relevant. The mapping is advisory: a spec-scope failure is silently skipped and the agent still reads the raw path list.
  • Plan-size hysteresis. Build mode switches to one consolidate pass when IMPLEMENTATION_PLAN.md exceeds SPEC_LOOP_PLAN_MAX, then builds even if planned work alone keeps the file over the threshold. The latch resets once the plan drops below the threshold.

Headless harness contract

Every supported harness must provide the same loop-level behaviour: run one non-interactive agent iteration in the repository root, receive the assembled prompt, allow local edits/validation/commits under the external sandbox, and stop without pushing or opening a PR.

Harness Prompt transport Cwd contract Unattended flag Model override Output mode Extra denial
Claude Code stdin to claude -p launched from repo root --dangerously-skip-permissions --model --output-format --disallowedTools denies push and gh
Codex stdin to codex exec - --cd "$ROOT" --dangerously-bypass-approvals-and-sandbox --model --json for stream JSON external sandbox and exec policy
Cursor positional prompt to cursor agent --print or cursor-agent --print --workspace "$ROOT" --force --trust --model --output-format external sandbox and Cursor policy
Gemini CLI --prompt "<prompt>" launched from repo root --approval-mode default --model --output-format tool-sandboxing and Gemini policy; headless actions needing confirmation are refused
OpenCode positional prompt to opencode run launched from repo root --auto --model --format json for stream JSON external sandbox and OpenCode policy
Kiro positional prompt to kiro-cli chat launched from repo root --no-interactive agent config (.kiro/agents) plain text only external sandbox and the agent-guard --kiro hook
Grok Build --prompt-file <assembled-prompt> --cwd "$ROOT" --always-approve --model plain; streaming-json for stream JSON --deny blocks git push and gh; external sandbox and agent guard remain active

SPEC_LOOP_EFFORT is the harness-neutral reasoning-effort band (low / medium / high). Unset means no effort flag is added. Invalid values fail at startup before the first iteration. Where a CLI offers more levels than the framework band, high maps to that CLI's top value.

Flag names and accepted values below were taken from each CLI's own --help (or the vendor docs when the binary was unavailable) at the time this mapping landed. They are version-dependent: an installed CLI that rejects a mapped value will fail the agent launch mid-loop. When bumping a harness CLI, re-check --help and update this table and spec_loop_launch_agent together.

Harness Effort flag Source low medium high
Claude Code --effort claude --help (low|medium|high|xhigh|max) low medium max
Codex -c model_reasoning_effort=… codex exec --help (-c/--config); key from Codex config low medium xhigh
OpenCode --variant opencode run --help (e.g. high, max, minimal) minimal medium max
Kiro --effort Kiro CLI effort docs (low|medium|high|xhigh|max) low medium max
Cursor (none — omitted silently) cursor agent --help (no effort/thinking-level flag) — — —
Gemini CLI (none — omitted silently) gemini --help (no effort/thinking-level flag) — — —
Grok Build --reasoning-effort grok 1.0.50 --help (--effort alias also accepted) low medium high

SPEC_LOOP_AGENT chooses the CLI. SPEC_LOOP_HARNESS chooses the invocation convention and defaults from the agent basename. Adding a new harness means extending both matrices above (headless contract and effort mapping), documenting the safety boundary, and updating spec_loop_launch_agent in lib.sh (plus the harness case in loop.sh) in the same change.

Out of scope

  • Auto-pushing, auto-opening PRs, auto-merging, or changing remote state.
  • Replacing the project sandbox or adding new filesystem/network permissions.
  • Teaching individual skills how to do their domain work; those contracts live in the corresponding functional specs.
  • Editing docs/rfcs/; RFCs remain the separate governance layer.

Acceptance criteria

  1. loop.sh accepts plan, build, update, and consolidate, plus a bare numeric build count, and rejects unknown modes or non-numeric iteration counts.
  2. Build/update iterations check out SPEC_LOOP_BASE, create a non-base branch, and stop if a commit lands on the base.
  3. Plan/build prompts receive both open PR context and local work-item branch context.
  4. Update prompts receive incremental-scope guidance from .last-sync when present, but the runner remains the only writer of .last-sync.
  5. Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Kiro, and Grok Build are documented in the headless harness matrix and implemented by the runner.
  6. Operator docs and prompts agree on branch naming: build uses bare <slug> branches; update uses sync-specs-<timestamp> branches.
  7. Security docs describe unattended agent flags as harness-specific agent-level bypasses that must run under the external sandbox.

Validation

bash -n tools/spec-loop/loop.sh
shellcheck tools/spec-loop/loop.sh
shellcheck tools/spec-loop/lib.sh tools/spec-loop/tests/test_runner_fixtures.sh
uv run --project tools/spec-validator --group dev spec-validate

Known gaps

  • Cursor, Gemini CLI, OpenCode, and Kiro rely on external policy/config plus the OS sandbox for push/PR denial. Claude Code and Grok Build additionally receive per-invocation hard-deny rules from the runner.
  • The deterministic scope helper this bullet used to call missing has shipped: loop.sh pipes the incremental diff through tools/spec-inventory's spec-scope.
  • The incremental diff's pathspec (.claude/skills tools docs/modes.md) no longer covers where skills live. .claude/skills/magpie-<name> entries are symlinks (through .agents/skills/) into plugins/magpie-<family>/skills/<alias>/, so an edit to a skill's content changes nothing under .claude/skills and drops out of the scoped list; so do plugins/magpie-setup/templates/, .github/, projects/, and every docs/ page other than docs/modes.md.