diff --git a/.agents/skills/afk/SKILL.md b/.agents/skills/afk/SKILL.md index 7d03be3ba92..aef9aab2de7 100644 --- a/.agents/skills/afk/SKILL.md +++ b/.agents/skills/afk/SKILL.md @@ -94,25 +94,30 @@ backend (tmux or herdr; see "Auto-discovered supervisor pane" below): - **Primary-pane busy guard** - `pane_is_busy` trusts Herdr native `busy` when available, otherwise matches rendered output against only the detected primary harness's signature. This narrow delivery guard never classifies a recorded worker task and never uses a global union of vendor patterns. -- **Composer-state guard** - `inject_msg` reads the full `empty`/`pending`/`pending-unproven`/`unknown` verdict from `fm_backend_composer_state` and injects only when it is affirmatively `empty`. - Every other or future verdict defers, including an unreadable pane, ambiguous geometry, a blank unidentified row, and a bare shell prompt left after the agent exits. +- **Composer-state guard** - `inject_msg` reads the full `empty`/`pending`/`pending-unproven`/`unknown` verdict from `fm_backend_composer_state`. + Affirmative `empty` injects. + `pending` always defers (a human draft or swallowed digest). + `unknown` defers unless the backend can prove otherwise. + Only herdr can, and only when a fresh styled (ANSI) re-read still shows a genuine agent composer container AND native agent-state is idle, so a false-unknown idle Claude composer cannot stall away-mode overnight. + A dead shell, a modal, an unidentified or blank row, a degraded unstyled read, and a Pi separator pair the identity gate rejected all have no such proof and still defer. + The container rule is one sentence: `unknown` may only mean "proven container, unjudgeable styling", never "proven container, unread text". + Native-hosted away auto-discovers the captain pane (`HERDR_PANE_ID`); it does not need a different flush target. Each adapter contributes only capture and capability facts to the fleet-wide screen classifier in `bin/fm-composer-lib.sh`, which owns every shape and verdict. - It preserves proven idle composers as empty but requires a genuine container around shell glyphs; see `docs/herdr-backend.md` "Composer and injection safety" for the operator contract. - `pane_input_pending` is the tested fail-closed predicate for callers that need to know whether the composer is unsafe: it treats every result except exact `empty` as pending. + See `docs/herdr-backend.md` "Composer and injection safety" for the operator contract. + `pane_input_pending` stays fail-closed for other callers: every result except exact `empty` is pending. -A busy primary pane, or any composer verdict other than `empty`, defers the injection; the buffered escalation survives in `state/.subsuper-escalations` and is retried on the next housekeeping tick. +A busy primary pane, a pending composer, or an unknown composer the backend cannot prove is a live idle agent composer, defers the injection; the buffered escalation survives in `state/.subsuper-escalations` and is retried on the next housekeeping tick. In afk mode the composer guard is belt-and-suspenders (no human is typing), but it protects against the race window between the captain returning and their message landing, a dead shell, and the daemon's own previous injection sitting unsent. -**Max-defer escape (the daemon must never silently wedge).** +**Max-defer recovery (the daemon must never silently wedge).** If anything stays buffered past `FM_MAX_DEFER_SECS` (default 300), the daemon -attempts one normal flush, which still requires an idle pane and an affirmatively empty composer. -The alarm is defense in depth rather than a substitute for keeping every genuinely idle supported composer injectable. +retries the flush path, including herdr native-idle delivery when the composer is unknown. If that submit cannot be confirmed, it raises a loud, rate-limited wedge alarm: an ERROR in the daemon log, a durable `state/.subsuper-inject-wedged` marker (surface it on the "while you were out" catch-up if present), a tmux status-line flash when applicable, and a configurable backend-independent active alert. `docs/wedge-alarm.md` owns the alert channel setup, and `docs/verification/supervision.md` "Wedge-alarm channels" owns active evidence. -So a guard false-positive becomes a visible stall, never an unbounded silent no-op. +A clipped idle composer is supposed to recover on that retry; a remaining stall stays visible instead of an unbounded silent no-op. ## Submit model @@ -172,7 +177,9 @@ the operational prefix lets firstmate distinguish it from a real captain message separator before injection, so submission is unambiguous regardless of harness. - **Busy and composer guards on the supervisor pane** - before injecting, the daemon runs the detected-primary-harness rendered busy guard and reads `fm_backend_composer_state` directly. - Only `empty` permits injection; `pending` protects half-typed or swallowed input, and `unknown` protects unreadable panes and bare dead-shell prompts. + `empty` injects. + `pending` protects half-typed or swallowed input. + `unknown` protects unreadable panes, modals, and dead shells, except on herdr when a fresh styled re-read still proves a genuine agent composer container and native agent-state is idle. Every other result preserves the buffer for retry, so the daemon never merges its digest into the captain's half-typed line or types it into a shell. - The active backend passes its capture plus declarative styled, cursor, identity, and row capabilities to the shared screen classifier; all structural recognition and verdict logic remains in `bin/fm-composer-lib.sh`. Styled captures let that owner remove dim/faint and dark-TRUECOLOR ghost or placeholder text while shape detection uses the ANSI-stripped screen, so a dark border is not lost with ghost content. @@ -180,14 +187,14 @@ the operational prefix lets firstmate distinguish it from a real captain message `FM_COMPOSER_IDLE_RE` overrides the shared idle-placeholder regex, but a match alone never bypasses the classifier's shape-specific position and ANSI de-emphasis safety gates. `FM_BUSY_REGEX` overrides the rendered delivery guards plus Grok's isolated task-state fallback. A blank or otherwise unidentified input row carries no positive container proof and defers injection, so a modal dialog or a mid-redraw pane is never an injection target. -- **Max-defer escape** - the daemon must never silently wedge. If anything stays - buffered past `FM_MAX_DEFER_SECS` (default 300s), the daemon attempts one - normal flush, which still requires an idle pane and an affirmatively empty composer. If that + The herdr unknown override does not weaken that: it requires the same positive container proof from a fresh styled capture, and refuses a bare shell row, a modal, an unidentified row, and any read that fell back to an unstyled capture. +- **Max-defer recovery** - the daemon must never silently wedge. If anything stays + buffered past `FM_MAX_DEFER_SECS` (default 300s), the daemon retries the flush, + including herdr native-idle delivery when the composer is unknown. If that cannot confirm a submit, it raises a loud, rate-limited wedge alarm: ERROR log, durable `state/.subsuper-inject-wedged` marker, a tmux status-line flash when - applicable, and a backend-independent active alert. A - composer false-positive surfaces as a visible stall, never an unbounded silent - no-op. + applicable, and a backend-independent active alert. A remaining stall is + visible; a clipped idle composer is supposed to recover on that retry. - **Verified type-once submit model** - the digest is typed once (`send-keys -l` on tmux, `pane send-text` on herdr), then submitted with Enter and verified. Enter is retried, Enter only and never a retype, until the backend submit diff --git a/.agents/skills/project-management/SKILL.md b/.agents/skills/project-management/SKILL.md index 8feb522bd0c..9690b7d695a 100644 --- a/.agents/skills/project-management/SKILL.md +++ b/.agents/skills/project-management/SKILL.md @@ -42,12 +42,18 @@ Choose that posture when adding or creating the project: - `direct-PR` pushes and opens a PR without the no-mistakes pipeline. - `local-only` has no required remote or PR and lands only through the approved local fast-forward path. - `no-mistakes-prod-only` is a conditional policy rather than one flat mode: genuinely internal-only tooling, automation, contributor or operator process, and release or submission work ships `direct-PR`, while product-facing, mixed, and uncertain work ships `no-mistakes`. +- `+hardened` is the highest-rigor choice on this list, adding the quality gate that runs before validation; it rides alongside one of the flat modes above rather than replacing it, so a hardened project is registered as `[no-mistakes +hardened]`, `[direct-PR +hardened]`, or `[local-only +hardened]`. `no-mistakes-prod-only` is the default for a newly added or created remote-backed project when the captain specifies nothing, and a project with no remote defaults to `local-only`. State that resolved default while confirming the source, local name, and posture instead of asking the captain to choose from scratch, and record a flat mode instead whenever they ask for one. Existing registry entries keep the meaning they already have and are never migrated or reinterpreted, so a legacy entry with no bracket stays `no-mistakes`. Registering a conditional policy is a one-time choice and never requires classifying any change; the per-task surface classification happens at each task's intake, and internal-only is never inferred from file location or project name. +`+hardened` is off for every project unless the captain asks for it, so a project registered without it is `standard`. +Refuse `+hardened` together with `no-mistakes-prod-only` and tell the captain to pick a flat delivery mode instead. +A conditional policy decides per task, so a quality standard that covers only part of a project is a posture nobody can state in one sentence. +`AGENTS.md` section 7 owns how each task's quality resolves at intake, and `bin/fm-project-mode.sh --quality` owns how the registered token is read. + The optional `+yolo` posture changes routine approval authority but does not change the delivery mode. Default it off for every project and every posture, and enable it only on the captain's explicit instruction. `AGENTS.md` section 7 owns the complete authority boundary and exceptions when it is on. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fcfc4cb2dfc..4a17474e142 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -129,10 +129,10 @@ jobs: tests-portable-serial: name: Behavior portable serial ${{ matrix.shard }} runs-on: ubuntu-latest - # Measured whole remainder is ~19 min of serial work; the balanced shards - # are ~4.8 min each. Cap is a hang tripwire with roughly 3x margin, not the - # expected healthy end of the lane. - timeout-minutes: 15 + # Measured whole remainder is ~42 min of serial work; the balanced shards + # are ~10.5 min each, plus about 2 min of setup. Cap is a hang tripwire with + # roughly 2x margin, not the expected healthy end of the lane. + timeout-minutes: 25 strategy: # Every shard reports so one failure never hides another shard's result. fail-fast: false diff --git a/AGENTS.md b/AGENTS.md index 81dc761f8c2..17166ec38ce 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -286,6 +286,7 @@ Resolve every ship task's concrete delivery mode and yolo posture at intake, and A current explicit captain instruction wins; otherwise the project's registry entry is the captain's standing posture, and dropping below its rigor needs a reason you can state. On a `no-mistakes-prod-only` project, classify the task's surface: internal-only tooling, automation, contributor or operator process, and release or submission work ships `direct-PR`, while product-facing, mixed, and uncertain work ships `no-mistakes`; never infer internal-only from file location or project name. An unregistered project or absent registry resolves to `no-mistakes` with yolo off, and the registration gap goes to the captain. +A task's quality posture resolves at intake with the same precedence, a current explicit captain instruction first, then the project's registered posture, then `standard`, with the one-line reason for any deviation recorded in the same backlog note. Record the resulting mode, yolo, and the one-line reason for any deviation in the backlog item note. Treat file or subsystem overlap as a risk signal rather than an automatic reason to wait, and dispatch isolated work immediately with no concurrency cap when each change can be independently implemented and validated and the selected delivery path can reconcile ordinary rebases or conflicts. diff --git a/README.md b/README.md index 8ed5226b171..4446ed5a39f 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ Launching a supported harness inside it instantiates your first mate - and makes - **A visible crew** - every crewmate works in its own tmux window, experimental herdr/zellij tab, cmux workspace, or Orca terminal you can watch or type into; the first mate reconciles. - **Disposable worktrees** - each task runs in a clean [treehouse](https://github.com/kunchenguid/treehouse) git worktree, or an Orca-managed worktree when `backend=orca`, so parallel work on one repo never collides. - **Two task shapes** - ship tasks deliver authorized changes; scout tasks leave standalone investigation reports when the intake contract warrants separate research. -- **Explicit project modes** - each project ships via `no-mistakes`, `direct-PR`, or `local-only`, with an optional `+yolo` autonomy flag. +- **Explicit project modes** - each project ships via `no-mistakes`, `direct-PR`, `local-only`, or one of those plus `+hardened` for the highest-rigor quality gate, with an optional `+yolo` autonomy flag. - **Optional secondmates** - opt in to persistent second mates that run from isolated firstmate homes with their own `FM_HOME`, state, projects, and session lock, either locally or as a whole home on an SSH-reachable host, with guarded updates and recovery that never turns an unavailable remote route into a local replacement. - **Event-driven, zero-token supervision** - a bash watcher sleeps on the fleet and wakes the first mate only when something needs you; verified primary harnesses also get a turn-end backstop that blocks or follows up on a blind stop when work is under way and supervision is not live. - **Optional Relay** - opt in with one local `.env` pairing token so firstmate can answer your public mentions on X and Discord alike, act on normal reversible mention requests through the same lifecycle as chat requests, acknowledge spawned work, and post up to three public-safe completion follow-ups within seven days for genuine milestones and the final outcome without changing non-Relay behavior; a final reply promised in a thread becomes durable state that is reconciled from disk, so a restart or a compacted conversation cannot lose it; dry-run preview records would-be replies and dismissals locally before go-live. diff --git a/bin/backends/herdr.sh b/bin/backends/herdr.sh index 1516b2c44e1..22d13910726 100644 --- a/bin/backends/herdr.sh +++ b/bin/backends/herdr.sh @@ -2602,14 +2602,18 @@ fm_backend_herdr_capture() { # printf '%s' "$out" | tail -n "$lines" } -fm_backend_herdr_capture_ansi() { # +# fm_backend_herdr_capture_ansi: the live viewport, styled. Composer +# classification needs what is on screen now. `recent` is scrollback, and +# tailing it to FM_COMPOSER_CAPTURE_LINES can drop Claude's opening ─ so an +# idle-between-turns pane classifies unknown and away-mode never injects. +# `visible` is already viewport-bounded, so the result is not tailed and the +# only job left is clamping --lines up to 200, where the small-N empty-read +# bug cannot apply. +fm_backend_herdr_capture_ansi() { # [lines] fm_backend_herdr_target_ready "$1" || return 1 - local lines=${2:-200} fetch out - case "$lines" in ''|*[!0-9]*) lines=200 ;; esac - fetch=$lines + local fetch=${2:-200} case "$fetch" in ''|*[!0-9]*) fetch=200 ;; *) [ "$fetch" -ge 200 ] || fetch=200 ;; esac - out=$(fm_backend_herdr_cli "$FM_BACKEND_HERDR_SESSION" pane read "$FM_BACKEND_HERDR_PANE" --source recent --lines "$fetch" --format ansi 2>/dev/null) || return 1 - printf '%s' "$out" | tail -n "$lines" + fm_backend_herdr_cli "$FM_BACKEND_HERDR_SESSION" pane read "$FM_BACKEND_HERDR_PANE" --source visible --lines "$fetch" --format ansi 2>/dev/null } # --- herdr composer capture and capability primitives ----------------------- @@ -2638,25 +2642,39 @@ fm_backend_herdr_composer_identity() { # -> "\t" fm_backend_herdr_agent_identity_raw "$FM_BACKEND_HERDR_SESSION" "$FM_BACKEND_HERDR_PANE" } -# fm_backend_herdr_composer_state: thin adapter - capture plus capabilities -# in, shared verdict out. The ANSI capture is preferred (styled=1 lets the +# fm_backend_herdr_composer_read: the ONE capture-classify-resolve-identity +# block this adapter has. The ANSI capture is preferred (styled=1 lets the # shared classifier strip ghost/placeholder text); when it fails on an older # herdr, the plain capture degrades the descriptor to styled=0 rather than # letting ghost text be misread as typed input. Identity is fetched lazily, # only when the classifier reports the verdict depends on it (a pi separator # pair below every other candidate), preserving this adapter's original # consult-only-when-needed behavior. -fm_backend_herdr_composer_state() { # -> empty|pending|pending-unproven|unknown - local target=$1 cap caps verdict identity - fm_backend_herdr_parse_target "$target" || { printf 'unknown'; return 0; } +# +# Returns 1 when no capture succeeded, and with [styled-only]=1 also when the +# ANSI capture failed - the away-mode override needs that refusal, because the +# plain fallback spells real typed text `unknown` instead of `pending`. +# Otherwise it publishes the whole read through globals, so a caller can act +# on the verdict AND the exact screen and identity that produced it without +# re-capturing: +# FM_BACKEND_HERDR_COMPOSER_CAP the captured screen +# FM_BACKEND_HERDR_COMPOSER_VERDICT resolved verdict, never need-identity +# FM_BACKEND_HERDR_COMPOSER_IDENTITY the identity consulted, else empty +# Both consumers share this body so the override's styled and container rules +# can never drift from the verdict they are qualifying. +fm_backend_herdr_composer_read() { # [styled-only] + local target=$1 styled_only=${2:-0} cap caps styled verdict identity='' + fm_backend_herdr_parse_target "$target" || return 1 if cap=$(fm_backend_herdr_capture_ansi "$target" "$FM_COMPOSER_CAPTURE_LINES" 2>/dev/null); then - caps=$(printf 'styled=1\ncursor=0\nidentity=1\nrows=%s' "$FM_COMPOSER_CAPTURE_LINES") + styled=1 + elif [ "$styled_only" = 1 ]; then + return 1 elif cap=$(fm_backend_herdr_capture "$target" "$FM_COMPOSER_CAPTURE_LINES"); then - caps=$(printf 'styled=0\ncursor=0\nidentity=1\nrows=%s' "$FM_COMPOSER_CAPTURE_LINES") + styled=0 else - printf 'unknown' - return 0 + return 1 fi + caps=$(printf 'styled=%s\ncursor=0\nidentity=1\nrows=%s' "$styled" "$FM_COMPOSER_CAPTURE_LINES") verdict=$(fm_composer_classify_screen "$caps" "$cap") if [ "$verdict" = need-identity ]; then if ! identity=$(fm_backend_herdr_composer_identity "$target" 2>/dev/null) || [ -z "$identity" ]; then @@ -2665,7 +2683,48 @@ fm_backend_herdr_composer_state() { # -> empty|pending|pending-unprove verdict=$(fm_composer_classify_screen "$caps" "$cap" '' "$identity") [ "$verdict" != need-identity ] || verdict=unknown fi - printf '%s' "$verdict" + FM_BACKEND_HERDR_COMPOSER_CAP=$cap + FM_BACKEND_HERDR_COMPOSER_VERDICT=$verdict + FM_BACKEND_HERDR_COMPOSER_IDENTITY=$identity + return 0 +} + +# fm_backend_herdr_composer_state: the fleet-wide verdict contract - capture +# plus capabilities in, shared verdict out. An unreadable pane is unknown. +fm_backend_herdr_composer_state() { # -> empty|pending|pending-unproven|unknown + fm_backend_herdr_composer_read "$1" || { printf 'unknown'; return 0; } + printf '%s' "$FM_BACKEND_HERDR_COMPOSER_VERDICT" +} + +# fm_backend_herdr_composer_unknown_deliverable: the narrow away-mode override +# behind an `unknown` composer verdict. It succeeds ONLY when every one of +# these holds, re-read fresh so a human who started typing since the verdict +# still wins: +# 1. The STYLED ANSI capture succeeded. The plain fallback spells real typed +# text `unknown` instead of `pending` (see the styled=0 degradation in +# bin/fm-composer-lib.sh), so a degraded read must keep deferring or the +# digest merges into a human's half-typed line. +# 2. The classifier still says `unknown` - proven `pending` never delivers. +# 3. The screen carries a genuine agent composer container whose content the +# classifier actually read. A pane whose harness exited leaves a bare +# login-shell row, and herdr keeps reporting agent_status=done (which +# maps to idle), so native state alone cannot tell a waiting agent from a +# dead shell. A modal or mid-redraw pane has no container either. Neither +# does a pi separated pair the identity gate rejected, because that +# verdict is reached without reading the region's text. +# 4. Native agent-state is idle: positive proof a registered agent is +# waiting between turns rather than mid-turn. +# Together these let a false-unknown composer (a clipped idle Claude the +# classifier cannot prove empty) deliver instead of stalling away mode for +# hours, without widening the target set any further. +fm_backend_herdr_composer_unknown_deliverable() { # + local target=$1 + fm_backend_herdr_composer_read "$target" 1 || return 1 + [ "$FM_BACKEND_HERDR_COMPOSER_VERDICT" = unknown ] || return 1 + fm_composer_screen_has_agent_container \ + "$FM_BACKEND_HERDR_COMPOSER_CAP" "$FM_BACKEND_HERDR_COMPOSER_IDENTITY" || return 1 + [ "$(fm_backend_herdr_busy_state "$target" 2>/dev/null)" = idle ] || return 1 + return 0 } # fm_backend_herdr_rendered_busy_state: busy|idle|unknown from the pane's diff --git a/bin/fm-backend.sh b/bin/fm-backend.sh index 2882f4a6af2..0b24c139233 100644 --- a/bin/fm-backend.sh +++ b/bin/fm-backend.sh @@ -819,6 +819,23 @@ fm_backend_composer_state() { # [expected-label] -> empty|pe esac } +# fm_backend_composer_unknown_deliverable: may a caller act on an `unknown` +# composer verdict for on other evidence? Default NO. Only a backend +# with a native agent-registration probe can answer yes, and only when that +# probe plus a styled re-read prove a live agent composer is waiting between +# turns; every other backend keeps `unknown` a hard defer. The away-mode +# daemon uses this so a false-unknown composer cannot stall delivery for +# hours, without any backend shape knowledge leaking into the daemon. +fm_backend_composer_unknown_deliverable() { # + local backend=$1 + shift + fm_backend_source "$backend" || return 1 + case "$backend" in + herdr) fm_backend_herdr_composer_unknown_deliverable "$@" ;; + *) return 1 ;; + esac +} + # fm_backend_target_exists: cheap, READ-ONLY existence check - does the # recorded TARGET endpoint still exist on BACKEND? Never starts a server or # session: for herdr this deliberately queries the pane directly instead of diff --git a/bin/fm-brief.sh b/bin/fm-brief.sh index 595557523e7..3941dd259a8 100755 --- a/bin/fm-brief.sh +++ b/bin/fm-brief.sh @@ -6,7 +6,7 @@ # description, acceptance criteria, and context, and may adjust other sections # when the task genuinely deviates (e.g. working an existing external PR instead # of shipping a new one). -# Usage: fm-brief.sh --mode [--herdr-lab] +# Usage: fm-brief.sh --mode [--quality ] [--herdr-lab] # fm-brief.sh --scout [--herdr-lab] # fm-brief.sh --dreamer [--herdr-lab] # fm-brief.sh --secondmate {...|--no-projects} @@ -48,9 +48,22 @@ # "Delivery contract: mode=" line. bin/fm-spawn.sh reads that line and refuses # to launch a ship task whose explicit --mode disagrees, so an adjusted brief and the # recorded task metadata cannot drift apart. +# --quality is the task's quality posture, resolved at intake the same way (AGENTS.md +# section 7) from the project's registered "+hardened" annotation, and it defaults to +# standard so every existing call site scaffolds exactly as before: +# standard the ordinary path: implement, then the mode's definition of done +# hardened a clean loop then a harden loop, both against the base commit fixed at +# spawn, both before validation, driven by bin/fm-quality.sh +# A hardened brief carries the sibling machine-readable line +# "Quality contract: quality=hardened" plus one short quality-gate section; a standard +# brief carries neither, so an absent line means standard and a standard brief stays +# byte-identical to what this scaffold produced before --quality existed. bin/fm-spawn.sh +# checks that line against its own --quality exactly as it checks the mode line. +# --quality is refused on scout, dreamer and secondmate scaffolds, for the same reason +# --mode is. # Ship briefs begin with a worktree-isolation assertion before the branch step. -# --mode is refused on scout and secondmate scaffolds: a scout's deliverable is a -# report rather than a merge, and a charter is not a delivery contract. +# --mode is refused on scout, dreamer and secondmate scaffolds: a scout or dreamer +# delivers a report rather than a merge, and a charter is not a delivery contract. # There is no --yolo flag here. The worker never owns approval decisions, so yolo is # a spawn-time and firstmate-side input only (AGENTS.md section 7). # Every scaffold's status protocol distinguishes the configured @@ -115,6 +128,8 @@ HERDR_LAB=0 NO_PROJECTS=0 MODE= MODE_SET=0 +QUALITY=standard +QUALITY_SET=0 POS=() want_value= for a in "$@"; do @@ -124,6 +139,7 @@ for a in "$@"; do esac case "$want_value" in mode) MODE=$a; MODE_SET=1 ;; + quality) QUALITY=$a; QUALITY_SET=1 ;; *) echo "error: internal parser state for --$want_value" >&2; exit 1 ;; esac want_value= @@ -137,6 +153,8 @@ for a in "$@"; do --no-projects) NO_PROJECTS=1 ;; --mode) want_value=mode ;; --mode=*) MODE=${a#--mode=}; MODE_SET=1 ;; + --quality) want_value=quality ;; + --quality=*) QUALITY=${a#--quality=}; QUALITY_SET=1 ;; # yolo never reaches the worker: it is firstmate's approval authority, not a # brief input. Refuse it loudly so it is never silently dropped here and then # believed to have been recorded. @@ -164,6 +182,19 @@ elif [ "$MODE_SET" -eq 1 ]; then echo "error: --mode applies only to ship briefs; a scout or dreamer delivers a report and a secondmate charter is not a delivery contract" >&2 exit 1 fi + +# Quality posture. Unlike --mode it has a safe default, so it is optional and only +# its VALUE is closed-set validated; a typo must never quietly scaffold a standard +# brief for a task firstmate resolved as hardened. +if [ "$KIND" = ship ]; then + case "$QUALITY" in + standard|hardened) ;; + *) echo "error: --quality must be one of standard, hardened (got '$QUALITY')" >&2; exit 1 ;; + esac +elif [ "$QUALITY_SET" -eq 1 ]; then + echo "error: --quality applies only to ship briefs; a scout or dreamer delivers a report and a secondmate charter is not a delivery contract" >&2 + exit 1 +fi [ "${#POS[@]}" -ge 1 ] || { echo "error: task id is required" >&2; exit 1; } ID=${POS[0]} @@ -479,17 +510,40 @@ echo "scaffolded: $BRIEF (dreamer; replace {TASK})" exit 0 fi +# The DOD's machine-readable contract header, owned in one place so the three +# mode bodies below cannot drift apart. A standard task emits the delivery line +# alone, exactly as this scaffold did before --quality existed; a hardened task +# adds the sibling quality line that bin/fm-spawn.sh checks against its own +# explicit --quality before launching, the same way it checks the delivery line. +CONTRACT_LINES="Delivery contract: mode=$MODE" +if [ "$QUALITY" = hardened ]; then + CONTRACT_LINES="$CONTRACT_LINES +Quality contract: quality=hardened" +fi + +# The hardened task's extra instructions. Deliberately short: bin/fm-quality.sh +# and its --help own the loop's mechanics, and a second copy here would drift. +IFS= read -r -d '' QUALITY_SECTION <" line that bin/fm-spawn.sh checks against its own -# explicit --mode before launching. +# delivery mode, validated above. Each body opens with $CONTRACT_LINES, built once +# just above. case "$MODE" in direct-PR) SETUP2="" RULE1='1. Never push to the default branch (push only your `fm/'"$ID"'` branch). Never merge a PR.' IFS= read -r -d '' DOD <"\` to start, and \`no-mistakes axi respond\` for each gate. @@ -544,6 +598,14 @@ esac # briefs stay byte-identical to the historical Bash 5 output. DOD=${DOD%$'\n'} +# A standard task's brief body is unchanged by --quality existing: nothing is +# prepended, so it stays byte-identical to the pre-quality scaffold. +if [ "$QUALITY" = hardened ]; then + DOD="$QUALITY_SECTION + +$DOD" +fi + cat > "$BRIEF" <. 1 covers every screen the classifier +# could not attribute to an agent composer at all: a dead login-shell prompt, +# a modal or mid-redraw pane with no identifiable input row, and a clipped +# structure. +# +# fm_composer_classify_screen answers `unknown` for BOTH "no container" and +# "a proven container whose content I cannot judge". A caller that wants to +# act on `unknown` on other evidence needs those apart, because typing into a +# no-container screen is exactly the hazard the strict blank-row rule exists +# to prevent. This is that split, kept here so no backend re-derives it. +# +# The pi separated pair needs [identity] - the same "\t" the +# adapter already fetched for the verdict - because _fm_composer_pi_verdict +# answers `unknown` and RETURNS EARLY, before reading a single content row, +# whenever the pair is over-tall or the identity is not exactly pi. Without +# the identity gate that early exit would let unread typed text pass as a +# proven container, which is the same leak the styled-capture rule closes for +# the degraded read. Only an exactly-pi identity over a PAIR_VALID region has +# had its content judged, so only that counts as a container here. +# +# Cursor-less only: it answers about the same selection cursor-less adapters +# (herdr, cmux, orca, zellij) already classify with. +fm_composer_screen_has_agent_container() { # [identity] + local identity=${2:-} plain + plain=$(printf '%s\n' "$1" | fm_composer_strip_ansi) + _fm_composer_scan_screen "$plain" '' + _fm_composer_select_cursorless "$plain" || return 1 + if [ "$FM_COMPOSER_SELECTED_KIND" = pi ]; then + [ "$FM_COMPOSER_SCAN_PI_PAIR_VALID" = 1 ] || return 1 + [ -n "$identity" ] || return 1 + [ "$identity" != probe-absent ] || return 1 + [ "${identity%%$'\t'*}" = pi ] || return 1 + fi + return 0 +} + # fm_composer_submit_retry_core: the ONE verify-and-retry-Enter submit loop # for the cursor-less backends (cmux, orca, zellij), parameterised by the # adapter's send-key and composer-state functions. The caller has already diff --git a/bin/fm-project-mode.sh b/bin/fm-project-mode.sh index 6a97ce2dfed..f739a50d17a 100755 --- a/bin/fm-project-mode.sh +++ b/bin/fm-project-mode.sh @@ -15,6 +15,12 @@ # - - (added ) -> no-mistakes off (legacy default) # - [] - (added ) -> off # - [ +yolo] - (added ) -> on +# - [ +yolo +hardened] - ... -> on, quality hardened +# +# Bracket grammar: the first token that does not begin with "+" is the mode, and +# every "+" token is position-independent. A "+" this version does not +# recognize is ignored rather than refused, so an older firstmate reading a newer +# registry keeps resolving the posture it does understand. # # Registered modes: # no-mistakes full pipeline -> PR -> configured merge authority (default) @@ -30,12 +36,34 @@ # AGENTS.md section 7 is the single owner of authority exceptions, including # ask-user contract expansion and stronger captain boundaries. # +# +hardened = the registered quality posture. From the captain's side this is the +# fourth option on the same list he picks from when he registers a project, after +# no-mistakes, direct-PR and local-only; mechanically it is a separate token, so a +# hardened project still carries one of those modes too. It is read with --quality +# rather than through the two-word line, which is unchanged. +# Absent means "standard": the ordinary path, with no extra quality loop. +# # --raw prints the registered annotation unmapped, so a caller that must tell a # conditional policy apart from a flat mode sees "no-mistakes-prod-only" itself. # +# --quality prints ONE word instead, "standard" or "hardened". It is a separate +# output path precisely so the two-word stdout contract above stays untouched. +# # An unknown/missing project or unknown mode falls back to "no-mistakes off" and warns # to stderr, so a typo never silently drops the gate. -# Usage: fm-project-mode.sh [--raw] +# The quality posture resolves independently of that fallback. +# A missing registry file, or a project absent from the registry, does yield "standard". +# An unrecognised mode token resets only the mode and the yolo flag and keeps a +# "+hardened" parsed beside it, because a typo in the mode must not silently drop the +# quality gate too; the unknown-mode warning still goes to stderr. +# "+hardened" beside "no-mistakes-prod-only" is the opposite case and drops to +# "standard" with its own stderr warning: a hardened project must pick a flat delivery +# mode, because a conditional policy decides per task and a quality standard covering +# only part of a project is not a statable posture +# (.agents/skills/project-management/SKILL.md "Delivery posture"). Unlike the typo, +# that combination parses cleanly and was ruled out on purpose. The mode still resolves +# to no-mistakes-prod-only, the two-word stdout is unchanged, and the exit stays 0. +# Usage: fm-project-mode.sh [--raw] [--quality] set -eu SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -44,50 +72,75 @@ FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" REG="$DATA/projects.md" RAW=0 -if [ "${1:-}" = "--raw" ]; then - RAW=1 - shift -fi -NAME=${1:?usage: fm-project-mode.sh [--raw] } +QUALITY_ONLY=0 +while [ "$#" -gt 0 ]; do + case "$1" in + --raw) RAW=1; shift ;; + --quality) QUALITY_ONLY=1; shift ;; + *) break ;; + esac +done +NAME=${1:?usage: fm-project-mode.sh [--raw] [--quality] } + +# One owner of the output shape, so the two-word default and the one-word +# --quality answer cannot drift apart across the fallback paths below. +emit() { # + if [ "$QUALITY_ONLY" -eq 1 ]; then + echo "$3" + else + echo "$1 $2" + fi +} if [ ! -f "$REG" ]; then echo "warn: no registry at $REG; defaulting $NAME to no-mistakes off" >&2 - echo "no-mistakes off" + emit no-mistakes off standard exit 0 fi -# awk emits " " (one line) or nothing if the project is absent. +# awk emits " " (one line) or nothing if the project is +# absent. A "+" token is never a mode, in any position, so the mode is the +# first bracket token that does not begin with "+". parsed=$(awk -v n="$NAME" ' $1=="-" && $2==n { - mode="no-mistakes"; yolo="off"; + mode="no-mistakes"; yolo="off"; quality="standard"; have_mode=0; if ($3 ~ /^\[/) { s=""; for (i=3; i<=NF; i++) { s = s (s==""?"":" ") $i; if ($i ~ /\]$/) break } gsub(/^\[|\]$/, "", s); # strip the surrounding brackets k = split(s, a, " "); - if (a[1] != "" && a[1] != "+yolo") mode = a[1]; - for (j=1; j<=k; j++) if (a[j]=="+yolo") yolo="on"; + for (j=1; j<=k; j++) { + if (a[j]=="+yolo") yolo="on"; + else if (a[j]=="+hardened") quality="hardened"; + else if (a[j] != "" && substr(a[j], 1, 1) != "+" && !have_mode) { mode=a[j]; have_mode=1 } + } } - print mode, yolo; exit + print mode, yolo, quality; exit } ' "$REG") if [ -z "$parsed" ]; then echo "warn: project \"$NAME\" not in registry; defaulting to no-mistakes off" >&2 - echo "no-mistakes off" + emit no-mistakes off standard exit 0 fi -mode=${parsed%% *} -yolo=${parsed##* } +read -r mode yolo quality <&2; mode=no-mistakes; yolo=off ;; esac case "$yolo" in on|off) ;; *) yolo=off ;; esac +case "$quality" in standard|hardened) ;; *) quality=standard ;; esac +if [ "$mode" = no-mistakes-prod-only ] && [ "$quality" = hardened ]; then + echo "warn: +hardened is refused alongside the conditional policy no-mistakes-prod-only for $NAME; a hardened project must pick a flat delivery mode (no-mistakes, direct-PR or local-only), so defaulting quality to standard" >&2 + quality=standard +fi # A conditional policy is not a task mode. Mechanical callers get its most # rigorous leg; --raw callers get the annotation itself (see the header). if [ "$RAW" -eq 0 ] && [ "$mode" = no-mistakes-prod-only ]; then mode=no-mistakes fi -echo "$mode $yolo" +emit "$mode" "$yolo" "$quality" diff --git a/bin/fm-promote.sh b/bin/fm-promote.sh index 0ed1fd06161..868feb6f543 100755 --- a/bin/fm-promote.sh +++ b/bin/fm-promote.sh @@ -9,8 +9,15 @@ # A scout records no delivery posture, so promotion is where this task's delivery # contract is decided: --mode and --yolo are REQUIRED and written into the meta # alongside the kind= flip. Firstmate resolves both at promotion time, having just -# read the scout's report (AGENTS.md section 7); data/projects.md holds the -# captain's standing posture as context, and this script never looks it up. +# read the scout's report (AGENTS.md section 7). data/projects.md holds the captain's +# standing posture, and this script reads only its quality half, only to print one +# advisory notice on stderr after a successful promotion; the delivery mode stays the +# caller's explicit decision. +# A promoted task deliberately records no quality= and no base_sha=. The base commit +# cannot be captured here, because the promoted worker resets to a clean +# default-branch base only afterwards, and a hardened record with no anchor would look +# complete to the quality loop while being unanchored. Both keys belong to the task +# that owns the base-capture question (bin/fm-quality.sh). # no-mistakes-prod-only is a registry policy rather than a task mode and is refused. # Usage: fm-promote.sh --mode --yolo set -eu @@ -120,6 +127,19 @@ TMP= fm_lock_release "$META_LOCK" META_LOCK_HELD=0 +# The quality sibling of the standing-posture notice in bin/fm-spawn.sh: advisory +# only, printed after the record is already rewritten so it can never affect the +# promotion. A record with no project=, a missing registry, or a failed lookup simply +# skips it, exactly as the spawn notice tolerates an empty standing mode. +PROMOTED_PROJECT=$(sed -n 's/^project=//p' "$META" | tail -n 1) +PROMOTED_PROJECT=${PROMOTED_PROJECT##*/} +if [ -n "$PROMOTED_PROJECT" ]; then + STANDING_QUALITY=$("$FM_ROOT/bin/fm-project-mode.sh" --quality "$PROMOTED_PROJECT" 2>/dev/null) || STANDING_QUALITY= + if [ "$STANDING_QUALITY" = hardened ]; then + echo "notice: $ID promotes carrying no quality posture while the standing posture for $PROMOTED_PROJECT is hardened - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 + fi +fi + HOME_Q=$(printf '%q' "$FM_HOME") echo "promoted $ID to ship mode=$MODE yolo=$YOLO (teardown protection restored)" echo "next: FM_HOME=$HOME_Q bin/fm-send.sh fm-$ID ''" diff --git a/bin/fm-quality-receipt.sh b/bin/fm-quality-receipt.sh new file mode 100755 index 00000000000..e5cdf33d34d --- /dev/null +++ b/bin/fm-quality-receipt.sh @@ -0,0 +1,366 @@ +#!/usr/bin/env bash +# fm-quality-receipt.sh - validate a quality-gate receipt against the D2 schema. +# +# The committed schema at docs/quality-receipt.schema.json is the owner of the +# JSON shape. This script is the check. docs/quality-gate.md owns the rationale, +# the D1 bounds including budget_minutes, and the verify-envelope decision. +# +# Usage: +# fm-quality-receipt.sh validate [--check-head ] [|-] +# fm-quality-receipt.sh schema +# fm-quality-receipt.sh -h | --help +# +# validate reads one JSON document from or stdin. +# Exit 0 on a valid receipt, 1 on an invalid one, 2 on usage or tool errors. +# +# Post-schema rules, because JSON Schema cannot state them: +# - finding ids are unique inside each findings array +# - each verify child's base_sha and head_sha equal the envelope's +# --check-head then requires head_sha to resolve to that tree's HEAD. +# FM_QUALITY_RECEIPT_SCHEMA overrides the schema path (test seam). +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +SCHEMA="${FM_QUALITY_RECEIPT_SCHEMA:-$FM_ROOT/docs/quality-receipt.schema.json}" +SELF="$SCRIPT_DIR/fm-quality-receipt.sh" + +fm_quality_receipt_usage() { + sed -n '2,21{s/^# \{0,1\}//;p;}' "$SELF" +} + +if ! command -v python3 >/dev/null 2>&1; then + printf 'fm-quality-receipt: python3 is required\n' >&2 + exit 2 +fi + +CMD="" +CHECK_HEAD="" +FILE="" +while [ "$#" -gt 0 ]; do + case "$1" in + -h|--help) + fm_quality_receipt_usage + exit 0 + ;; + schema) + [ -z "$CMD" ] || { + printf 'fm-quality-receipt: unexpected extra command %s\n' "$1" >&2 + exit 2 + } + CMD=schema + shift + ;; + validate) + [ -z "$CMD" ] || { + printf 'fm-quality-receipt: unexpected extra command %s\n' "$1" >&2 + exit 2 + } + CMD=validate + shift + ;; + --check-head) + [ "$#" -ge 2 ] || { + printf 'fm-quality-receipt: --check-head requires a git dir\n' >&2 + exit 2 + } + CHECK_HEAD=$2 + shift 2 + ;; + --) + shift + break + ;; + -) + [ -z "$FILE" ] || { + printf 'fm-quality-receipt: unexpected extra argument %s\n' "$1" >&2 + exit 2 + } + FILE=- + shift + ;; + -*) + printf 'fm-quality-receipt: unknown option %s\n' "$1" >&2 + exit 2 + ;; + *) + [ -z "$FILE" ] || { + printf 'fm-quality-receipt: unexpected extra argument %s\n' "$1" >&2 + exit 2 + } + FILE=$1 + shift + ;; + esac +done + +[ -z "$CMD" ] && { + fm_quality_receipt_usage >&2 + exit 2 +} + +if [ "$CMD" = schema ]; then + [ -z "$CHECK_HEAD" ] && [ -z "$FILE" ] || { + printf 'fm-quality-receipt: schema takes no extra arguments\n' >&2 + exit 2 + } + [ -f "$SCHEMA" ] || { + printf 'fm-quality-receipt: schema file missing: %s\n' "$SCHEMA" >&2 + exit 2 + } + cat "$SCHEMA" + exit 0 +fi + +[ "$CMD" = validate ] || { + printf 'fm-quality-receipt: unknown command %s\n' "$CMD" >&2 + exit 2 +} + +[ -f "$SCHEMA" ] || { + printf 'fm-quality-receipt: schema file missing: %s\n' "$SCHEMA" >&2 + exit 2 +} + +[ -n "$FILE" ] || FILE=- + +exec python3 - "$SCHEMA" "$CHECK_HEAD" "$FILE" <<'PY' +from __future__ import annotations + +import json +import re +import subprocess +import sys + + +class SchemaError(Exception): + def __init__(self, path: str, message: str) -> None: + super().__init__(f"{path}: {message}") + self.path = path + self.message = message + + +def is_int(value: object) -> bool: + return isinstance(value, int) and not isinstance(value, bool) + + +def is_number(value: object) -> bool: + return is_int(value) or isinstance(value, float) + + +def unescape(part: str) -> str: + return part.replace("~1", "/").replace("~0", "~") + + +def resolve(ref: str, root: dict) -> dict: + if not ref.startswith("#/"): + raise SchemaError("$", f"unsupported $ref {ref}") + cur: object = root + for part in ref[2:].split("/"): + if not isinstance(cur, dict): + raise SchemaError("$", f"broken $ref {ref}") + key = unescape(part) + if key not in cur: + raise SchemaError("$", f"broken $ref {ref}") + cur = cur[key] + if not isinstance(cur, dict) and not isinstance(cur, bool): + raise SchemaError("$", f"broken $ref {ref}") + return cur # type: ignore[return-value] + + +def matches(instance: object, schema: object, root: dict) -> bool: + try: + validate(instance, schema, root, "$") + except SchemaError: + return False + return True + + +def validate(instance: object, schema: object, root: dict, path: str) -> None: + if schema is True: + return + if schema is False: + raise SchemaError(path, "not allowed") + if not isinstance(schema, dict): + raise SchemaError(path, "invalid schema") + if "$ref" in schema: + validate(instance, resolve(schema["$ref"], root), root, path) + return + if "allOf" in schema: + for sub in schema["allOf"]: + validate(instance, sub, root, path) + if "if" in schema: + if matches(instance, schema["if"], root): + if "then" in schema: + validate(instance, schema["then"], root, path) + elif "else" in schema: + validate(instance, schema["else"], root, path) + if "not" in schema: + if matches(instance, schema["not"], root): + raise SchemaError(path, "matched a forbidden schema") + expected_type = schema.get("type") + if expected_type == "object": + if not isinstance(instance, dict): + raise SchemaError(path, "expected object") + elif expected_type == "array": + if not isinstance(instance, list): + raise SchemaError(path, "expected array") + elif expected_type == "string": + if not isinstance(instance, str): + raise SchemaError(path, "expected string") + elif expected_type == "integer": + if not is_int(instance): + raise SchemaError(path, "expected integer") + elif expected_type == "number": + if not is_number(instance): + raise SchemaError(path, "expected number") + elif expected_type is not None: + raise SchemaError(path, f"unsupported type {expected_type}") + if "const" in schema and instance != schema["const"]: + raise SchemaError(path, f"expected {schema['const']!r}") + if "enum" in schema and instance not in schema["enum"]: + raise SchemaError(path, f"expected one of {schema['enum']!r}") + if "pattern" in schema: + if not isinstance(instance, str) or re.search(schema["pattern"], instance) is None: + raise SchemaError(path, f"expected to match {schema['pattern']}") + if "minLength" in schema: + if not isinstance(instance, str) or len(instance) < schema["minLength"]: + raise SchemaError(path, f"shorter than {schema['minLength']}") + if "minimum" in schema and is_number(instance) and instance < schema["minimum"]: + raise SchemaError(path, f"below minimum {schema['minimum']}") + if isinstance(instance, list): + if "minItems" in schema and len(instance) < schema["minItems"]: + raise SchemaError(path, f"fewer than {schema['minItems']} items") + item_schema = schema.get("items") + if item_schema is not None: + for i, item in enumerate(instance): + validate(item, item_schema, root, f"{path}/{i}") + if isinstance(instance, dict): + if "minProperties" in schema and len(instance) < schema["minProperties"]: + raise SchemaError(path, f"fewer than {schema['minProperties']} properties") + props = schema.get("properties", {}) + required = schema.get("required", []) + for key in required: + if key not in instance: + raise SchemaError(f"{path}/{key}", "required") + additional = schema.get("additionalProperties", True) + for key, value in instance.items(): + child = f"{path}/{key}" + if key in props: + validate(value, props[key], root, child) + elif additional is False: + raise SchemaError(child, "additional property") + elif additional is not True: + validate(value, additional, root, child) + + +def unique_finding_ids(node: object, path: str) -> None: + if isinstance(node, list): + for i, item in enumerate(node): + unique_finding_ids(item, f"{path}/{i}") + return + if not isinstance(node, dict): + return + findings = node.get("findings") + if isinstance(findings, list): + seen: dict[str, int] = {} + for i, item in enumerate(findings): + if not isinstance(item, dict): + continue + finding_id = item.get("id") + if not isinstance(finding_id, str): + continue + if finding_id in seen: + raise SchemaError( + f"{path}/findings/{i}/id", + f"duplicate id {finding_id!r} (also {path}/findings/{seen[finding_id]}/id)", + ) + seen[finding_id] = i + for key, value in node.items(): + unique_finding_ids(value, f"{path}/{key}") + + +def verify_child_shas(receipt: object) -> None: + if not isinstance(receipt, dict): + return + if receipt.get("phase") != "verify": + return + phases = receipt.get("phases") + if not isinstance(phases, list): + return + for i, child in enumerate(phases): + if not isinstance(child, dict): + continue + for field in ("base_sha", "head_sha"): + if child.get(field) != receipt.get(field): + raise SchemaError( + f"$/phases/{i}/{field}", + f"must equal envelope {field}", + ) + + +def git_rev_parse(git_dir: str, *args: str) -> str: + proc = subprocess.run( + ["git", "-C", git_dir, "rev-parse", *args], + check=False, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + if proc.returncode != 0: + detail = proc.stderr.strip() or "git rev-parse failed" + raise SchemaError("$/head_sha", detail) + return proc.stdout.strip() + + +def check_head(receipt: object, git_dir: str) -> None: + if not isinstance(receipt, dict): + raise SchemaError("$", "expected object") + head_sha = receipt.get("head_sha") + if not isinstance(head_sha, str): + raise SchemaError("$/head_sha", "required") + actual = git_rev_parse(git_dir, "HEAD") + try: + resolved = git_rev_parse(git_dir, "--verify", f"{head_sha}^{{commit}}") + except SchemaError as exc: + raise SchemaError("$/head_sha", f"does not resolve in {git_dir}: {exc.message}") from exc + if resolved != actual: + raise SchemaError( + "$/head_sha", + f"{resolved} is not HEAD {actual}", + ) + + +def main() -> int: + schema_path, check_head_dir, source = sys.argv[1], sys.argv[2], sys.argv[3] + try: + with open(schema_path, encoding="utf-8") as handle: + schema = json.load(handle) + except (OSError, json.JSONDecodeError) as exc: + print(f"fm-quality-receipt: cannot read schema: {exc}", file=sys.stderr) + return 2 + try: + if source == "-": + raw = sys.stdin.read() + else: + with open(source, encoding="utf-8") as handle: + raw = handle.read() + receipt = json.loads(raw) + except (OSError, json.JSONDecodeError) as exc: + print(f"fm-quality-receipt: cannot read receipt: {exc}", file=sys.stderr) + return 1 + try: + validate(receipt, schema, schema, "$") + unique_finding_ids(receipt, "$") + verify_child_shas(receipt) + if check_head_dir: + check_head(receipt, check_head_dir) + except SchemaError as exc: + print(f"fm-quality-receipt: {exc}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) +PY diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index 685c133c22f..f07250ad430 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Spawn a direct report: a crewmate in a treehouse or Orca worktree, or a # secondmate in its isolated firstmate home. -# Usage: fm-spawn.sh --mode --yolo [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] +# Usage: fm-spawn.sh --mode --yolo [--quality ] [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] # fm-spawn.sh --scout [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] # fm-spawn.sh [] [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] --secondmate # --mode and --yolo are this task's delivery contract, REQUIRED for every ship @@ -16,6 +16,15 @@ # loud one-line deviation notice is printed and the spawn continues. # no-mistakes-prod-only is a registry policy rather than a task mode and is # refused as a flag value. +# --quality is this task's quality posture, resolved at intake the same way from +# the project's registered "+hardened" annotation (bin/fm-project-mode.sh +# --quality). Unlike --mode it has a safe default, so it is optional on a ship +# spawn and defaults to standard, and it is refused on --scout and --secondmate +# spawns. A ship spawn reads the brief's "Quality contract: quality=" +# line and REFUSES a mismatch exactly as it does for the delivery line. An absent +# line reads as standard rather than as a legacy gap, so it agrees silently with +# --quality standard, while --quality hardened against a brief that never told +# the worker to run the loop is a refusal. # fm-spawn.sh --relaunch [--harness ] [--model ] [--effort ] # --relaunch launches a replacement agent for an EXISTING task into that # task's own recorded endpoint and worktree instead of creating either. It is @@ -191,6 +200,11 @@ # A ship task records the explicit mode/yolo it was passed; a secondmate spawn records # mode=secondmate, yolo=off, home=, and projects=; a scout records neither, and both the # success line and state/.meta omit them. +# A ship task additionally records quality= (the resolved posture) and base_sha= (the +# commit its worktree starts from). base_sha is captured ONCE, at the fresh spawn, and +# a relaunch reads it back rather than recapturing it: the hardened quality loop commits +# each round, so a base recaptured later - or a gate reading HEAD~1 - would narrow its +# view to the newest commits while still reporting success. # Every fresh spawn or relaunch records a new spawn_gen= incarnation token so durable # consumers can distinguish a replacement worker that reuses the same task id. # When the home session's frozen trace-context decision is enabled (see @@ -287,6 +301,8 @@ EFFORT= BACKEND_ARG= MODE= YOLO= +QUALITY= +BASE_SHA= TRACEPARENT_ARG= HARNESS_SET=0 MODEL_SET=0 @@ -294,6 +310,7 @@ EFFORT_SET=0 BACKEND_SET=0 MODE_SET=0 YOLO_SET=0 +QUALITY_SET=0 TRACEPARENT_SET=0 RELAUNCH=0 POS=() @@ -310,6 +327,7 @@ for a in "$@"; do backend) BACKEND_ARG=$a; BACKEND_SET=1 ;; mode) MODE=$a; MODE_SET=1 ;; yolo) YOLO=$a; YOLO_SET=1 ;; + quality) QUALITY=$a; QUALITY_SET=1 ;; traceparent) TRACEPARENT_ARG=$a; TRACEPARENT_SET=1 ;; *) echo "error: internal parser state for --$want_value" >&2; exit 1 ;; esac @@ -332,6 +350,8 @@ for a in "$@"; do --mode=*) MODE=${a#--mode=}; MODE_SET=1 ;; --yolo) want_value=yolo ;; --yolo=*) YOLO=${a#--yolo=}; YOLO_SET=1 ;; + --quality) want_value=quality ;; + --quality=*) QUALITY=${a#--quality=}; QUALITY_SET=1 ;; --traceparent) want_value=traceparent ;; --traceparent=*) TRACEPARENT_ARG=${a#--traceparent=}; TRACEPARENT_SET=1 ;; *) POS+=("$a") ;; @@ -344,6 +364,7 @@ done [ "$BACKEND_SET" -eq 0 ] || [ -n "$BACKEND_ARG" ] || { echo "error: --backend requires a non-empty value" >&2; exit 1; } [ "$MODE_SET" -eq 0 ] || [ -n "$MODE" ] || { echo "error: --mode requires a non-empty value" >&2; exit 1; } [ "$YOLO_SET" -eq 0 ] || [ -n "$YOLO" ] || { echo "error: --yolo requires a non-empty value" >&2; exit 1; } +[ "$QUALITY_SET" -eq 0 ] || [ -n "$QUALITY" ] || { echo "error: --quality requires a non-empty value" >&2; exit 1; } [ "$TRACEPARENT_SET" -eq 0 ] || [ -n "$TRACEPARENT_ARG" ] || { echo "error: --traceparent requires a non-empty value" >&2; exit 1; } # A parent-delivered carrier replaces this home's own resolution, so it is # refused unless it is a secondmate spawn carrying a strictly valid W3C value. @@ -372,6 +393,7 @@ if [ "$RELAUNCH" -eq 1 ]; then [ "$KIND_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded kind; --scout/--secondmate cannot override it" >&2; exit 1; } [ "$MODE_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded delivery mode; --mode cannot override it" >&2; exit 1; } [ "$YOLO_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded yolo posture; --yolo cannot override it" >&2; exit 1; } + [ "$QUALITY_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded quality posture; --quality cannot override it" >&2; exit 1; } else # Delivery contract (AGENTS.md section 7). A ship task's mode and yolo are # firstmate's per-task decision, so they are required and closed-set validated @@ -397,6 +419,14 @@ else on|off) ;; *) echo "error: --yolo must be on or off (got '$YOLO')" >&2; exit 1 ;; esac + # Quality has a safe default, so it is optional; only its value is closed-set + # validated, because a typo must never quietly ship a task firstmate resolved + # as hardened down the standard path. + [ "$QUALITY_SET" -eq 1 ] || QUALITY=standard + case "$QUALITY" in + standard|hardened) ;; + *) echo "error: --quality must be one of standard, hardened (got '$QUALITY')" >&2; exit 1 ;; + esac else [ "$MODE_SET" -eq 0 ] || { echo "error: --mode applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 @@ -406,6 +436,10 @@ else echo "error: --yolo applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 exit 1 } + [ "$QUALITY_SET" -eq 0 ] || { + echo "error: --quality applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 + exit 1 + } fi fi @@ -770,6 +804,8 @@ spawn_abort_cleanup() { echo "kind=$KIND" [ -z "${MODE:-}" ] || echo "mode=$MODE" [ -z "${YOLO:-}" ] || echo "yolo=$YOLO" + [ -z "${QUALITY:-}" ] || echo "quality=$QUALITY" + [ -z "${BASE_SHA:-}" ] || echo "base_sha=$BASE_SHA" echo "tasktmp=${TASK_TMP:-}" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" @@ -887,6 +923,7 @@ if [ "${#POS[@]}" -gt 0 ] && [ "${POS[0]}" != "$idpart" ] && case "$idpart" in * # spanning several modes is two invocations rather than a silent mixed dispatch. [ "$MODE_SET" -eq 0 ] || shared_args+=(--mode "$MODE") [ "$YOLO_SET" -eq 0 ] || shared_args+=(--yolo "$YOLO") + [ "$QUALITY_SET" -eq 0 ] || shared_args+=(--quality "$QUALITY") for pair in "${POS[@]}"; do case "$pair" in *=*) : ;; @@ -1031,6 +1068,15 @@ if [ "$RELAUNCH" -eq 1 ]; then [ -n "$KIND" ] || KIND=ship MODE=$(fm_meta_get "$RELAUNCH_META" mode) YOLO=$(fm_meta_get "$RELAUNCH_META" yolo) + # Read back, never recaptured: the loop's whole measurement is anchored on the + # base this task actually started from (see the header). A ship task recorded + # before quality existed carries no quality= line, and absent means standard - + # the same reading the brief check applies - so it is normalized here rather + # than left empty and refused against its own brief. Its base_sha stays absent + # rather than being invented from a HEAD the worker has already moved. + QUALITY=$(fm_meta_get "$RELAUNCH_META" quality) + [ "$KIND" != ship ] || [ -n "$QUALITY" ] || QUALITY=standard + BASE_SHA=$(fm_meta_get "$RELAUNCH_META" base_sha) RELAUNCH_WT=$(fm_meta_get "$RELAUNCH_META" worktree) [ -n "$RELAUNCH_WT" ] && [ -d "$RELAUNCH_WT" ] || { echo "error: task $ID's recorded worktree '${RELAUNCH_WT:-none}' is missing; refusing to relaunch without the local copy its work lives in" >&2 @@ -1726,6 +1772,17 @@ if [ "$KIND" = ship ]; then echo "error: delivery mismatch for $ID: the brief says mode=$BRIEF_MODE but this spawn passed --mode $MODE; correct the flag or re-scaffold the brief so the worker's instructions and the task record agree" >&2 exit 1 fi + # The same agreement check for the quality posture. A standard brief carries no + # quality line at all, so an absent line IS the standard posture rather than a + # legacy gap: --quality standard agrees with it silently, and --quality hardened + # against a brief that never gave the worker the quality-gate section is the + # drift this refuses. + BRIEF_QUALITY=$(sed -n 's/^Quality contract: quality=\([^ ]*\).*$/\1/p' "$BRIEF" | head -n 1) + [ -n "$BRIEF_QUALITY" ] || BRIEF_QUALITY=standard + if [ "$BRIEF_QUALITY" != "$QUALITY" ]; then + echo "error: quality mismatch for $ID: the brief says quality=$BRIEF_QUALITY but this spawn passed --quality $QUALITY; correct the flag or re-scaffold the brief so the worker's instructions and the task record agree" >&2 + exit 1 + fi # The registry holds the captain's standing posture, so dropping below it is # allowed (a current explicit captain instruction wins) but never silent. An # unregistered project resolves to the same no-mistakes standing default, which @@ -1736,6 +1793,13 @@ if [ "$KIND" = ship ]; then && [ "$(delivery_rigor_rank "$MODE")" -lt "$(delivery_rigor_rank "$STANDING_MODE")" ]; then echo "notice: $ID ships mode=$MODE while the standing posture for $PROJ_NAME is $STANDING_MODE - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 fi + # The same notice for the quality posture, which is a plain two-value token: there + # is no conditional policy to exclude, so a hardened standing posture shipped as a + # standard task is the only downgrade there is. Advisory only, like the mode notice. + STANDING_QUALITY=$("$FM_ROOT/bin/fm-project-mode.sh" --quality "$PROJ_NAME" 2>/dev/null) || STANDING_QUALITY= + if [ "$STANDING_QUALITY" = hardened ] && [ "$QUALITY" = standard ]; then + echo "notice: $ID ships quality=$QUALITY while the standing posture for $PROJ_NAME is $STANDING_QUALITY - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 + fi fi BRIEF_DIR_REAL=$(cd "$(dirname "$BRIEF")" && pwd -P) @@ -2812,6 +2876,16 @@ else fi fi +# The immutable anchor for a hardened task's quality loop, captured once, here, +# while the worktree still sits on the base it was reset to. Every later phase +# measures a diff against THIS commit; a relaunch reads it back from the record +# above rather than recapturing it, because by then the loop has committed rounds +# of its own and a fresh capture would narrow the gate while still reporting +# success. A worktree git cannot read leaves it absent rather than wrong. +if [ "$RELAUNCH" -eq 0 ] && [ "$KIND" = ship ]; then + BASE_SHA=$(git -C "$WT" rev-parse HEAD 2>/dev/null || true) +fi + META_WINDOW=$T [ "$BACKEND" = orca ] && META_WINDOW=$W SPAWN_GEN="s$(date +%s).${BASHPID:-$$}.$RANDOM" @@ -2826,7 +2900,7 @@ fi preserve_relaunch_meta() { awk -F= ' BEGIN { - split("window endpoint_task_id worktree project harness kind mode yolo tasktmp model effort busy_gen spawn_gen traceparent backend herdr_session herdr_workspace_id herdr_tab_id herdr_pane_id zellij_session zellij_tab_id zellij_pane_id orca_worktree_id terminal cmux_workspace_id cmux_surface_id home projects control_relaunch_tx", keys, " ") + split("window endpoint_task_id worktree project harness kind mode yolo quality base_sha tasktmp model effort busy_gen spawn_gen traceparent backend herdr_session herdr_workspace_id herdr_tab_id herdr_pane_id zellij_session zellij_tab_id zellij_pane_id orca_worktree_id terminal cmux_workspace_id cmux_surface_id home projects control_relaunch_tx", keys, " ") for (i in keys) owned[keys[i]] = 1 } !($1 in owned) @@ -2841,6 +2915,8 @@ preserve_relaunch_meta() { echo "kind=$KIND" [ -z "$MODE" ] || echo "mode=$MODE" [ -z "$YOLO" ] || echo "yolo=$YOLO" + [ -z "$QUALITY" ] || echo "quality=$QUALITY" + [ -z "$BASE_SHA" ] || echo "base_sha=$BASE_SHA" echo "tasktmp=$TASK_TMP" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" diff --git a/bin/fm-supervise-daemon.sh b/bin/fm-supervise-daemon.sh index 6403f1e4fc7..7cac1d40ae6 100755 --- a/bin/fm-supervise-daemon.sh +++ b/bin/fm-supervise-daemon.sh @@ -57,10 +57,11 @@ # the marker would forget it). # Crewmates are autonomous, so a delayed stale response does not stall a # healthy crewmate's own progress. -# Buffered escalation delivery also has a max-defer alarm: if a digest stays -# undelivered past FM_MAX_DEFER_SECS, the daemon retries a normal flush and -# writes state/.subsuper-inject-wedged and attempts a configurable active -# alert if submit still cannot be confirmed. +# Buffered escalation delivery also has a max-defer recovery: if a digest +# stays undelivered past FM_MAX_DEFER_SECS, the daemon retries a normal +# flush (including the herdr native-idle unknown override in inject_msg). +# Only if that still cannot confirm a submit does it write +# state/.subsuper-inject-wedged and attempt a configurable active alert. # - Cheap heartbeat catch-all: every HEARTBEAT_SCAN_SECS the daemon greps all # state/*.status for a captain-relevant line the per-wake classifier might # have missed (e.g. a status verb outside CAPTAIN_RE) and escalates it. @@ -117,7 +118,7 @@ # FM_COMPOSER_IDLE_RE optional shared classifier override; see # docs/configuration.md for its safety gates # FM_MAX_DEFER_SECS max seconds a buffered escalation may sit -# undelivered before one normal flush attempt; +# undelivered before one recovery flush; # if that cannot confirm a submit, a wedge # alarm fires (default 300; 0 disables) # FM_WEDGE_ALARM_CHANNEL override config/wedge-alarm with a single @@ -212,8 +213,9 @@ ESCALATE_BATCH_SECS_DEFAULT=90 HEARTBEAT_SCAN_SECS_DEFAULT=300 HOUSEKEEPING_TICK_DEFAULT=15 # Max time a buffered escalation may sit undelivered before the daemon retries -# the normal flush path and, if that cannot confirm a submit, raises a loud wedge -# alarm. The escape hatch makes a guard false-positive visible instead of silent. +# the flush path (including herdr native-idle delivery when the composer is +# unknown) and, if that cannot confirm a submit, raises a loud wedge alarm. +# The escape hatch is a recovery attempt first, then a visible stall. MAX_DEFER_SECS_DEFAULT=300 WEDGE_ALARM_TIMEOUT_SECS_DEFAULT=10 WEDGE_ALARM_LAST_EPOCH=0 @@ -1050,9 +1052,10 @@ housekeeping() { # fi fi - # (1b) max-defer escape. If anything is still buffered past MAX_DEFER_SECS, - # retry the normal delivery path. If that still cannot confirm, raise a loud - # wedge alarm while preserving the buffer. + # (1b) max-defer recovery. If anything is still buffered past MAX_DEFER_SECS, + # retry the flush path. A herdr pane whose composer is unknown can deliver + # here when the backend proves a live idle agent composer. If submit still + # cannot be confirmed, raise a loud wedge alarm while preserving the buffer. max_defer=${FM_MAX_DEFER_SECS:-$MAX_DEFER_SECS_DEFAULT} if afk_active "$state" && [ "$max_defer" -gt 0 ] && [ -s "$state/.subsuper-escalations" ]; then oldest=$(_oldest_line_age "$state/.subsuper-escalations") @@ -1199,10 +1202,13 @@ window_for_task() { # [state] # path it means native agent-state observed a real turn start. # Pending means Enter was swallowed; unknown is treated as undelivered by # this strict daemon path. -# - COMPOSER GUARD before typing: if the cursor line already has real content -# after dim/faint ghost text and borders are ignored (a human's half-typed -# line, or a previous injection's unsent text), defer entirely - injecting -# would merge with the human's text. +# - COMPOSER GUARD before typing: pending text defers so we never merge with +# a human draft. Empty proceeds. Unknown defers unless the backend can +# prove otherwise (fm_backend_composer_unknown_deliverable): on herdr a +# styled re-read that still shows a genuine agent composer, plus native +# agent-state idle, delivers a clipped idle Claude the classifier cannot +# prove empty. A dead shell, a modal, and an unidentified row have no +# container, so they keep deferring. inject_msg() { # [state] local msg=$1 state target backend retries sleep_s verdict composer encoded state="${2:-$(_state_root)}" @@ -1230,19 +1236,25 @@ inject_msg() { # [state] log "inject deferred: supervisor pane busy (agent mid-turn)" return 1 fi - # b) Composer-guard: inject ONLY into a confirmed-empty GENUINE agent - # composer. The shared classifier (fm_backend_composer_state -> - # fm_composer_classify_content, bin/fm-composer-lib.sh) reports 'pending' - # for real unsubmitted text (a human's half-typed line, or a swallowed - # prior injection) and 'unknown' for a bare dead-shell prompt (the agent - # exited to its login shell) or an unreadable pane. Neither is a safe - # target - typing the escalation into a shell could execute it - so defer - # on anything that is not affirmatively 'empty'. A deferred escalation - # stays buffered for the next cycle or the catch-up flush. + # b) Composer-guard: inject into a confirmed-empty GENUINE agent composer. + # The shared classifier reports 'pending' for real unsubmitted text and + # 'unknown' for a dead shell, an unidentified row, or a container it + # cannot judge. Pending ALWAYS defers. Unknown defers unless the backend + # proves the pane is a live agent composer waiting between turns + # (fm_backend_composer_unknown_deliverable, herdr only): a + # false-unknown composer must not stall away-mode for hours, but a dead + # shell, a modal, an unidentified row and a degraded unstyled read all + # still defer. Native-hosted away uses the same captain pane; it does + # not need a different target. composer=$(fm_backend_composer_state "$backend" "$target" 2>/dev/null) if [ "$composer" != empty ]; then - log "inject deferred: supervisor composer not confirmed-empty (state=${composer:-unknown}: pending input, dead-shell prompt, or unreadable pane)" - return 1 + if [ "$composer" = unknown ] \ + && fm_backend_composer_unknown_deliverable "$backend" "$target" 2>/dev/null; then + log "inject: composer unknown but $backend proves a live idle agent composer; delivering" + else + log "inject deferred: supervisor composer not confirmed-empty (state=${composer:-unknown}: pending input, dead-shell prompt, or unreadable pane)" + return 1 + fi fi # (4) Type the digest ONCE, then submit with Enter (retry Enter only, never # retype) via the shared submit primitive. Success = the backend confirms diff --git a/bin/fm-test-run.sh b/bin/fm-test-run.sh index d96f31a4e3b..94f4fd7f19e 100755 --- a/bin/fm-test-run.sh +++ b/bin/fm-test-run.sh @@ -374,6 +374,7 @@ family_for_basename() { fm-kimi-harness.test.sh|fm-muse-harness.test.sh|fm-herdr-lab.test.sh|fm-lint.test.sh|\ fm-lint-workflows.test.sh|\ fm-operational-input.test.sh|fm-pi-primary-types.test.sh|\ + fm-quality-receipt.test.sh|\ fm-send-popup-settle.test.sh|fm-send-settle.test.sh|\ fm-subagent-pretool-check.test.sh|\ fm-supervision-instructions.test.sh|fm-task-delivery.test.sh|\ @@ -611,77 +612,129 @@ list_portable_serial() { # procedure. portable_serial_weight_hints() { cat <<'EOF' -tests/fm-afk-inject-e2e.test.sh 34019 -tests/fm-afk-pi-herdr-return-e2e.test.sh 42 -tests/fm-afk-return.test.sh 1105 -tests/fm-ask-user-authority.test.sh 68 -tests/fm-backend-cmux-smoke.test.sh 29 -tests/fm-backend-cmux.test.sh 2349 -tests/fm-backend-herdr-focus-flash-e2e.test.sh 21 -tests/fm-backend-orca.test.sh 12041 -tests/fm-backend-tmux-smoke.test.sh 314 -tests/fm-backend-zellij-smoke.test.sh 21 -tests/fm-backend-zellij.test.sh 4225 -tests/fm-backend.test.sh 16370 -tests/fm-backlog-handoff.test.sh 2786 -tests/fm-bearings-snapshot.test.sh 60103 -tests/fm-bootstrap.test.sh 21912 -tests/fm-busy-adapter-wiring.test.sh 13962 -tests/fm-busy-state.test.sh 607 -tests/fm-calm-pi-extension.test.sh 203 -tests/fm-claude-stop-autoarm-live-e2e.test.sh 19 -tests/fm-claude-stop-autoarm.test.sh 60521 +tests/fm-afk-inject-e2e.test.sh 35308 +tests/fm-afk-pi-herdr-return-e2e.test.sh 131 +tests/fm-afk-return.test.sh 1455 +tests/fm-agy-harness.test.sh 21984 +tests/fm-ask-user-authority.test.sh 129 +tests/fm-backend-cmux-smoke.test.sh 132 +tests/fm-backend-cmux.test.sh 3575 +tests/fm-backend-herdr-focus-flash-e2e.test.sh 130 +tests/fm-backend-orca.test.sh 15440 +tests/fm-backend-tmux-smoke.test.sh 403 +tests/fm-backend-zellij-smoke.test.sh 126 +tests/fm-backend-zellij.test.sh 20845 +tests/fm-backend.test.sh 16699 +tests/fm-backlog-handoff.test.sh 4444 +tests/fm-bearings-board.test.sh 3684 +tests/fm-bearings-snapshot.test.sh 58886 +tests/fm-bootstrap.test.sh 48787 +tests/fm-busy-adapter-wiring.test.sh 14992 +tests/fm-busy-state.test.sh 668 +tests/fm-calm-pi-extension.test.sh 338 +tests/fm-classify-decision-key.test.sh 1077 +tests/fm-claude-stop-autoarm-live-e2e.test.sh 129 +tests/fm-claude-stop-autoarm.test.sh 7812 +tests/fm-cmux-claude-composer-live-e2e.test.sh 130 tests/fm-codex-continuity-live-e2e.test.sh 19 -tests/fm-daemon.test.sh 15140 -tests/fm-documentation-audiences.test.sh 572 -tests/fm-fleet-snapshot-view.test.sh 5902 -tests/fm-fleet-sync.test.sh 16417 -tests/fm-gate-refuse.test.sh 2839 +tests/fm-composer-matrix-live-e2e.test.sh 125 +tests/fm-control-relaunch.test.sh 30317 +tests/fm-control.test.sh 14240 +tests/fm-cursor-harness.test.sh 948 +tests/fm-cursor-primary-live-e2e.test.sh 125 +tests/fm-cursor-primary.test.sh 49958 +tests/fm-daemon.test.sh 36219 +tests/fm-documentation-audiences.test.sh 725 +tests/fm-dreamer.test.sh 1714 +tests/fm-fleet-snapshot-view.test.sh 6080 +tests/fm-fleet-sync.test.sh 19988 +tests/fm-gate-refuse.test.sh 3719 tests/fm-gitignore-config.test.sh 28 -tests/fm-gotmp.test.sh 308 -tests/fm-grok-continuity-live-e2e.test.sh 19 -tests/fm-grok-stop-live-e2e.test.sh 19 -tests/fm-guard-stale-banner.test.sh 2917 -tests/fm-herdr-session-cleanup.test.sh 4802 -tests/fm-kimi-harness.test.sh 12590 -tests/fm-opencode-primary-live-e2e.test.sh 18 -tests/fm-operational-input.test.sh 184 -tests/fm-pending-reply.test.sh 7328 -tests/fm-pi-primary-live-e2e.test.sh 19 -tests/fm-pi-watch-extension.test.sh 16386 -tests/fm-pr-check-security.test.sh 199573 -tests/fm-procevent.test.sh 42789 -tests/fm-public-followup.test.sh 23365 +tests/fm-gotmp.test.sh 829 +tests/fm-grok-continuity-live-e2e.test.sh 125 +tests/fm-grok-stop-live-e2e.test.sh 130 +tests/fm-guard-stale-banner.test.sh 26033 +tests/fm-harness-liveness-drift-live-e2e.test.sh 131 +tests/fm-herdr-session-cleanup.test.sh 15962 +tests/fm-herdr-submit-confirm-live-e2e.test.sh 125 +tests/fm-herdr-version-floor-live-e2e.test.sh 131 +tests/fm-hindsight.test.sh 627 +tests/fm-inactive-reconcile.test.sh 14922 +tests/fm-kimi-harness.test.sh 27439 +tests/fm-landing-remote.test.sh 14802 +tests/fm-lint-workflows.test.sh 897 +tests/fm-memory-compile.test.sh 3246 +tests/fm-memory-verify.test.sh 6939 +tests/fm-merge-local.test.sh 559 +tests/fm-muse-harness.test.sh 28041 +tests/fm-muse-signals-live-e2e.test.sh 131 +tests/fm-on.test.sh 9266 +tests/fm-opencode-primary-live-e2e.test.sh 129 +tests/fm-operational-input.test.sh 296 +tests/fm-peek-remote.test.sh 813 +tests/fm-pending-reply.test.sh 48528 +tests/fm-pi-primary-live-e2e.test.sh 130 +tests/fm-pi-watch-extension.test.sh 17326 +tests/fm-pr-check-security.test.sh 236787 +tests/fm-procevent-when.test.sh 15490 +tests/fm-procevent.test.sh 53421 +tests/fm-project-origin.test.sh 190 +tests/fm-public-followup.test.sh 42932 tests/fm-quota-array-dispatch-live-e2e.test.sh 19 -tests/fm-secondmate-harness.test.sh 87895 -tests/fm-secondmate-lifecycle-e2e.test.sh 4929 -tests/fm-secondmate-liveness.test.sh 12553 -tests/fm-secondmate-safety.test.sh 24432 -tests/fm-secondmate-sync.test.sh 12289 -tests/fm-send-secondmate-marker-herdr-e2e.test.sh 27 -tests/fm-send-secondmate-marker.test.sh 2136 -tests/fm-session-start.test.sh 37289 -tests/fm-sessionstart-nudge.test.sh 264 -tests/fm-sessionstart-instruction-refresh-live-e2e.test.sh 19 -tests/fm-shared-captain-inheritance.test.sh 3506 -tests/fm-spawn-dispatch-profile.test.sh 41351 -tests/fm-spawn-worktree-settle.test.sh 4598 -tests/fm-startup-memory-budget.test.sh 4260 -tests/fm-subagent-pretool-check.test.sh 901 -tests/fm-supervision-events.test.sh 413 -tests/fm-tangle-guard.test.sh 7230 -tests/fm-teardown-endpoint-safety.test.sh 1073 -tests/fm-teardown.test.sh 23237 -tests/fm-test-isolation-proof.test.sh 326 -tests/fm-turnend-guard.test.sh 5986 -tests/fm-update.test.sh 1894 -tests/fm-vendor-auth-probe.test.sh 42796 -tests/fm-wake-daemon-lifecycle-e2e.test.sh 4284 -tests/fm-wake-drain-unread-status.test.sh 4000 -tests/fm-wake-queue.test.sh 22787 -tests/fm-watch-checkpoint.test.sh 3943 -tests/fm-watch-triage.test.sh 113051 -tests/fm-watcher-lock.test.sh 98342 +tests/fm-remote-backlog-handoff.test.sh 19859 +tests/fm-remote-doctor.test.sh 4424 +tests/fm-remote-entrypoint.test.sh 186 +tests/fm-remote-job-orphan-reap.test.sh 4748 +tests/fm-remote-job.test.sh 51523 +tests/fm-remote-reply.test.sh 36687 +tests/fm-remote-secondmate-lifecycle-e2e.test.sh 176573 +tests/fm-remote-secondmate-parent-binding.test.sh 13520 +tests/fm-remote-secondmate-trace-context.test.sh 38461 +tests/fm-secondmate-harness.test.sh 127421 +tests/fm-secondmate-lifecycle-e2e.test.sh 6320 +tests/fm-secondmate-liveness.test.sh 8786 +tests/fm-secondmate-safety.test.sh 40101 +tests/fm-secondmate-sync.test.sh 13518 +tests/fm-send-remote-delivery.test.sh 4255 +tests/fm-send-resolve-key.test.sh 13926 +tests/fm-send-secondmate-marker-herdr-e2e.test.sh 129 +tests/fm-send-secondmate-marker.test.sh 11230 +tests/fm-session-lock-ancestry.test.sh 1260 +tests/fm-session-start.test.sh 123175 +tests/fm-sessionstart-hook-live-e2e.test.sh 131 +tests/fm-sessionstart-instruction-refresh-live-e2e.test.sh 125 +tests/fm-sessionstart-nudge.test.sh 72748 +tests/fm-shared-captain-inheritance.test.sh 4729 +tests/fm-spawn-dispatch-profile.test.sh 60232 +tests/fm-spawn-pool-base-freshen.test.sh 14993 +tests/fm-spawn-worktree-settle.test.sh 4927 +tests/fm-startup-memory-budget.test.sh 6382 +tests/fm-startup-network.test.sh 50157 +tests/fm-stow-cascade.test.sh 3063 +tests/fm-subagent-pretool-check.test.sh 2836 +tests/fm-supervision-events.test.sh 503 +tests/fm-sync-axi.test.sh 27465 +tests/fm-tangle-guard.test.sh 13291 +tests/fm-task-delivery.test.sh 1929 +tests/fm-teardown-endpoint-safety.test.sh 3457 +tests/fm-teardown.test.sh 89147 +tests/fm-test-fixture-cleanup.test.sh 1644 +tests/fm-test-isolation-proof.test.sh 1648 +tests/fm-tmux-agent-liveness.test.sh 1822 +tests/fm-trace-context-lib.test.sh 242 +tests/fm-trace-context-spawn.test.sh 33761 +tests/fm-turnend-guard.test.sh 18372 +tests/fm-update.test.sh 5014 +tests/fm-vendor-auth-probe.test.sh 43466 +tests/fm-wake-daemon-lifecycle-e2e.test.sh 5508 +tests/fm-wake-drain-open-decisions-cursor.test.sh 42395 +tests/fm-wake-drain-open-decisions.test.sh 4791 +tests/fm-wake-drain-unread-status.test.sh 10088 +tests/fm-wake-queue.test.sh 30520 +tests/fm-watch-arm.test.sh 64624 +tests/fm-watch-checkpoint.test.sh 5305 +tests/fm-watch-triage.test.sh 137843 +tests/fm-watcher-lock.test.sh 82693 EOF } @@ -1203,6 +1256,7 @@ families_for_changed_path() { bin/fm-tmux-lib.sh|bin/fm-marker-lib.sh|bin/fm-operational-input.sh|bin/fm-tasks-axi-lib.sh|\ bin/fm-vendor-auth-probe.sh|\ bin/fm-primary-scope-lib.sh|bin/fm-project-mode.sh|bin/fm-promote.sh|\ + bin/fm-quality-receipt.sh|\ bin/fm-ff-lib.sh|bin/fm-gotmp*|bin/*pretool*) printf '%s\n' pure-contract-unit ;; @@ -1222,7 +1276,8 @@ families_for_changed_path() { printf '%s\n' pure-contract-unit ;; .github/*|.tasks.toml|AGENTS.md|CLAUDE.md|CONTRIBUTING.md|\ - docs/configuration.md|docs/supervision-protocols/*) + docs/configuration.md|docs/supervision-protocols/*|\ + docs/quality-gate.md|docs/quality-receipt.schema.json) printf '%s\n' pure-contract-unit ;; tests/lib.sh|tests/*-helpers.sh) diff --git a/docs/architecture.md b/docs/architecture.md index 520c881893f..afd37d5257a 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -100,10 +100,13 @@ Pane existence, busy checks, composer checks, capture, and verified submit route The retries-exhausted queued-Enter decision is owned by `fm_composer_queued_enter_verdict` in `bin/fm-composer-lib.sh`; tmux and herdr provide only their backend-specific busy signals. Composer classification has one shared owner, `bin/fm-composer-lib.sh`: tmux, herdr, Zellij, Orca, and cmux contribute only a screen capture plus declarative styled, cursor, identity, and row capabilities, while the shared classifier owns every shape and the `empty`/`pending`/`pending-unproven`/`unknown` verdict. `fm-spawn.sh` also routes Kimi launch readiness through that classifier instead of carrying another shape copy. -The daemon injects only into an affirmatively `empty` composer, so every other or future verdict defers; positive container proof is required, and a blank unidentified row or bare dead-shell prompt cannot receive an escalation. +The daemon injects into an affirmatively `empty` composer. +Pending text always defers. +Unknown defers unless the backend proves the pane is a live agent composer waiting between turns; only herdr can, requiring a fresh styled re-read that still shows a genuine composer container plus native agent-state idle, so a clipped idle Claude composer cannot stall away-mode overnight while a dead shell, a modal, an unidentified row, and a degraded unstyled read still cannot receive an escalation. +Native-hosted away auto-discovers the captain pane and does not use a second flush target. The current operator boundary is in [Composer and injection safety](herdr-backend.md#composer-and-injection-safety). Unsupported supervisor backends refuse at daemon startup. -Stalled escalation delivery writes `state/.subsuper-inject-wedged` and attempts a configured backend-independent active alert after `FM_MAX_DEFER_SECS` instead of silently deferring forever. +Stalled escalation delivery retries the flush after `FM_MAX_DEFER_SECS`, including herdr native-idle delivery when the composer is unknown, and only then writes `state/.subsuper-inject-wedged` and attempts a configured backend-independent active alert. On an unmarked return, `bin/fm-afk-return.sh` owns ordered shutdown, durable catch-up evidence, and the fail-closed gate that keeps ordinary work behind every live firstmate-actionable blocker. `fm-send.sh` selects a pre-Enter popup-settle for slash commands and for codex `$...` skill invocations using metadata-routed target `harness=` values, then adds its own `FM_SEND_SETTLE` pause after successful text sends so immediate peeks catch the receiving turn starting; the sub-supervisor uses only the shared submit core and does not pay that post-submit pause. @@ -249,9 +252,10 @@ The `data/secondmates.md` line contract is owned by the [`secondmate-provisionin `no-mistakes` tasks run the full validation pipeline, `direct-PR` tasks open PRs without that pipeline, and `local-only` tasks stay local until firstmate performs an approved fast-forward merge. Each task's mode and `yolo` posture are firstmate's decision at intake and are passed explicitly to `bin/fm-brief.sh`, `bin/fm-spawn.sh`, and `bin/fm-promote.sh`, which refuse a ship task that does not carry them. -A ship brief records its mode as a fixed machine-readable line and the spawn refuses to launch on a different one, so the worker's instructions and the recorded task delivery cannot diverge. -`data/projects.md` records each project's standing posture and optional `+yolo` flag as the captain's default and as context for that decision, including the conditional `no-mistakes-prod-only` policy; a ship spawn that drops below the registered rigor prints a deviation notice and continues. -`bin/fm-project-mode.sh` remains the one registry parser for the mechanical consumers that have no task in hand: fleet sync's `local-only` skip and home seeding's refusal and no-mistakes initialization. +A ship brief records its mode as a fixed machine-readable line, and a hardened ship brief records its quality posture as a sibling line; the spawn refuses to launch on a value that disagrees with either, so the worker's instructions and the recorded task contract cannot diverge. +`data/projects.md` records each project's standing posture, its optional `+hardened` quality posture, and its optional `+yolo` flag as the captain's default and as context for that decision, including the conditional `no-mistakes-prod-only` policy, which `+hardened` may not ride; a ship spawn or a promotion that drops below either registered posture prints a deviation notice and continues. +The project-owned quality-gate contract and the receipt a hardened run must emit are owned by [`quality-gate.md`](quality-gate.md). +`bin/fm-project-mode.sh` remains the one registry parser, both for the standing-posture notices above and for the mechanical consumers that have no task in hand: fleet sync's `local-only` skip and home seeding's refusal and no-mistakes initialization. When a selected delivery path calls for a diff, `bin/fm-review-diff.sh` refreshes the authoritative base and, when task meta records `pr=`, always fetches and compares against `refs/pull//head` by default (recorded `pr_head=` is only an offline fallback) before falling back to the local branch with a warning. Where a no-mistakes pipeline stores evidence in the repo, it publishes that PR-viewable validation evidence to an orphan evidence branch that shares no history with code branches, so it never enters the crew branch or the default branch. This repo uses that setting, and its own `.no-mistakes/` directory remains local state that stays gitignored and is rejected by CI if tracked; [`configuration.md`](configuration.md) owns the setting. diff --git a/docs/configuration.md b/docs/configuration.md index 69f26ccb1c1..fabe3f08404 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -131,7 +131,7 @@ Selecting any other supervisor backend, including `zellij`, `orca`, or `cmux`, r ## Away-mode wedge alarm channels (config/wedge-alarm) -When away-mode injection wedges past `FM_MAX_DEFER_SECS`, the sub-supervisor raises a loud, rate-limited alarm. +When away-mode injection stays undelivered past `FM_MAX_DEFER_SECS`, the sub-supervisor retries the flush, including herdr native-idle delivery when the composer is unknown, and only then raises a loud, rate-limited alarm. Beyond the durable `state/.subsuper-inject-wedged` marker and the tmux status-line flash, it attempts a configured backend-independent active alert that can reach the captain even when every pane and its backend status-line is unreadable. `config/wedge-alarm` (local, gitignored) lists channel directives, one per non-empty, non-comment line; every listed non-`off` channel fires, best-effort. `FM_WEDGE_ALARM_CHANNEL` overrides the file with a single directive. @@ -658,7 +658,7 @@ FM_FLEET_SYNC_PACKED_REFS_LOCK_RETRY_WAIT_SECS=1 # seconds fm-fleet-sync.sh wait FM_FLEET_SYNC_PACKED_REFS_LOCK_AGE_SECS=30 # min mtime age before fm-fleet-sync.sh treats a leftover packed-refs.lock as provably stale FM_BUSY_REGEX= # optional override for rendered delivery guards and Grok's isolated task-state fallback; converted worker state ignores it FM_COMPOSER_IDLE_RE= # optional fleet-wide idle-placeholder regex override (bin/fm-composer-lib.sh); a match alone does not prove emptiness because shape-specific position and ANSI de-emphasis safety gates still apply -FM_COMPOSER_CAPTURE_LINES=20 # fleet-wide bound for tail-capture composer reads; tmux instead supplies its bounded visible pane, while the other adapters use this small window so stale scrollback banners stay out of the candidate set +FM_COMPOSER_CAPTURE_LINES=20 # fleet-wide bound for the tail/limit composer reads used by cmux, orca, zellij, and herdr's unstyled fallback, so stale scrollback banners stay out of the candidate set; tmux and herdr's styled composer read instead supply their own bounded visible viewport FM_COMPOSER_PI_MAX_LINES=8 # fleet-wide: maximum rows admitted between an identity-corroborated separator pair (Pi's, and agy's verified `>` shape); taller or ambiguous candidates stay unknown FM_COMPOSER_GHOST_LUMA_MAX=128 # fleet-wide: max perceived luminance (0.299R+0.587G+0.114B, 0-255) for a TRUECOLOR foreground to count as de-emphasised ghost/placeholder text and be stripped; dim/faint (SGR 2) is stripped regardless. Assumes a dark terminal theme (bin/fm-composer-lib.sh's fm_composer_strip_ghost, used by styled tmux, herdr, and Zellij reads) GROK_HOME= # optional Grok config home for firstmate's global grok turn-end hook; defaults to ~/.grok @@ -671,7 +671,7 @@ FM_SUPERVISOR_BACKEND= # optional supervisor pane backend override; FM_SUPERVISOR_TARGET= # optional supervisor pane target override; tmux target or herdr :, otherwise auto-detected FM_INJECT_SKIP=heartbeat # |-prefixes force-self-handled bypassing classification; empty disables FM_ESCALATE_BATCH_SECS=90 # buffer window for batched escalation digests; 0 = flush immediately -FM_MAX_DEFER_SECS=300 # max buffered escalation age before retry plus wedge alarm; 0 disables +FM_MAX_DEFER_SECS=300 # max buffered escalation age before recovery flush, then wedge alarm; 0 disables FM_WEDGE_ALARM_CHANNEL= # override config/wedge-alarm with one active-alert directive for the wedge alarm; off|auto|osascript|herdr|command:; absent = auto (macOS -> an OS notification) FM_WEDGE_ALARM_EXEC= # notifier seam: route every channel (osascript, herdr, command:) through this command as ` `; "discard" fires nothing; unset in production; the daemon defaults it to "discard" when sourced so no test posts a real notification (docs/wedge-alarm.md) FM_WEDGE_ALARM_TIMEOUT_SECS=10 # maximum seconds for each osascript, herdr, override, or command: notifier before its watchdog terminates it and continues to the next channel; invalid or zero values use 10 diff --git a/docs/documentation-audiences.json b/docs/documentation-audiences.json index 9597279923e..66cd9b89817 100644 --- a/docs/documentation-audiences.json +++ b/docs/documentation-audiences.json @@ -288,6 +288,10 @@ "path": "docs/orca-backend.md", "audience": "operator-current" }, + { + "path": "docs/quality-gate.md", + "audience": "maintainer-architecture" + }, { "path": "docs/remote-secondmates.md", "audience": "operator-current" diff --git a/docs/fm-test-portable-shards.md b/docs/fm-test-portable-shards.md index 5cf681a5019..38554b95e53 100644 --- a/docs/fm-test-portable-shards.md +++ b/docs/fm-test-portable-shards.md @@ -64,28 +64,37 @@ Each shard is still strictly serial in itself, and separate runners mean no two `.github/workflows/ci.yml` derives the same `n` from `strategy.job-total` rather than a literal, so changing the shard count in either file without the other fails the lane loudly instead of leaving part of the required suite unrun. Assignment is longest-processing-time bin packing over per-script duration hints embedded in `bin/fm-test-run.sh`. -The hints came from that run's `fm-test-timing-portable-serial` artifact on 2026-08-02, where the lane ran 69 scripts in 1143762 ms of serial work. +The hints came from the four `Behavior portable serial` shard logs of CI run [32555114248](https://github.com/BohnBawerick/firstmate/actions/runs/32555114248) on 2026-08-22, where the lane holds 123 scripts and about 2529858 ms of serial work. +Three short scripts had no measured value in that run because shard 4 was cancelled before reaching them; they keep their earlier hints. A script with no hint gets the conservative `PORTABLE_SERIAL_DEFAULT_WEIGHT_MS` default. Hints only affect balance: the coverage guard keeps the partition complete and disjoint whatever they say, so a stale hint costs a slower shard rather than lost coverage. +Let the hints go stale for long enough and that slower shard hits the job timeout, which is what happened on run 32555114248: 46 unhinted scripts pushed shard 4 to 904 s against a 445 s shard 2. | Lane | Script count | Estimated duration | |---|---:|---:| -| `portable-serial-1of4` | 15 | 285945 ms (~285.9 s) | -| `portable-serial-2of4` | 18 | 285944 ms (~285.9 s) | -| `portable-serial-3of4` | 17 | 285929 ms (~285.9 s) | -| `portable-serial-4of4` | 19 | 285944 ms (~285.9 s) | -| imbalance | | 16 ms | +| `portable-serial-1of4` | 29 | 632451 ms (~632.5 s) | +| `portable-serial-2of4` | 30 | 632453 ms (~632.5 s) | +| `portable-serial-3of4` | 33 | 632452 ms (~632.5 s) | +| `portable-serial-4of4` | 31 | 632502 ms (~632.5 s) | +| imbalance | | 51 ms | -The single longest script, `tests/fm-pr-check-security.test.sh` at 199573 ms, is the floor for any shard count. +The single longest script, `tests/fm-pr-check-security.test.sh` at 236787 ms, is the floor for any shard count. Refresh the hints by downloading the per-shard timing artifacts from a green CI run, replacing the `portable_serial_weight_hints` table in `bin/fm-test-run.sh` with the measured `path`/`duration_ms` pairs, and updating the table above: ```sh -gh run download -R kunchenguid/firstmate --pattern 'fm-test-timing-portable-serial-*' -D /tmp/fm-serial +gh run download --pattern 'fm-test-timing-portable-serial-*' -D /tmp/fm-serial jq -r '.scripts[] | [.path, .duration_ms] | @tsv' /tmp/fm-serial/*.json | LC_ALL=C sort bin/fm-test-run.sh --check-coverage ``` +A shard that hit its timeout uploads no artifact. +Read its `FM_TEST_END` lines out of the job log instead, and keep the earlier hint for any script the cancelled shard never reached: + +```sh +gh run view --job --log | grep -o 'FM_TEST_END.*' +``` + ## Coverage guard `bin/fm-test-run.sh --check-coverage` verifies that both parallel lanes partition the proven-isolated set. @@ -108,7 +117,7 @@ Portable shards, each portable serial shard, and the Herdr lane upload runner-ge | Lane | Bound | Rationale | |---|---|---| | portable parallel 1/2 | job `timeout-minutes: 10` | The measured shard sums are about three minutes and the timeout is a hang tripwire. | -| portable serial 1-4 | job `timeout-minutes: 15` | Each balanced shard is about five minutes, leaving roughly 3x hang-tripwire margin. | +| portable serial 1-4 | job `timeout-minutes: 25` | Each balanced shard is about 10.5 minutes of script time plus about 2 minutes of setup, leaving roughly 2x hang-tripwire margin. | | Herdr | family-run step `timeout-minutes: 20`; job `timeout-minutes: 75` backstop | Healthy runs finish around 7 minutes, so the step bound is the hang tripwire (cleanup and timing artifacts still upload) while the job cap stays a last-resort backstop. | Timeouts are hang tripwires rather than expected healthy durations. diff --git a/docs/herdr-backend.md b/docs/herdr-backend.md index 03d3669935f..0b40a1fcbea 100644 --- a/docs/herdr-backend.md +++ b/docs/herdr-backend.md @@ -228,8 +228,9 @@ The composer verdict itself is deliberately unchanged: a right-aligned status to The poll density bounds the residual possibility of an extremely fast complete turn; a missed native transition falls through to the composer verdict rather than reporting a false swallow. `pane read --lines N` can return empty output when N is below the viewport height. -The capture owner requests at least 200 lines from Herdr and trims locally to the caller's bound. -This generous floor is required for small composer and peek reads. +Both capture owners therefore request at least 200 lines from Herdr. +The plain scrollback capture then trims locally to the caller's bound, which is what small peek reads need. +The composer's `visible` ANSI capture is deliberately not trimmed: the viewport is already the bound, and a further tail can clip the composer's opening rule (see "Composer and injection safety"). Herdr's native agent state can read idle while a harness waits on its own long foreground tool. The shared crew-state path therefore accepts a native `busy` as evidence of activity but never a native `idle` as evidence that a worker has stopped; the task's own semantic busy state (`bin/fm-busy-lib.sh`) decides that. @@ -238,7 +239,8 @@ A human-blocked permission dialog has no busy banner and still surfaces. ## Composer and injection safety Herdr has no direct cursor-row primitive. -The adapter is a thin capture: it hands a bounded ANSI tail plus Herdr's capability facts to the fleet-wide classifier in `bin/fm-composer-lib.sh`, which owns every shape - bordered boxes, bare agent-glyph rows (including muse's `⟩`, which the adapter's retired local pattern silently omitted), opencode's left bar, and the Pi separator region this adapter pioneered, admitted only when native `agent get` identity is exactly Pi and state is idle, done, or blocked. +The adapter is a thin capture: composer classification reads the live `visible` viewport as a styled ANSI snapshot, then hands that plus Herdr's capability facts to the fleet-wide classifier in `bin/fm-composer-lib.sh`, which owns every shape - bordered boxes, bare agent-glyph rows (including muse's `⟩`, which the adapter's retired local pattern silently omitted), opencode's left bar, and the Pi separator region this adapter pioneered, admitted only when native `agent get` identity is exactly Pi and state is idle, done, or blocked. +`recent` scrollback is the wrong composer source: a short tail of it can drop Claude's opening `─` while keeping the idle `❯` and closing rule, which used to classify unknown for an entire away run. A working Pi, pending middle row, missing identity, incomplete separator pair, or over-tall candidate remains unknown or pending. Identity stays a lazy second read, consulted only when a separator pair could change the verdict. @@ -247,8 +249,13 @@ ANSI capture preserves de-emphasized placeholder style. If the ANSI capture ever fails, the plain fallback declares itself unstyled and the classifier degrades a glyph row carrying trailing text to `unknown` instead of misreading ghost suggestions as typed input, which safely defers injection and eventually raises the wedge alarm. A bare shell prompt is never an empty agent composer. -Away-mode injection proceeds only on an affirmative `empty` result, never on unknown. -This prevents a dead agent pane from receiving and possibly executing an escalation as shell input. +Away-mode injection proceeds on an affirmative `empty` result. +On herdr it also proceeds on `unknown` when `fm_backend_herdr_composer_unknown_deliverable` proves all of: the styled ANSI re-read succeeded, the verdict is still unknown rather than pending, the screen carries a genuine agent composer container, and native agent-state is idle. +A dead login shell is not a container, so `agent_status=done` mapping to idle cannot make it an injection target; a modal, an unidentified row, and a read that fell back to the unstyled capture are all refused for the same reason. +A Pi separator pair counts as a container only when the native identity is exactly Pi and the pair is inside `FM_COMPOSER_PI_MAX_LINES`, because the shared verdict rejects an over-tall or non-Pi pair without reading its content rows, and an unread region may hold the captain's unsent draft. +Both refusals are the same rule: `unknown` may only ever mean "proven container, unjudgeable styling", never "proven container, unread text". +The unstyled fallback matters here for the same reason: it spells real typed text `unknown` instead of `pending`, so accepting it would merge the digest into the captain's half-typed line. +`fm_backend_herdr_composer_read` is the single capture-classify-resolve-identity body behind both `fm_backend_herdr_composer_state` and this override, so the override's styled and container rules cannot drift from the verdict they qualify. The current operational envelope starts with U+2063 and `FIRSTMATE_OP: `. The separate routed-request carrier uses `[fm-from-firstmate]` plus U+2063. diff --git a/docs/quality-gate.md b/docs/quality-gate.md new file mode 100644 index 00000000000..b22f5977e9c --- /dev/null +++ b/docs/quality-gate.md @@ -0,0 +1,150 @@ +# Quality gate contract and receipt + +The project-owned quality-gate file and the receipt its commands print. +This is the moved owner for those two contracts from the parked Stage 0 spec, revised after the Stage 0a pilot on quota-axi. + +This page describes a capability. +Firstmate itself is not a project the bar is applied to, and this repository does not ship a `.quality-gate.yaml`. + +`bin/fm-quality.sh`, the loop controller, is not in this revision. +It consumes the receipt schema defined here, so the schema had to land first. + +## D1. `.quality-gate.yaml` + +Lives at the root of a hardened project, committed. +The name is vendor-neutral on purpose. +Nothing in the file mentions firstmate, so the file and its CI job survive if firstmate is never used on that repo again. + +```yaml +# .quality-gate.yaml +version: 1 + +# The one command CI runs. Exit 0 or 1. Prints one D2 verify receipt on stdout. +verify: "make quality" + +# The ordinary test suite. The loop runs this after every round's edits +# and reverts the round if it goes red. +test: "pnpm test" + +# The two phases, for the pre-flight loop only. CI does not read these. +clean: + command: "pnpm run quality:complexity" + threshold: + crap_max: 15 + +harden: + command: "pnpm run quality:mutation" + threshold: + kill_rate_min: 0.80 + exclude: + - "src/generated/**" + +bounds: + max_iterations: 4 + no_progress_limit: 2 + budget_usd: 8 + budget_minutes: 20 +``` + +The two threshold numbers in that example are the original design values. +They are captain decisions, not part of this revision, and this page does not change them. + +Field rules: + +- `version` is required and is refused if unknown, so a future schema change fails loudly rather than being half-read. +- `verify` is required. Everything else is optional, and a missing phase means that phase reports `not-applicable`. +- Thresholds are per project. There is no universal default, and `fm-quality.sh` must not invent one. +- `bounds` has the defaults listed above, applied when the key is absent. +- Commands run through the platform shell from the repo root, the same convention no-mistakes uses for `commands.*`. + +### `bounds.budget_minutes` + +The parked spec gave `max_iterations`, `no_progress_limit`, and `budget_usd` only. +Stage 0a showed the cost that actually decides affordability is measurement wall clock, and it is spent before any harness call exists to enforce spend against. +A two-hour measurement costs nothing in tokens. + +`budget_minutes` is the missing bound. +The default of 20 is the Stage 0a-affordable window: four vitest-runner measurements on the diffs that were timed. +A project whose engine is slower must set a higher number rather than overrunning a bound that was never written down. +`budget_usd` stays, because the agent-turn spend is a different resource and is still unmeasured. + +## D2. The receipt + +Every phase command prints one JSON object matching [`quality-receipt.schema.json`](quality-receipt.schema.json). +`bin/fm-quality-receipt.sh` is the check, and `bin/fm-quality-receipt.sh schema` reprints that file. + +`schema_version` stays `1`. +This revises unpublished v1 in place. +No production receipts exist yet. +The Stage 0a pilot objects that passed the old checker are not valid against this revision, and that is intentional. + +What breaks, and why: + +- `head_sha` is now required. + Without it a drifted `base_sha` reports `not-applicable` and exits 0, which is the silent miss the design called easiest to get wrong and hardest to notice. + A docs-only change still has a distinct head and base. + A receipt that cannot show both cannot tell those apart from an anchor that drifted onto `HEAD`. +- `survivors[]` is now `findings[]`. + A complexity offender is not a survivor, has no mutant, and is not `killable`. + The required classification enum was forcing a lie. +- Each finding has a stable `id`. + `file` plus `line` collide. + The pilot saw two distinct surviving mutants on one line, and three on another. + "No progress" has to mean the same findings, not the same count. +- `detail` replaces `mutant`, because the field is not mutation-specific. +- `engine` (name and version) and `threshold` (the numbers this outcome was judged against) are required on `clean` and `harden`. + The same code scored 2 to 37 points apart on two runners. + A receipt with no engine identity is not comparable to any other receipt. +- `duration_ms` is a required top-level integer. + Wall clock is the resource the bound is for, so it is not an optional key inside `metrics`. +- `phase: "verify"` is no longer a flat object with mixed findings. + +`outcome` is still the one field a caller reads to decide anything. +`metrics` is still an open map of numbers, because a Python project and a TypeScript project will not report the same keys. + +### Classification + +`clean` findings use `over-threshold`. +`harden` findings use `killable`, `equivalent`, `unreachable`, `unsupported`, or `defect`. +A clean finding classified `killable` is invalid, which is the point of splitting the vocabulary. + +Harden `id` values come from the engine's own stable mutant id when it has one. +Clean `id` values are a per-function identity the phase command controls, typically `file:line:name`. +Ids are unique inside one `findings` array. +They are not unique across the two phases of a verify receipt, because the two engines do not share an id space. + +### Verify emits one object, with `phases[]` + +`verify` prints one envelope object, not two objects, and not a flattened mix. + +One command is still the CI contract. +It exits 0 or 1, prints one JSON document on stdout, and exposes one `outcome` to branch on. +Two raw objects would preserve per-phase mapping and lose that single outcome. +One flat object with mixed `findings[]` preserves the single outcome and loses the mapping. +That is what the pilot had to do, prefixing metric keys by hand and filing complexity rows next to mutants under the same `killable` label. + +The envelope therefore carries `phase: "verify"`, the folded `outcome`, the same `base_sha` and `head_sha`, the wall-clock `duration_ms` of the wrapper, and a `phases[]` array of complete `clean` and `harden` receipts. +It does not carry `findings`, `engine`, or `threshold` of its own: those belong on the child that produced them. + +Phase commands still emit one `clean` or `harden` object each. +Only the verify wrapper builds the envelope. +Each child's `base_sha` and `head_sha` must equal the envelope's, so a wrapper cannot glue receipts from two trees. + +The folded `outcome` is a D4 rule, not a schema constraint. +`blocked` outranks `defect-found`, which outranks `exhausted` and `stuck`, which outrank `pass`. +`not-applicable` is the envelope outcome only when every child is `not-applicable`. + +## Checking a receipt + +```sh +bin/fm-quality-receipt.sh validate +bin/fm-quality-receipt.sh validate --check-head +bin/fm-quality-receipt.sh schema +``` + +`--check-head` resolves `head_sha` in that tree and requires it to be that tree's `HEAD`. +A receipt that stuffed a constant in `head_sha` fails as soon as `HEAD` moves. +The Stage 0a fail-open, a `not-applicable` object with no `head_sha` at all, fails even without that flag. + +`FM_QUALITY_RECEIPT_SCHEMA` may point the validator at a different schema file. +That seam exists so tests can prove the committed schema is the owner rather than a list of constants inside the script. diff --git a/docs/quality-receipt.schema.json b/docs/quality-receipt.schema.json new file mode 100644 index 00000000000..70c0129bd13 --- /dev/null +++ b/docs/quality-receipt.schema.json @@ -0,0 +1,210 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "firstmate quality receipt", + "type": "object", + "required": [ + "schema_version", + "phase", + "outcome", + "base_sha", + "head_sha", + "duration_ms" + ], + "additionalProperties": false, + "properties": { + "schema_version": { "const": 1 }, + "phase": { "enum": ["clean", "harden", "verify"] }, + "outcome": { "$ref": "#/$defs/outcome" }, + "base_sha": { "$ref": "#/$defs/sha" }, + "head_sha": { "$ref": "#/$defs/sha" }, + "duration_ms": { "type": "integer", "minimum": 0 }, + "engine": { "$ref": "#/$defs/engine" }, + "threshold": { "$ref": "#/$defs/threshold" }, + "metrics": { "$ref": "#/$defs/metrics" }, + "findings": { + "type": "array", + "items": { "$ref": "#/$defs/finding" } + }, + "phases": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/phase_result" } + }, + "exclusions": { + "type": "array", + "items": { "type": "string" } + }, + "notes": { "type": "string" } + }, + "allOf": [ + { + "if": { + "properties": { "phase": { "const": "verify" } }, + "required": ["phase"] + }, + "then": { + "required": ["phases"], + "properties": { + "phases": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/phase_result" } + }, + "findings": false, + "engine": false, + "threshold": false + } + }, + "else": { + "$ref": "#/$defs/phase_result" + } + } + ], + "$defs": { + "sha": { + "type": "string", + "pattern": "^[0-9a-f]{7,40}$" + }, + "outcome": { + "enum": [ + "pass", + "blocked", + "not-applicable", + "exhausted", + "stuck", + "defect-found" + ] + }, + "engine": { + "type": "object", + "required": ["name", "version"], + "additionalProperties": false, + "properties": { + "name": { "type": "string", "minLength": 1 }, + "version": { "type": "string", "minLength": 1 } + } + }, + "threshold": { + "type": "object", + "minProperties": 1, + "additionalProperties": { "type": "number" } + }, + "metrics": { + "type": "object", + "additionalProperties": { "type": "number" } + }, + "finding": { + "type": "object", + "required": ["id", "file", "classification"], + "additionalProperties": false, + "properties": { + "id": { "type": "string", "minLength": 1 }, + "file": { "type": "string", "minLength": 1 }, + "line": { "type": "integer", "minimum": 1 }, + "detail": { "type": "string" }, + "classification": { + "enum": [ + "killable", + "equivalent", + "unreachable", + "unsupported", + "defect", + "over-threshold" + ] + }, + "note": { "type": "string" } + } + }, + "phase_result": { + "type": "object", + "required": [ + "schema_version", + "phase", + "outcome", + "base_sha", + "head_sha", + "duration_ms", + "engine", + "threshold", + "findings" + ], + "additionalProperties": false, + "properties": { + "schema_version": { "const": 1 }, + "phase": { "enum": ["clean", "harden"] }, + "outcome": { "$ref": "#/$defs/outcome" }, + "base_sha": { "$ref": "#/$defs/sha" }, + "head_sha": { "$ref": "#/$defs/sha" }, + "duration_ms": { "type": "integer", "minimum": 0 }, + "engine": { "$ref": "#/$defs/engine" }, + "threshold": { "$ref": "#/$defs/threshold" }, + "metrics": { "$ref": "#/$defs/metrics" }, + "findings": { + "type": "array", + "items": { "$ref": "#/$defs/finding" } + }, + "exclusions": { + "type": "array", + "items": { "type": "string" } + }, + "notes": { "type": "string" } + }, + "allOf": [ + { + "if": { + "properties": { "phase": { "const": "clean" } }, + "required": ["phase"] + }, + "then": { + "properties": { + "findings": { + "type": "array", + "items": { + "allOf": [ + { "$ref": "#/$defs/finding" }, + { + "properties": { + "classification": { "const": "over-threshold" } + } + } + ] + } + } + } + } + }, + { + "if": { + "properties": { "phase": { "const": "harden" } }, + "required": ["phase"] + }, + "then": { + "properties": { + "findings": { + "type": "array", + "items": { + "allOf": [ + { "$ref": "#/$defs/finding" }, + { + "properties": { + "classification": { + "enum": [ + "killable", + "equivalent", + "unreachable", + "unsupported", + "defect" + ] + } + } + } + ] + } + } + } + } + } + ] + } + } +} diff --git a/docs/scripts.md b/docs/scripts.md index 11e06955f26..8cf4fde5e63 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -62,7 +62,8 @@ The shared no-mistakes gate refusal for fleet lifecycle entrypoints is summarize | `backends/orca.sh` | Experimental Orca backend adapter owning both worktree and terminal | | `backends/cmux.sh` | Experimental cmux session-provider adapter | | `fm-config-push.sh` | Push declared inherited local material to live local or remote secondmates and send the placement-specific config reread when changed | -| `fm-project-mode.sh` | Resolve a project's registered delivery posture from `data/projects.md` for fleet sync and home seeding | +| `fm-project-mode.sh` | Resolve a project's registered delivery and quality postures from `data/projects.md` | +| `fm-quality-receipt.sh` | Validate a quality-gate receipt against the D2 schema, or print that schema | | `fm-merge-local.sh` | Fast-forward a `local-only` project or Firstmate's own repository local default branch after approval | | `fm-review-diff.sh` | Review a crewmate branch or resolved PR head against the authoritative base | | `fm-marker-lib.sh` | Compatibility entry point for the from-firstmate carrier owned by `fm-operational-input.sh` | diff --git a/docs/verification/runtime-backends.md b/docs/verification/runtime-backends.md index 537685875cb..61918f1ddc3 100644 --- a/docs/verification/runtime-backends.md +++ b/docs/verification/runtime-backends.md @@ -237,7 +237,8 @@ The CLI matrix was checked directly: | Explicit session routing | `herdr ... --session ` | Reached the named session even while another server was running. | | Literal send | `herdr pane send-text --session ` | Left text unsubmitted until Enter. | | Keys | `herdr pane send-keys enter|escape|ctrl+c|up --session ` | Enter and Escape worked; Ctrl-C interrupted foreground work. `up` was checked separately on 0.8.0 (2026-08-13) against `cat -v`: `up` and `Up` emit the real `^[[A`, while `arrow_up`, `ArrowUp`, and `up_arrow` exit nonzero, so only `Up`/`up` is wired into the adapter's key vocabulary. | -| Capture | `herdr pane read --source recent --lines N` | Small N could return empty below viewport height; a 200-line request plus local trim was stable. | +| Capture | `herdr pane read --source recent --lines N` | Small N could return empty below viewport height; a 200-line request plus local trim was stable. This remains the shape of `fm_backend_herdr_capture`, the plain scrollback read used by the rendered busy footer and the peek paths. | +| Composer capture | `herdr pane read --source visible --lines N --format ansi` | The composer read is separate and uses the live viewport. `--lines` is still clamped up to at least 200 so the small-N empty read cannot apply, and the result is NOT locally tailed: `visible` is already viewport-bounded, and tailing it dropped Claude's opening `─` from the idle pair. Verified 2026-08-22 on Herdr 0.8.0 with Claude Code 2.1.239 (see "Composer capture source"). | | Native state | `herdr agent get ` | Working and done transitions were visible on some harnesses; live Claude Code 2.1.236 on Herdr 0.8.0 kept `agent_status=idle` for an entire landed turn, including a multi-second tool call, so submit confirmation falls through to the shared composer verdict. Native `busy` remains positive activity evidence, while native `idle` cannot close a turn and the adapter's semantic lifecycle decides worker state. | | Restart | guarded named-session stop then start | Workspace, tab, pane, and labels persisted; the agent process and registration did not. | | Close | `herdr pane close --session ` | The exact one-pane task tab closed; closing a final tab could remove the workspace. | @@ -245,6 +246,18 @@ The CLI matrix was checked directly: All destructive verification used `bin/fm-herdr-lab.sh` with a non-default `fm-lab-` name and a byte-identical default-session tripwire. No ambient `herdr server stop` command is a supported test operation. +### Composer capture source + +Measured 2026-08-22 against Herdr 0.8.0 and Claude Code 2.1.239 in an isolated `fm-lab-` session, after an overnight away run logged 1555 `composer=unknown` defers on a not-busy captain pane. + +`herdr pane read --source visible --lines 200 --format ansi` succeeded and returned the live viewport. +An idle-between-turns Claude pane carried the full `─` / `❯` / `─` composer pair in that viewport and classified `empty`, so `inject_msg` delivered on the ordinary empty path (rc=0). +On the same pane `fm_backend_herdr_composer_unknown_deliverable` returned 1, because the verdict was `empty` rather than `unknown`. +That is the intended split: the empty path is the normal delivery route, and the unknown override never fires on a pane the classifier can already prove empty. + +The retired composer read was `--source recent` plus a local tail to `FM_COMPOSER_CAPTURE_LINES`. +That combination could keep the idle `❯` and the closing `─` while dropping the matching opening `─`, which is the clipped pair that classified `unknown` all night. + ### Submit confirmation Measured 2026-08-19 against Herdr 0.8.0 and Claude Code 2.1.236 in an isolated `fm-lab-` session. diff --git a/docs/wedge-alarm.md b/docs/wedge-alarm.md index cfee3784b42..0a23a98ce02 100644 --- a/docs/wedge-alarm.md +++ b/docs/wedge-alarm.md @@ -1,7 +1,8 @@ # Away-mode injection wedge alarm The away-mode sub-supervisor (`bin/fm-supervise-daemon.sh`) buffers escalations and injects them into Firstmate's own pane. -When injection cannot confirm a submit past `FM_MAX_DEFER_SECS`, `inject_wedge_alarm` raises a loud, rate-limited alarm so the stall never stays invisible. +When injection cannot confirm a submit past `FM_MAX_DEFER_SECS`, the daemon retries the flush, including herdr native-idle delivery when the composer is unknown. +If that recovery still cannot confirm, `inject_wedge_alarm` raises a loud, rate-limited alarm so the stall never stays invisible. The active alert is pane-independent because a tmux status-line flash has no cross-backend equivalent and cannot reach an unattended captain reliably. The durable marker and tmux flash remain as additional signals. diff --git a/tests/fm-backend-herdr.test.sh b/tests/fm-backend-herdr.test.sh index d82321dce17..c5dd17f09b3 100755 --- a/tests/fm-backend-herdr.test.sh +++ b/tests/fm-backend-herdr.test.sh @@ -3160,6 +3160,155 @@ test_composer_state_claude_unbordered_prompt_is_empty() { pass "fm_backend_herdr_composer_state: a real-claude unbordered '❯' prompt row (no border box in view) reads empty" } +test_composer_state_claude_clipped_closing_rule_is_empty() { + local dir log resp fb out + dir="$TMP_ROOT/composer-claude-clipped-rule"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + # 2026-08-22: a 20-line recent tail can keep idle ❯ and the closing ─ while + # dropping the opening ─. Eight or more ─ make this a Pi separator; without + # the immediate-glyph exception it classified unknown all night. + printf '\xe2\x9d\xaf\n────────────────────────────────\n firstmate on main · Opus 5\n \xe2\x86\x90 for agents\n' > "$resp/1.out" + fb=$(make_herdr_fakebin "$dir") + out=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = empty ] || fail "clipped idle Claude (❯ then closing ─, no opening ─) must read empty, got '$out'" + pass "fm_backend_herdr_composer_state: clipped idle Claude closing rule reads empty" +} + +test_composer_capture_uses_visible_source() { + local dir log resp fb + dir="$TMP_ROOT/composer-visible-source"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + printf '❯\n' > "$resp/1.out" + fb=$(make_herdr_fakebin "$dir") + PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" >/dev/null + assert_contains "$(cat "$log")" $'\x1f''--source'$'\x1f''visible' \ + "composer capture must read the live viewport, not scrollback" + pass "fm_backend_herdr_composer_state: pane read uses --source visible" +} + +# --- the away-mode unknown-composer override -------------------------------- +# +# fm_backend_herdr_composer_unknown_deliverable is the ONLY thing that lets the +# away daemon type into a pane whose composer verdict is `unknown`. These four +# cases pin its whole contract: the one screen it must accept, and the three +# hazards it must refuse. + +test_unknown_deliverable_accepts_styled_container_when_native_idle() { + local dir log resp fb + dir="$TMP_ROOT/unknown-deliverable-ok"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + # A fully dim-rendered idle Claude row: ghost stripping empties the whole + # row, glyph included, so the shared classifier cannot prove it empty and + # answers unknown - but the bare agent-glyph container IS proven. + printf '\033[2m❯ Try "fix the typecheck error"\033[0m\n' > "$resp/1.out" + printf '{"result":{"agent":{"agent_status":"idle"}}}\n' > "$resp/2.out" + fb=$(make_herdr_fakebin "$dir") + PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_unknown_deliverable default:w1:p2' "$ROOT" \ + || fail "a styled unknown over a proven agent composer with native idle must be deliverable" + pass "fm_backend_herdr_composer_unknown_deliverable: accepts a styled unknown over a proven agent composer when native state is idle" +} + +test_unknown_deliverable_refuses_unstyled_fallback_with_typed_text() { + local dir log resp fb out + dir="$TMP_ROOT/unknown-deliverable-unstyled"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + # The captain came back and typed an unsent line. On an older herdr the ANSI + # read fails, the plain fallback declares styled=0, and REAL typed text is + # spelled `unknown` instead of `pending`. Delivering there would merge the + # digest into the captain's line. + echo 1 > "$resp/1.exit" + printf '❯ land the parked workers\n' > "$resp/2.out" + echo 1 > "$resp/3.exit" + printf '{"result":{"agent":{"agent_status":"idle"}}}\n' > "$resp/4.out" + fb=$(make_herdr_fakebin "$dir") + out=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = unknown ] || fail "the unstyled fallback should spell typed text unknown, got '$out'" + if PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_unknown_deliverable default:w1:p2' "$ROOT"; then + fail "a degraded unstyled read must never be deliverable - it hides real typed text as unknown" + fi + pass "fm_backend_herdr_composer_unknown_deliverable: refuses a degraded unstyled read carrying the captain's typed text" +} + +test_unknown_deliverable_refuses_dead_shell_reporting_done() { + local dir log resp fb dir2 resp2 fb2 native + dir="$TMP_ROOT/unknown-deliverable-done-maps-idle"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + # The harness exited to its login shell while herdr still reports + # agent_status=done, which maps to native idle. Native state alone cannot + # tell that from an agent waiting between turns; the missing composer + # container can. + printf '{"result":{"agent":{"agent_status":"done"}}}\n' > "$resp/1.out" + fb=$(make_herdr_fakebin "$dir") + native=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_busy_state default:w1:p2' "$ROOT" ) + [ "$native" = idle ] || fail "agent_status=done must still map to native idle for this hazard to be real, got '$native'" + dir2="$TMP_ROOT/unknown-deliverable-dead-shell"; mkdir -p "$dir2/responses"; resp2="$dir2/responses" + printf 'paiva@box firstmate %% \n' > "$resp2/1.out" + printf '{"result":{"agent":{"agent_status":"done"}}}\n' > "$resp2/2.out" + fb2=$(make_herdr_fakebin "$dir2") + if PATH="$fb2:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp2" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_unknown_deliverable default:w1:p2' "$ROOT"; then + fail "a bare login-shell row must never be deliverable, even while herdr reports done (native idle)" + fi + pass "fm_backend_herdr_composer_unknown_deliverable: refuses a dead login shell whose agent_status=done maps to native idle" +} + +test_unknown_deliverable_refuses_unidentified_row() { + local dir log resp fb + dir="$TMP_ROOT/unknown-deliverable-unidentified"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + # A mid-redraw or borderless dialog pane: no glyph, no box, no left bar, no + # separator pair. A digest plus Enter here would answer whatever is + # highlighted, which away mode must never do. + printf 'Do you want to proceed?\n 1. Yes\n 2. No, tell Claude what to do differently\n' > "$resp/1.out" + printf '{"result":{"agent":{"agent_status":"idle"}}}\n' > "$resp/2.out" + fb=$(make_herdr_fakebin "$dir") + if PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_unknown_deliverable default:w1:p2' "$ROOT"; then + fail "an unidentified row with no container proof must never be deliverable" + fi + pass "fm_backend_herdr_composer_unknown_deliverable: refuses an unidentified row with no container proof" +} + +test_unknown_deliverable_refuses_unread_separator_pair_with_typed_text() { + local dir log resp fb native case_id agent idx + # _fm_composer_pi_verdict answers `unknown` and returns EARLY, before it + # reads a single content row, when the separator pair is over-tall or the + # native identity is not exactly pi. Both screens below hold the captain's + # unsent draft inside that pair, so treating them as a proven container + # would type the digest straight into it - the same "typed text spelled + # unknown" leak the styled-capture rule closes for the degraded read. + for case_id in over-tall non-pi; do + dir="$TMP_ROOT/unknown-deliverable-pair-$case_id"; mkdir -p "$dir/responses" + log="$dir/log"; resp="$dir/responses"; : > "$log" + case "$case_id" in + over-tall) agent=pi ;; + non-pi) agent=claude ;; + esac + { + printf '─────────────────────────────────────────────────────\n' + if [ "$case_id" = over-tall ]; then + for idx in $(seq 1 9); do printf 'land the parked worker row %s\n' "$idx"; done + else + printf 'land the parked workers\n' + fi + printf '─────────────────────────────────────────────────────\n' + } > "$resp/1.out" + printf '{"result":{"agent":{"agent":"%s","agent_status":"idle"}}}\n' "$agent" > "$resp/2.out" + printf '{"result":{"agent":{"agent":"%s","agent_status":"idle"}}}\n' "$agent" > "$resp/3.out" + fb=$(make_herdr_fakebin "$dir") + if PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_unknown_deliverable lab:w1:p2' "$ROOT"; then + fail "a '$case_id' separator pair holding unsent text must never be deliverable" + fi + : > "$log"; rm -f "$resp/.count" + printf '{"result":{"agent":{"agent":"%s","agent_status":"idle"}}}\n' "$agent" > "$resp/1.out" + native=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_busy_state lab:w1:p2' "$ROOT" ) + [ "$native" = idle ] || fail "the '$case_id' hazard needs native idle to be real, got '$native'" + done + pass "fm_backend_herdr_composer_unknown_deliverable: refuses an over-tall or non-Pi separator pair whose text was never read" +} + test_composer_state_claude_unbordered_prompt_is_pending() { local dir log resp fb out dir="$TMP_ROOT/composer-claude-bare-pending"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" @@ -4527,6 +4676,13 @@ test_composer_state_pi_separator_real_text_is_pending test_composer_state_pi_incomplete_separator_below_stale_generic_is_unknown test_composer_state_pi_separator_requires_safe_native_identity test_composer_state_claude_unbordered_prompt_is_empty +test_composer_state_claude_clipped_closing_rule_is_empty +test_composer_capture_uses_visible_source +test_unknown_deliverable_accepts_styled_container_when_native_idle +test_unknown_deliverable_refuses_unstyled_fallback_with_typed_text +test_unknown_deliverable_refuses_dead_shell_reporting_done +test_unknown_deliverable_refuses_unidentified_row +test_unknown_deliverable_refuses_unread_separator_pair_with_typed_text test_composer_state_claude_unbordered_prompt_is_pending test_composer_state_bare_prompt_below_stale_bordered_banner_wins test_composer_state_claude_dim_prompt_suggestion_ghost_is_empty diff --git a/tests/fm-backend.test.sh b/tests/fm-backend.test.sh index ece981b1222..370b879cea2 100755 --- a/tests/fm-backend.test.sh +++ b/tests/fm-backend.test.sh @@ -1083,6 +1083,20 @@ test_spawn_explicit_backend_flag_beats_autodetect_herdr_env() { pass "fm-spawn.sh: explicit --backend tmux wins over an ambient HERDR_ENV=1 auto-detect marker" } +# The away daemon may act on an `unknown` composer verdict only when the +# backend positively proves a live idle agent composer. Every backend without +# a native agent-registration probe must refuse, so `unknown` stays a hard +# defer everywhere it cannot be disproved. +test_composer_unknown_deliverable_default_is_refusal() { + local backend + for backend in tmux orca cmux zellij bogus-backend; do + if fm_backend_composer_unknown_deliverable "$backend" "some:target" 2>/dev/null; then + fail "backend '$backend' must never permit delivery through an unknown composer" + fi + done + pass "fm_backend_composer_unknown_deliverable: every backend without a native agent probe refuses by default" +} + test_spawn_autodetect_nesting_resolves_tmux_silently() { local proj wt data id state config out fb proj="$TMP_ROOT/nest-project"; wt="$TMP_ROOT/nest-wt"; data="$TMP_ROOT/nest-data" @@ -1113,6 +1127,7 @@ test_spawn_autodetect_nesting_resolves_tmux_silently() { pass "fm-spawn.sh: auto-detect resolves nested tmux-in-herdr to tmux and stays silent end to end" } +test_composer_unknown_deliverable_default_is_refusal test_backend_name_precedence test_backend_detect_precedence test_backend_detect_cmux_fallback_bundle_id diff --git a/tests/fm-brief.test.sh b/tests/fm-brief.test.sh index 97368085ce0..9f69d5c65b6 100755 --- a/tests/fm-brief.test.sh +++ b/tests/fm-brief.test.sh @@ -961,6 +961,97 @@ test_task_id_reuse_refused_and_preserves_retained_report() { pass "fm-brief: task id reuse is refused, reports directory contents, gives a way forward, and preserves retained artifacts" } +# --- quality posture -------------------------------------------------------- + +# The load-bearing case for the quality wiring: a ship brief scaffolded WITHOUT +# --quality must be the brief this scaffold produced before --quality existed. +# Proven by executing the real scaffold twice - once with no flag, once with the +# explicit default - and comparing the generated files byte for byte, plus the +# two negative assertions that say what "unchanged" means here: no contract line +# and no quality-gate section reach a standard worker. +test_standard_quality_leaves_the_ship_brief_untouched() { + local home brief_default brief_explicit mode n=0 + home="$TMP_ROOT/quality-standard-home" + mkdir -p "$home/data" + for mode in no-mistakes direct-PR local-only; do + n=$((n + 1)) + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qstd-d$n" some-proj --mode "$mode" >/dev/null 2>&1 \ + || fail "$mode: a ship brief with no --quality should scaffold" + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qexp-d$n" some-proj --mode "$mode" --quality standard >/dev/null 2>&1 \ + || fail "$mode: an explicit --quality standard brief should scaffold" + brief_default="$home/data/brief-qstd-d$n/brief.md" + brief_explicit="$home/data/brief-qexp-d$n/brief.md" + # The task id is the only text that legitimately differs between the two. + sed "s/brief-qexp-d$n/brief-qstd-d$n/g" "$brief_explicit" > "$home/normalized-d$n" + cmp -s "$brief_default" "$home/normalized-d$n" \ + || fail "$mode: --quality standard changed the generated brief (diff: $(diff "$brief_default" "$home/normalized-d$n" | head -5))" + assert_no_grep "Quality contract:" "$brief_default" \ + "$mode: a standard brief recorded a quality contract line" + assert_no_grep "# Quality gate" "$brief_default" \ + "$mode: a standard brief carried the hardened quality-gate section" + grep -qx "Delivery contract: mode=$mode" "$brief_default" \ + || fail "$mode: the delivery contract line did not survive the quality wiring" + done + pass "fm-brief.sh: a standard ship brief is byte-identical with and without --quality, and carries no quality text" +} + +# A hardened brief must tell the worker the four things the loop depends on, and +# must record the machine-readable sibling line bin/fm-spawn.sh checks. Each fact +# is asserted on the generated file, not on the scaffold's source. +test_hardened_brief_records_the_contract_and_the_gate() { + local home brief mode n=0 + home="$TMP_ROOT/quality-hardened-home" + mkdir -p "$home/data" + for mode in no-mistakes direct-PR local-only; do + n=$((n + 1)) + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qhard-e$n" some-proj --mode "$mode" --quality hardened >/dev/null 2>&1 \ + || fail "$mode: a hardened ship brief should scaffold" + brief="$home/data/brief-qhard-e$n/brief.md" + grep -qx "Delivery contract: mode=$mode" "$brief" \ + || fail "$mode: the hardened brief lost its delivery contract line" + grep -qx "Quality contract: quality=hardened" "$brief" \ + || fail "$mode: the hardened brief did not record its machine-readable quality contract line" + assert_grep "# Quality gate" "$brief" "$mode: the hardened brief carried no quality-gate section" + assert_grep 'base_sha=' "$brief" "$mode: the hardened brief did not name the fixed base commit" + # shellcheck disable=SC2016 # A literal backticked phrase from the brief, matched fixed-string. + assert_grep 'never against `HEAD~1`' "$brief" "$mode: the hardened brief did not warn off HEAD~1" + assert_grep 'clean loop first, then the harden loop' "$brief" "$mode: the hardened brief did not order the two loops" + assert_grep 'before you start on that definition of done' "$brief" "$mode: the hardened brief did not put the loops before the definition of done" + assert_grep 'fm-quality.sh' "$brief" "$mode: the hardened brief did not name the script that drives the loop" + assert_grep 'Do not hand-roll either loop' "$brief" "$mode: the hardened brief did not forbid hand-rolling the loop" + assert_grep 'real product defect' "$brief" "$mode: the hardened brief did not say to report a defect rather than test around it" + assert_no_grep "EOF" "$brief" "$mode: the hardened brief leaked a heredoc EOF marker" + assert_grep "{TASK}" "$brief" "$mode: the hardened brief lost the {TASK} placeholder" + done + pass "fm-brief.sh: a hardened ship brief records the quality contract line and the short quality-gate section" +} + +# --quality has a safe default, so it is optional - but a typo must stop the +# scaffold rather than quietly producing a standard brief for a task firstmate +# resolved as hardened, and a scout, dreamer, or charter must refuse it outright +# rather than accepting and discarding it. +test_quality_is_closed_set_and_refused_where_it_does_not_apply() { + local home out status label args expect + home="$TMP_ROOT/quality-refused-home" + mkdir -p "$home/data" + while IFS='|' read -r label args expect; do + [ -n "$label" ] || continue + # shellcheck disable=SC2086 # args is an intentional word-split arg list + out=$(FM_HOME="$home" "$ROOT/bin/fm-brief.sh" $args 2>&1) + status=$? + [ "$status" -ne 0 ] || fail "$label: expected a non-zero exit" + assert_contains "$out" "$expect" "$label: refusal did not explain why" + done <<'ROWS' +unknown quality value|brief-qref-f1 some-proj --mode no-mistakes --quality nope|--quality must be one of standard, hardened +empty quality value|brief-qref-f2 some-proj --mode no-mistakes --quality|requires a value +quality on a scout brief|brief-qref-f3 some-proj --scout --quality hardened|--quality applies only to ship briefs +quality on a dreamer brief|brief-qref-f4 some-proj --dreamer --quality hardened|--quality applies only to ship briefs +quality on a secondmate charter|brief-qref-f5 --secondmate --no-projects --quality hardened|--quality applies only to ship briefs +ROWS + assert_absent "$home/data/brief-qref-f1/brief.md" "a refused quality value still wrote a brief" + pass "fm-brief.sh: --quality is closed-set validated and refused on scout, dreamer, and charter scaffolds" +} + test_script_parses test_no_heredoc_in_command_substitution test_help_includes_entire_header @@ -985,3 +1076,6 @@ test_status_protocol_shows_documented_decision_key_placement test_scout_and_secondmate_load_decision_hold_policy test_scout_and_secondmate_scaffold test_task_id_reuse_refused_and_preserves_retained_report +test_standard_quality_leaves_the_ship_brief_untouched +test_hardened_brief_records_the_contract_and_the_gate +test_quality_is_closed_set_and_refused_where_it_does_not_apply diff --git a/tests/fm-composer-lib.test.sh b/tests/fm-composer-lib.test.sh index 54f7909b4d0..ddbda78f13f 100755 --- a/tests/fm-composer-lib.test.sh +++ b/tests/fm-composer-lib.test.sh @@ -188,6 +188,26 @@ test_matrix_claude_bare_nbsp_row() { pass "matrix: claude's ❯+NBSP row reads empty on every profile in both locales (#1988)" } +test_matrix_claude_clipped_closing_rule_is_empty() { + # 2026-08-22 away-mode wedge: a 20-line herdr `recent` tail can keep Claude's + # idle ❯ and the closing ─ while dropping the matching opening ─. That used + # to fail cursorless selection (unpaired Pi separator below the glyph) and + # classify unknown for the whole away run. The glyph plus an immediately + # following rule is still Claude's idle composer. A separator that is NOT + # the next row stays unknown, so a transcript leftover cannot outrank a + # clipped live Pi pair. + local clipped typed far + clipped=$'❯'"$NBSP"$'\n────────────────────────\n bypass permissions on · 1 shell' + assert_screen "clipped claude idle on herdr" empty "$CAPS_STYLED" "$clipped" '' probe-absent + assert_screen "clipped claude idle on zellij" empty "$CAPS_STYLED_NOID" "$clipped" + assert_screen "clipped claude idle on cmux/orca" empty "$CAPS_PLAIN" "$clipped" + typed=$'❯ land the parked workers\n────────────────────────\n bypass permissions' + assert_screen "clipped claude typed on herdr" pending "$CAPS_STYLED" "$typed" '' probe-absent + far=$'❯\nstatus line\n────────────────────────' + assert_screen "glyph then distant separator on herdr" unknown "$CAPS_STYLED" "$far" '' probe-absent + pass "matrix: a clipped Claude closing rule immediately under idle ❯ reads empty, not unknown" +} + test_matrix_codex_dim_hint_row() { # Real idle codex: bold `›`, reset, then an SGR-2 dim hint. Styled captures # strip the ghost and prove empty; plain captures must defer as unknown - @@ -664,6 +684,7 @@ test_idle_placeholder_is_empty test_idle_placeholder_case_mode_is_explicit test_real_text_is_pending test_matrix_claude_bare_nbsp_row +test_matrix_claude_clipped_closing_rule_is_empty test_matrix_codex_dim_hint_row test_matrix_muse_truecolor_glyph_survives_signal_loss test_matrix_cursor_reverse_video_placeholder_remnant diff --git a/tests/fm-control-relaunch.test.sh b/tests/fm-control-relaunch.test.sh index 9a7b4285bab..6d2f0737377 100755 --- a/tests/fm-control-relaunch.test.sh +++ b/tests/fm-control-relaunch.test.sh @@ -297,6 +297,54 @@ test_relaunch_preserves_durable_task_metadata() { pass "fm-control relaunch: durable task metadata survives replacement launch publication" } +# The quality posture and its base commit are the two records a hardened task's +# quality loop reads back, so a replacement agent must inherit both unchanged. +# base_sha is the one that is easy to get wrong and hard to notice: each loop +# round commits, so a base recaptured at relaunch would quietly move forward and +# narrow every later measurement to the newest work while still reporting success. +# This drives the worktree's HEAD past the recorded base before relaunching, so a +# recapture would be visible rather than coincidentally equal. +test_relaunch_reuses_the_quality_posture_and_base_commit() { + local dir out rc base moved + dir=$(new_case quality-anchor rl40) + add_ship_task "$dir" rl40 claude + base=$(git -C "$dir/wt" rev-parse HEAD) + { + printf 'quality=hardened\n' + printf 'base_sha=%s\n' "$base" + } >> "$dir/home/state/rl40.meta" + # A hardened task's instructions carry the contract lines the spawn re-checks + # on every launch, so the replacement worker cannot be handed instructions that + # disagree with the task's own record. + { + printf '\n# Definition of done\n' + printf 'Delivery contract: mode=no-mistakes\n' + printf 'Quality contract: quality=hardened\n' + } >> "$dir/home/data/rl40/brief.md" + + # A round of the loop lands on the branch, exactly as it would in real work. + printf 'a killed mutant\n' > "$dir/wt/round-1.txt" + git -C "$dir/wt" add round-1.txt + git -C "$dir/wt" -c user.email=t@example.com -c user.name=t commit --quiet -m "quality round 1" + moved=$(git -C "$dir/wt" rev-parse HEAD) + [ "$moved" != "$base" ] || fail "the fixture failed to move HEAD past the recorded base" + + out=$(run_control "$dir" rl40 relaunch --note "continuing the quality loop"); rc=$? + expect_code 0 "$rc" "a hardened task should relaunch"$'\n'"$out" + [ "$(meta_field "$dir" rl40 quality)" = hardened ] \ + || fail "the quality posture must survive relaunch, got '$(meta_field "$dir" rl40 quality)'" + [ "$(meta_field "$dir" rl40 base_sha)" = "$base" ] \ + || fail "base_sha must be read back, not recaptured: got '$(meta_field "$dir" rl40 base_sha)', expected $base" + [ "$(meta_field "$dir" rl40 base_sha)" != "$moved" ] \ + || fail "base_sha was recaptured at relaunch and now points at the loop's own newest commit" + [ "$(grep -c '^quality=' "$dir/home/state/rl40.meta")" = 1 ] \ + || fail "relaunch left more than one quality= line in the task record" + [ "$(grep -c '^base_sha=' "$dir/home/state/rl40.meta")" = 1 ] \ + || fail "relaunch left more than one base_sha= line in the task record" + [ "$(meta_field "$dir" rl40 mode)" = no-mistakes ] || fail "the delivery mode must survive alongside it" + pass "fm-control relaunch: the quality posture survives and the base commit is read back, never recaptured" +} + test_relaunch_serializes_concurrent_durable_metadata_publication() { local dir control_pid link_pid rc i=0 traceparent prepare ready exported release dir=$(new_case metadata-race rl28) @@ -1284,6 +1332,9 @@ test_spawn_relaunch_refuses_contradicting_flags() { out=$(run_spawn "$dir" rl16 --relaunch --scout); rc=$? expect_code 1 "$rc" "--scout should be refused alongside --relaunch" assert_contains "$out" "recorded kind" "the refusal should name the recorded kind rule" + out=$(run_spawn "$dir" rl16 --relaunch --quality hardened); rc=$? + expect_code 1 "$rc" "--quality should be refused alongside --relaunch" + assert_contains "$out" "recorded quality posture" "the refusal should name the recorded quality rule" out=$(run_spawn "$dir" rl16 "$dir/proj" --relaunch); rc=$? expect_code 1 "$rc" "a project positional should be refused alongside --relaunch" assert_contains "$out" "takes the task id only" "the refusal should name the positional rule" @@ -1314,6 +1365,7 @@ test_spawn_relaunch_refuses_a_pane_outside_the_worktree() { test_same_harness_relaunch_keeps_identity_and_reuses_the_endpoint test_relaunch_preserves_durable_task_metadata +test_relaunch_reuses_the_quality_posture_and_base_commit test_relaunch_serializes_concurrent_durable_metadata_publication test_disabled_relaunch_clears_prior_trace_context test_relaunch_appends_the_progress_note_to_the_instructions diff --git a/tests/fm-daemon.test.sh b/tests/fm-daemon.test.sh index f124d9335ac..5a678c0c3f3 100755 --- a/tests/fm-daemon.test.sh +++ b/tests/fm-daemon.test.sh @@ -1968,6 +1968,7 @@ test_inject_msg_defers_on_dead_shell_unknown() { fm_backend_target_exists() { return 0; } pane_is_busy() { return 1; } fm_backend_composer_state() { printf 'unknown'; } + fm_backend_composer_unknown_deliverable() { return 1; } fm_backend_send_text_submit() { fail "send_text_submit must NOT run when the composer is a dead shell (unknown)"; } if FM_SUPERVISOR_BACKEND=herdr FM_SUPERVISOR_TARGET="default:w1:p2" inject_msg "hello" "$state"; then fail "inject_msg should defer (never inject) when the composer reads unknown (dead shell / unreadable)" @@ -1976,6 +1977,50 @@ test_inject_msg_defers_on_dead_shell_unknown() { pass "inject_msg: defers on a dead-shell/unreadable composer (unknown), never typing the escalation into a shell" } +test_inject_msg_herdr_unknown_native_idle_delivers() { + local dir state + dir=$(make_supercase inject-herdr-unknown-idle) + state="$dir/state" + afk_enter "$state" + ( + fm_backend_target_exists() { return 0; } + pane_is_busy() { return 1; } + fm_backend_composer_state() { printf 'unknown'; } + fm_backend_composer_unknown_deliverable() { + [ "$1" = herdr ] || fail "unknown_deliverable backend '$1'" + return 0 + } + fm_backend_send_text_submit() { + [ "$1" = herdr ] || fail "send_text_submit backend '$1'" + printf 'empty' + } + FM_SUPERVISOR_BACKEND=herdr FM_SUPERVISOR_TARGET="default:w1:p2" inject_msg "hello" "$state" \ + || fail "inject_msg should deliver when herdr native state is idle even if the composer reads unknown" + ) || fail "herdr unknown+idle inject_msg subshell failed" + pass "inject_msg: herdr native idle delivers through an unknown composer (clipped idle Claude)" +} + +test_max_defer_herdr_unknown_native_idle_flushes() { + local dir state + dir=$(make_supercase maxdefer-herdr-unknown-idle) + state="$dir/state" + escalate_add "$state" "done: PR https://x/y/pull/9" + echo $(( $(date +%s) - 600 )) > "$state/.subsuper-escalations.since" + afk_enter "$state" + ( + fm_backend_target_exists() { return 0; } + pane_is_busy() { return 1; } + fm_backend_composer_state() { printf 'unknown'; } + fm_backend_composer_unknown_deliverable() { return 0; } + fm_backend_send_text_submit() { printf 'empty'; } + FM_SUPERVISOR_BACKEND=herdr FM_SUPERVISOR_TARGET="default:w1:p2" \ + FM_ESCALATE_BATCH_SECS=99999 FM_MAX_DEFER_SECS=60 housekeeping "$state" + ) || fail "max-defer herdr unknown+idle housekeeping subshell failed" + [ ! -s "$state/.subsuper-escalations" ] || fail "buffer not cleared after herdr native-idle max-defer recovery" + [ ! -e "$state/.subsuper-inject-wedged" ] || fail "wedge alarm left behind after herdr native-idle max-defer recovery" + pass "max-defer: herdr native idle recovers an unknown composer instead of only alarming" +} + test_inject_msg_defers_on_unrecognized_composer_state() { local dir state dir=$(make_supercase inject-future-composer-state) @@ -2095,4 +2140,6 @@ test_inject_msg_herdr_composer_guard_defers test_inject_msg_herdr_pane_gone_defers test_inject_msg_herdr_submits_through_backend_dispatch test_inject_msg_defers_on_dead_shell_unknown +test_inject_msg_herdr_unknown_native_idle_delivers +test_max_defer_herdr_unknown_native_idle_flushes test_inject_msg_defers_on_unrecognized_composer_state diff --git a/tests/fm-quality-receipt.test.sh b/tests/fm-quality-receipt.test.sh new file mode 100755 index 00000000000..a939b989e97 --- /dev/null +++ b/tests/fm-quality-receipt.test.sh @@ -0,0 +1,661 @@ +#!/usr/bin/env bash +# Behavior tests for bin/fm-quality-receipt.sh and the D2 receipt schema. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +TMP_ROOT=$(fm_test_tmproot fm-quality-receipt) +RECEIPT="$ROOT/bin/fm-quality-receipt.sh" + +validate() { + "$RECEIPT" validate "$@" +} + +# Build a JSON receipt from a python literal on stdin. +write_receipt() { + python3 -c 'import json,sys; sys.stdout.write(json.dumps(eval(sys.stdin.read())))' >"$1" \ + || fail "could not write receipt $1" +} + +BASE=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +HEAD=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + +# A valid clean pass. Callers override fields by name. +clean_pass_py() { + cat <