From 16b36f3d1e060cc672d1208ea4b2cd31391a9225 Mon Sep 17 00:00:00 2001 From: PP <121104417+BohnBawerick@users.noreply.github.com> Date: Sat, 22 Aug 2026 03:20:09 +0800 Subject: [PATCH 01/24] feat: plumb the registered hardened quality posture through to the task record Reads a project's registered "+hardened" annotation and carries it to the worker's instructions and the task's durable record, so the quality loop that bin/fm-quality.sh will drive has a posture and a fixed base commit to work from. That script is not part of this change; it is referenced by name only. - bin/fm-project-mode.sh: --quality prints one word, standard or hardened. The two-word stdout its three callers parse is untouched, so it gets its own output path. The bracket grammar is now position-tolerant: a "+"-prefixed token is a flag and never a mode, so "[+hardened local-only]" resolves the mode behind it instead of reading the flag as an unknown mode. Unrecognized flags are still ignored rather than refused. - bin/fm-brief.sh: --quality standard|hardened, defaulting to standard and refused on scout, dreamer, and secondmate scaffolds. A hardened brief records the sibling "Quality contract: quality=hardened" line and one short quality gate section; a standard brief records neither and stays byte-identical to the pre-quality scaffold. - bin/fm-spawn.sh: the brief's quality line must agree with --quality, the same check the delivery line already gets, in both directions. quality= and base_sha= land in the task record; the base commit is captured once at spawn and read back on relaunch, never recaptured, because the loop commits each round and a later capture would narrow the gate while still reporting success. - AGENTS.md: one sentence placing quality resolution at intake. Tests execute the real interfaces. The load-bearing ones prove a project without "+hardened" and a brief scaffolded without --quality behave exactly as before: the two-word stdout is pinned across every annotation form, the two scaffolds are compared byte for byte, and the task record's key set is pinned so only quality= and base_sha= are additive. --- AGENTS.md | 1 + bin/fm-brief.sh | 79 +++++++- bin/fm-project-mode.sh | 72 ++++++-- bin/fm-spawn.sh | 73 +++++++- tests/fm-brief.test.sh | 94 ++++++++++ tests/fm-control-relaunch.test.sh | 52 ++++++ tests/fm-task-delivery.test.sh | 291 +++++++++++++++++++++++++++++- 7 files changed, 633 insertions(+), 29 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 81dc761f8c2..17166ec38ce 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -286,6 +286,7 @@ Resolve every ship task's concrete delivery mode and yolo posture at intake, and A current explicit captain instruction wins; otherwise the project's registry entry is the captain's standing posture, and dropping below its rigor needs a reason you can state. On a `no-mistakes-prod-only` project, classify the task's surface: internal-only tooling, automation, contributor or operator process, and release or submission work ships `direct-PR`, while product-facing, mixed, and uncertain work ships `no-mistakes`; never infer internal-only from file location or project name. An unregistered project or absent registry resolves to `no-mistakes` with yolo off, and the registration gap goes to the captain. +A task's quality posture resolves at intake with the same precedence, a current explicit captain instruction first, then the project's registered posture, then `standard`, with the one-line reason for any deviation recorded in the same backlog note. Record the resulting mode, yolo, and the one-line reason for any deviation in the backlog item note. Treat file or subsystem overlap as a risk signal rather than an automatic reason to wait, and dispatch isolated work immediately with no concurrency cap when each change can be independently implemented and validated and the selected delivery path can reconcile ordinary rebases or conflicts. diff --git a/bin/fm-brief.sh b/bin/fm-brief.sh index 595557523e7..00cd3ab3002 100755 --- a/bin/fm-brief.sh +++ b/bin/fm-brief.sh @@ -6,7 +6,7 @@ # description, acceptance criteria, and context, and may adjust other sections # when the task genuinely deviates (e.g. working an existing external PR instead # of shipping a new one). -# Usage: fm-brief.sh --mode [--herdr-lab] +# Usage: fm-brief.sh --mode [--quality ] [--herdr-lab] # fm-brief.sh --scout [--herdr-lab] # fm-brief.sh --dreamer [--herdr-lab] # fm-brief.sh --secondmate {...|--no-projects} @@ -48,6 +48,18 @@ # "Delivery contract: mode=" line. bin/fm-spawn.sh reads that line and refuses # to launch a ship task whose explicit --mode disagrees, so an adjusted brief and the # recorded task metadata cannot drift apart. +# --quality is the task's quality posture, resolved at intake the same way (AGENTS.md +# section 7) from the project's registered "+hardened" annotation, and it defaults to +# standard so every existing call site scaffolds exactly as before: +# standard the ordinary path: implement, then the mode's definition of done +# hardened a clean loop then a harden loop, both against the base commit fixed at +# spawn, both before validation, driven by bin/fm-quality.sh +# A hardened brief carries the sibling machine-readable line +# "Quality contract: quality=hardened" plus one short quality-gate section; a standard +# brief carries neither, so an absent line means standard and a standard brief stays +# byte-identical to what this scaffold produced before --quality existed. bin/fm-spawn.sh +# checks that line against its own --quality exactly as it checks the mode line. +# --quality is refused on scout and secondmate scaffolds for the same reason --mode is. # Ship briefs begin with a worktree-isolation assertion before the branch step. # --mode is refused on scout and secondmate scaffolds: a scout's deliverable is a # report rather than a merge, and a charter is not a delivery contract. @@ -115,6 +127,8 @@ HERDR_LAB=0 NO_PROJECTS=0 MODE= MODE_SET=0 +QUALITY=standard +QUALITY_SET=0 POS=() want_value= for a in "$@"; do @@ -124,6 +138,7 @@ for a in "$@"; do esac case "$want_value" in mode) MODE=$a; MODE_SET=1 ;; + quality) QUALITY=$a; QUALITY_SET=1 ;; *) echo "error: internal parser state for --$want_value" >&2; exit 1 ;; esac want_value= @@ -137,6 +152,8 @@ for a in "$@"; do --no-projects) NO_PROJECTS=1 ;; --mode) want_value=mode ;; --mode=*) MODE=${a#--mode=}; MODE_SET=1 ;; + --quality) want_value=quality ;; + --quality=*) QUALITY=${a#--quality=}; QUALITY_SET=1 ;; # yolo never reaches the worker: it is firstmate's approval authority, not a # brief input. Refuse it loudly so it is never silently dropped here and then # believed to have been recorded. @@ -164,6 +181,19 @@ elif [ "$MODE_SET" -eq 1 ]; then echo "error: --mode applies only to ship briefs; a scout or dreamer delivers a report and a secondmate charter is not a delivery contract" >&2 exit 1 fi + +# Quality posture. Unlike --mode it has a safe default, so it is optional and only +# its VALUE is closed-set validated; a typo must never quietly scaffold a standard +# brief for a task firstmate resolved as hardened. +if [ "$KIND" = ship ]; then + case "$QUALITY" in + standard|hardened) ;; + *) echo "error: --quality must be one of standard, hardened (got '$QUALITY')" >&2; exit 1 ;; + esac +elif [ "$QUALITY_SET" -eq 1 ]; then + echo "error: --quality applies only to ship briefs; a scout or dreamer delivers a report and a secondmate charter is not a delivery contract" >&2 + exit 1 +fi [ "${#POS[@]}" -ge 1 ] || { echo "error: task id is required" >&2; exit 1; } ID=${POS[0]} @@ -479,17 +509,40 @@ echo "scaffolded: $BRIEF (dreamer; replace {TASK})" exit 0 fi +# The DOD's machine-readable contract header, owned in one place so the three +# mode bodies below cannot drift apart. A standard task emits the delivery line +# alone, exactly as this scaffold did before --quality existed; a hardened task +# adds the sibling quality line that bin/fm-spawn.sh checks against its own +# explicit --quality before launching, the same way it checks the delivery line. +CONTRACT_LINES="Delivery contract: mode=$MODE" +if [ "$QUALITY" = hardened ]; then + CONTRACT_LINES="$CONTRACT_LINES +Quality contract: quality=hardened" +fi + +# The hardened task's extra instructions. Deliberately short: bin/fm-quality.sh +# and its --help own the loop's mechanics, and a second copy here would drift. +IFS= read -r -d '' QUALITY_SECTION <" line that bin/fm-spawn.sh checks against its own -# explicit --mode before launching. +# delivery mode, validated above. Each body opens with $CONTRACT_LINES, built once +# just above. case "$MODE" in direct-PR) SETUP2="" RULE1='1. Never push to the default branch (push only your `fm/'"$ID"'` branch). Never merge a PR.' IFS= read -r -d '' DOD <"\` to start, and \`no-mistakes axi respond\` for each gate. @@ -544,6 +597,14 @@ esac # briefs stay byte-identical to the historical Bash 5 output. DOD=${DOD%$'\n'} +# A standard task's brief body is unchanged by --quality existing: nothing is +# prepended, so it stays byte-identical to the pre-quality scaffold. +if [ "$QUALITY" = hardened ]; then + DOD="$QUALITY_SECTION + +$DOD" +fi + cat > "$BRIEF" < - (added ) -> no-mistakes off (legacy default) # - [] - (added ) -> off # - [ +yolo] - (added ) -> on +# - [ +yolo +hardened] - ... -> on, quality hardened +# +# Bracket grammar: the first token that does not begin with "+" is the mode, and +# every "+" token is position-independent. A "+" this version does not +# recognize is ignored rather than refused, so an older firstmate reading a newer +# registry keeps resolving the posture it does understand. # # Registered modes: # no-mistakes full pipeline -> PR -> configured merge authority (default) @@ -30,12 +36,23 @@ # AGENTS.md section 7 is the single owner of authority exceptions, including # ask-user contract expansion and stronger captain boundaries. # +# +hardened = the registered quality posture. From the captain's side this is the +# fourth option on the same list he picks from when he registers a project, after +# no-mistakes, direct-PR and local-only; mechanically it is a separate token, so a +# hardened project still carries one of those modes too. It is read with --quality +# rather than through the two-word line, which is unchanged. +# Absent means "standard": the ordinary path, with no extra quality loop. +# # --raw prints the registered annotation unmapped, so a caller that must tell a # conditional policy apart from a flat mode sees "no-mistakes-prod-only" itself. # +# --quality prints ONE word instead, "standard" or "hardened". It is a separate +# output path precisely so the two-word stdout contract above stays untouched. +# # An unknown/missing project or unknown mode falls back to "no-mistakes off" and warns -# to stderr, so a typo never silently drops the gate. -# Usage: fm-project-mode.sh [--raw] +# to stderr, so a typo never silently drops the gate; --quality falls back to +# "standard" on the same inputs. +# Usage: fm-project-mode.sh [--raw] [--quality] set -eu SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -44,50 +61,71 @@ FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" REG="$DATA/projects.md" RAW=0 -if [ "${1:-}" = "--raw" ]; then - RAW=1 - shift -fi -NAME=${1:?usage: fm-project-mode.sh [--raw] } +QUALITY_ONLY=0 +while [ "$#" -gt 0 ]; do + case "$1" in + --raw) RAW=1; shift ;; + --quality) QUALITY_ONLY=1; shift ;; + *) break ;; + esac +done +NAME=${1:?usage: fm-project-mode.sh [--raw] [--quality] } + +# One owner of the output shape, so the two-word default and the one-word +# --quality answer cannot drift apart across the fallback paths below. +emit() { # + if [ "$QUALITY_ONLY" -eq 1 ]; then + echo "$3" + else + echo "$1 $2" + fi +} if [ ! -f "$REG" ]; then echo "warn: no registry at $REG; defaulting $NAME to no-mistakes off" >&2 - echo "no-mistakes off" + emit no-mistakes off standard exit 0 fi -# awk emits " " (one line) or nothing if the project is absent. +# awk emits " " (one line) or nothing if the project is +# absent. A "+" token is never a mode, in any position, so the mode is the +# first bracket token that does not begin with "+". parsed=$(awk -v n="$NAME" ' $1=="-" && $2==n { - mode="no-mistakes"; yolo="off"; + mode="no-mistakes"; yolo="off"; quality="standard"; have_mode=0; if ($3 ~ /^\[/) { s=""; for (i=3; i<=NF; i++) { s = s (s==""?"":" ") $i; if ($i ~ /\]$/) break } gsub(/^\[|\]$/, "", s); # strip the surrounding brackets k = split(s, a, " "); - if (a[1] != "" && a[1] != "+yolo") mode = a[1]; - for (j=1; j<=k; j++) if (a[j]=="+yolo") yolo="on"; + for (j=1; j<=k; j++) { + if (a[j]=="+yolo") yolo="on"; + else if (a[j]=="+hardened") quality="hardened"; + else if (a[j] != "" && substr(a[j], 1, 1) != "+" && !have_mode) { mode=a[j]; have_mode=1 } + } } - print mode, yolo; exit + print mode, yolo, quality; exit } ' "$REG") if [ -z "$parsed" ]; then echo "warn: project \"$NAME\" not in registry; defaulting to no-mistakes off" >&2 - echo "no-mistakes off" + emit no-mistakes off standard exit 0 fi -mode=${parsed%% *} -yolo=${parsed##* } +read -r mode yolo quality <&2; mode=no-mistakes; yolo=off ;; esac case "$yolo" in on|off) ;; *) yolo=off ;; esac +case "$quality" in standard|hardened) ;; *) quality=standard ;; esac # A conditional policy is not a task mode. Mechanical callers get its most # rigorous leg; --raw callers get the annotation itself (see the header). if [ "$RAW" -eq 0 ] && [ "$mode" = no-mistakes-prod-only ]; then mode=no-mistakes fi -echo "$mode $yolo" +emit "$mode" "$yolo" "$quality" diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index 685c133c22f..b9e9f805425 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Spawn a direct report: a crewmate in a treehouse or Orca worktree, or a # secondmate in its isolated firstmate home. -# Usage: fm-spawn.sh --mode --yolo [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] +# Usage: fm-spawn.sh --mode --yolo [--quality ] [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] # fm-spawn.sh --scout [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] # fm-spawn.sh [] [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] --secondmate # --mode and --yolo are this task's delivery contract, REQUIRED for every ship @@ -16,6 +16,15 @@ # loud one-line deviation notice is printed and the spawn continues. # no-mistakes-prod-only is a registry policy rather than a task mode and is # refused as a flag value. +# --quality is this task's quality posture, resolved at intake the same way from +# the project's registered "+hardened" annotation (bin/fm-project-mode.sh +# --quality). Unlike --mode it has a safe default, so it is optional on a ship +# spawn and defaults to standard, and it is refused on --scout and --secondmate +# spawns. A ship spawn reads the brief's "Quality contract: quality=" +# line and REFUSES a mismatch exactly as it does for the delivery line. An absent +# line reads as standard rather than as a legacy gap, so it agrees silently with +# --quality standard, while --quality hardened against a brief that never told +# the worker to run the loop is a refusal. # fm-spawn.sh --relaunch [--harness ] [--model ] [--effort ] # --relaunch launches a replacement agent for an EXISTING task into that # task's own recorded endpoint and worktree instead of creating either. It is @@ -191,6 +200,11 @@ # A ship task records the explicit mode/yolo it was passed; a secondmate spawn records # mode=secondmate, yolo=off, home=, and projects=; a scout records neither, and both the # success line and state/.meta omit them. +# A ship task additionally records quality= (the resolved posture) and base_sha= (the +# commit its worktree starts from). base_sha is captured ONCE, at the fresh spawn, and +# a relaunch reads it back rather than recapturing it: the hardened quality loop commits +# each round, so a base recaptured later - or a gate reading HEAD~1 - would narrow its +# view to the newest commits while still reporting success. # Every fresh spawn or relaunch records a new spawn_gen= incarnation token so durable # consumers can distinguish a replacement worker that reuses the same task id. # When the home session's frozen trace-context decision is enabled (see @@ -287,6 +301,8 @@ EFFORT= BACKEND_ARG= MODE= YOLO= +QUALITY= +BASE_SHA= TRACEPARENT_ARG= HARNESS_SET=0 MODEL_SET=0 @@ -294,6 +310,7 @@ EFFORT_SET=0 BACKEND_SET=0 MODE_SET=0 YOLO_SET=0 +QUALITY_SET=0 TRACEPARENT_SET=0 RELAUNCH=0 POS=() @@ -310,6 +327,7 @@ for a in "$@"; do backend) BACKEND_ARG=$a; BACKEND_SET=1 ;; mode) MODE=$a; MODE_SET=1 ;; yolo) YOLO=$a; YOLO_SET=1 ;; + quality) QUALITY=$a; QUALITY_SET=1 ;; traceparent) TRACEPARENT_ARG=$a; TRACEPARENT_SET=1 ;; *) echo "error: internal parser state for --$want_value" >&2; exit 1 ;; esac @@ -332,6 +350,8 @@ for a in "$@"; do --mode=*) MODE=${a#--mode=}; MODE_SET=1 ;; --yolo) want_value=yolo ;; --yolo=*) YOLO=${a#--yolo=}; YOLO_SET=1 ;; + --quality) want_value=quality ;; + --quality=*) QUALITY=${a#--quality=}; QUALITY_SET=1 ;; --traceparent) want_value=traceparent ;; --traceparent=*) TRACEPARENT_ARG=${a#--traceparent=}; TRACEPARENT_SET=1 ;; *) POS+=("$a") ;; @@ -344,6 +364,7 @@ done [ "$BACKEND_SET" -eq 0 ] || [ -n "$BACKEND_ARG" ] || { echo "error: --backend requires a non-empty value" >&2; exit 1; } [ "$MODE_SET" -eq 0 ] || [ -n "$MODE" ] || { echo "error: --mode requires a non-empty value" >&2; exit 1; } [ "$YOLO_SET" -eq 0 ] || [ -n "$YOLO" ] || { echo "error: --yolo requires a non-empty value" >&2; exit 1; } +[ "$QUALITY_SET" -eq 0 ] || [ -n "$QUALITY" ] || { echo "error: --quality requires a non-empty value" >&2; exit 1; } [ "$TRACEPARENT_SET" -eq 0 ] || [ -n "$TRACEPARENT_ARG" ] || { echo "error: --traceparent requires a non-empty value" >&2; exit 1; } # A parent-delivered carrier replaces this home's own resolution, so it is # refused unless it is a secondmate spawn carrying a strictly valid W3C value. @@ -372,6 +393,7 @@ if [ "$RELAUNCH" -eq 1 ]; then [ "$KIND_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded kind; --scout/--secondmate cannot override it" >&2; exit 1; } [ "$MODE_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded delivery mode; --mode cannot override it" >&2; exit 1; } [ "$YOLO_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded yolo posture; --yolo cannot override it" >&2; exit 1; } + [ "$QUALITY_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded quality posture; --quality cannot override it" >&2; exit 1; } else # Delivery contract (AGENTS.md section 7). A ship task's mode and yolo are # firstmate's per-task decision, so they are required and closed-set validated @@ -397,6 +419,14 @@ else on|off) ;; *) echo "error: --yolo must be on or off (got '$YOLO')" >&2; exit 1 ;; esac + # Quality has a safe default, so it is optional; only its value is closed-set + # validated, because a typo must never quietly ship a task firstmate resolved + # as hardened down the standard path. + [ "$QUALITY_SET" -eq 1 ] || QUALITY=standard + case "$QUALITY" in + standard|hardened) ;; + *) echo "error: --quality must be one of standard, hardened (got '$QUALITY')" >&2; exit 1 ;; + esac else [ "$MODE_SET" -eq 0 ] || { echo "error: --mode applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 @@ -406,6 +436,10 @@ else echo "error: --yolo applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 exit 1 } + [ "$QUALITY_SET" -eq 0 ] || { + echo "error: --quality applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 + exit 1 + } fi fi @@ -770,6 +804,8 @@ spawn_abort_cleanup() { echo "kind=$KIND" [ -z "${MODE:-}" ] || echo "mode=$MODE" [ -z "${YOLO:-}" ] || echo "yolo=$YOLO" + [ -z "${QUALITY:-}" ] || echo "quality=$QUALITY" + [ -z "${BASE_SHA:-}" ] || echo "base_sha=$BASE_SHA" echo "tasktmp=${TASK_TMP:-}" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" @@ -887,6 +923,7 @@ if [ "${#POS[@]}" -gt 0 ] && [ "${POS[0]}" != "$idpart" ] && case "$idpart" in * # spanning several modes is two invocations rather than a silent mixed dispatch. [ "$MODE_SET" -eq 0 ] || shared_args+=(--mode "$MODE") [ "$YOLO_SET" -eq 0 ] || shared_args+=(--yolo "$YOLO") + [ "$QUALITY_SET" -eq 0 ] || shared_args+=(--quality "$QUALITY") for pair in "${POS[@]}"; do case "$pair" in *=*) : ;; @@ -1031,6 +1068,15 @@ if [ "$RELAUNCH" -eq 1 ]; then [ -n "$KIND" ] || KIND=ship MODE=$(fm_meta_get "$RELAUNCH_META" mode) YOLO=$(fm_meta_get "$RELAUNCH_META" yolo) + # Read back, never recaptured: the loop's whole measurement is anchored on the + # base this task actually started from (see the header). A ship task recorded + # before quality existed carries no quality= line, and absent means standard - + # the same reading the brief check applies - so it is normalized here rather + # than left empty and refused against its own brief. Its base_sha stays absent + # rather than being invented from a HEAD the worker has already moved. + QUALITY=$(fm_meta_get "$RELAUNCH_META" quality) + [ "$KIND" != ship ] || [ -n "$QUALITY" ] || QUALITY=standard + BASE_SHA=$(fm_meta_get "$RELAUNCH_META" base_sha) RELAUNCH_WT=$(fm_meta_get "$RELAUNCH_META" worktree) [ -n "$RELAUNCH_WT" ] && [ -d "$RELAUNCH_WT" ] || { echo "error: task $ID's recorded worktree '${RELAUNCH_WT:-none}' is missing; refusing to relaunch without the local copy its work lives in" >&2 @@ -1726,6 +1772,17 @@ if [ "$KIND" = ship ]; then echo "error: delivery mismatch for $ID: the brief says mode=$BRIEF_MODE but this spawn passed --mode $MODE; correct the flag or re-scaffold the brief so the worker's instructions and the task record agree" >&2 exit 1 fi + # The same agreement check for the quality posture. A standard brief carries no + # quality line at all, so an absent line IS the standard posture rather than a + # legacy gap: --quality standard agrees with it silently, and --quality hardened + # against a brief that never gave the worker the quality-gate section is the + # drift this refuses. + BRIEF_QUALITY=$(sed -n 's/^Quality contract: quality=\([^ ]*\).*$/\1/p' "$BRIEF" | head -n 1) + [ -n "$BRIEF_QUALITY" ] || BRIEF_QUALITY=standard + if [ "$BRIEF_QUALITY" != "$QUALITY" ]; then + echo "error: quality mismatch for $ID: the brief says quality=$BRIEF_QUALITY but this spawn passed --quality $QUALITY; correct the flag or re-scaffold the brief so the worker's instructions and the task record agree" >&2 + exit 1 + fi # The registry holds the captain's standing posture, so dropping below it is # allowed (a current explicit captain instruction wins) but never silent. An # unregistered project resolves to the same no-mistakes standing default, which @@ -2812,6 +2869,16 @@ else fi fi +# The immutable anchor for a hardened task's quality loop, captured once, here, +# while the worktree still sits on the base it was reset to. Every later phase +# measures a diff against THIS commit; a relaunch reads it back from the record +# above rather than recapturing it, because by then the loop has committed rounds +# of its own and a fresh capture would narrow the gate while still reporting +# success. A worktree git cannot read leaves it absent rather than wrong. +if [ "$RELAUNCH" -eq 0 ] && [ "$KIND" = ship ]; then + BASE_SHA=$(git -C "$WT" rev-parse HEAD 2>/dev/null || true) +fi + META_WINDOW=$T [ "$BACKEND" = orca ] && META_WINDOW=$W SPAWN_GEN="s$(date +%s).${BASHPID:-$$}.$RANDOM" @@ -2826,7 +2893,7 @@ fi preserve_relaunch_meta() { awk -F= ' BEGIN { - split("window endpoint_task_id worktree project harness kind mode yolo tasktmp model effort busy_gen spawn_gen traceparent backend herdr_session herdr_workspace_id herdr_tab_id herdr_pane_id zellij_session zellij_tab_id zellij_pane_id orca_worktree_id terminal cmux_workspace_id cmux_surface_id home projects control_relaunch_tx", keys, " ") + split("window endpoint_task_id worktree project harness kind mode yolo quality base_sha tasktmp model effort busy_gen spawn_gen traceparent backend herdr_session herdr_workspace_id herdr_tab_id herdr_pane_id zellij_session zellij_tab_id zellij_pane_id orca_worktree_id terminal cmux_workspace_id cmux_surface_id home projects control_relaunch_tx", keys, " ") for (i in keys) owned[keys[i]] = 1 } !($1 in owned) @@ -2841,6 +2908,8 @@ preserve_relaunch_meta() { echo "kind=$KIND" [ -z "$MODE" ] || echo "mode=$MODE" [ -z "$YOLO" ] || echo "yolo=$YOLO" + [ -z "$QUALITY" ] || echo "quality=$QUALITY" + [ -z "$BASE_SHA" ] || echo "base_sha=$BASE_SHA" echo "tasktmp=$TASK_TMP" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" diff --git a/tests/fm-brief.test.sh b/tests/fm-brief.test.sh index 97368085ce0..9f69d5c65b6 100755 --- a/tests/fm-brief.test.sh +++ b/tests/fm-brief.test.sh @@ -961,6 +961,97 @@ test_task_id_reuse_refused_and_preserves_retained_report() { pass "fm-brief: task id reuse is refused, reports directory contents, gives a way forward, and preserves retained artifacts" } +# --- quality posture -------------------------------------------------------- + +# The load-bearing case for the quality wiring: a ship brief scaffolded WITHOUT +# --quality must be the brief this scaffold produced before --quality existed. +# Proven by executing the real scaffold twice - once with no flag, once with the +# explicit default - and comparing the generated files byte for byte, plus the +# two negative assertions that say what "unchanged" means here: no contract line +# and no quality-gate section reach a standard worker. +test_standard_quality_leaves_the_ship_brief_untouched() { + local home brief_default brief_explicit mode n=0 + home="$TMP_ROOT/quality-standard-home" + mkdir -p "$home/data" + for mode in no-mistakes direct-PR local-only; do + n=$((n + 1)) + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qstd-d$n" some-proj --mode "$mode" >/dev/null 2>&1 \ + || fail "$mode: a ship brief with no --quality should scaffold" + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qexp-d$n" some-proj --mode "$mode" --quality standard >/dev/null 2>&1 \ + || fail "$mode: an explicit --quality standard brief should scaffold" + brief_default="$home/data/brief-qstd-d$n/brief.md" + brief_explicit="$home/data/brief-qexp-d$n/brief.md" + # The task id is the only text that legitimately differs between the two. + sed "s/brief-qexp-d$n/brief-qstd-d$n/g" "$brief_explicit" > "$home/normalized-d$n" + cmp -s "$brief_default" "$home/normalized-d$n" \ + || fail "$mode: --quality standard changed the generated brief (diff: $(diff "$brief_default" "$home/normalized-d$n" | head -5))" + assert_no_grep "Quality contract:" "$brief_default" \ + "$mode: a standard brief recorded a quality contract line" + assert_no_grep "# Quality gate" "$brief_default" \ + "$mode: a standard brief carried the hardened quality-gate section" + grep -qx "Delivery contract: mode=$mode" "$brief_default" \ + || fail "$mode: the delivery contract line did not survive the quality wiring" + done + pass "fm-brief.sh: a standard ship brief is byte-identical with and without --quality, and carries no quality text" +} + +# A hardened brief must tell the worker the four things the loop depends on, and +# must record the machine-readable sibling line bin/fm-spawn.sh checks. Each fact +# is asserted on the generated file, not on the scaffold's source. +test_hardened_brief_records_the_contract_and_the_gate() { + local home brief mode n=0 + home="$TMP_ROOT/quality-hardened-home" + mkdir -p "$home/data" + for mode in no-mistakes direct-PR local-only; do + n=$((n + 1)) + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qhard-e$n" some-proj --mode "$mode" --quality hardened >/dev/null 2>&1 \ + || fail "$mode: a hardened ship brief should scaffold" + brief="$home/data/brief-qhard-e$n/brief.md" + grep -qx "Delivery contract: mode=$mode" "$brief" \ + || fail "$mode: the hardened brief lost its delivery contract line" + grep -qx "Quality contract: quality=hardened" "$brief" \ + || fail "$mode: the hardened brief did not record its machine-readable quality contract line" + assert_grep "# Quality gate" "$brief" "$mode: the hardened brief carried no quality-gate section" + assert_grep 'base_sha=' "$brief" "$mode: the hardened brief did not name the fixed base commit" + # shellcheck disable=SC2016 # A literal backticked phrase from the brief, matched fixed-string. + assert_grep 'never against `HEAD~1`' "$brief" "$mode: the hardened brief did not warn off HEAD~1" + assert_grep 'clean loop first, then the harden loop' "$brief" "$mode: the hardened brief did not order the two loops" + assert_grep 'before you start on that definition of done' "$brief" "$mode: the hardened brief did not put the loops before the definition of done" + assert_grep 'fm-quality.sh' "$brief" "$mode: the hardened brief did not name the script that drives the loop" + assert_grep 'Do not hand-roll either loop' "$brief" "$mode: the hardened brief did not forbid hand-rolling the loop" + assert_grep 'real product defect' "$brief" "$mode: the hardened brief did not say to report a defect rather than test around it" + assert_no_grep "EOF" "$brief" "$mode: the hardened brief leaked a heredoc EOF marker" + assert_grep "{TASK}" "$brief" "$mode: the hardened brief lost the {TASK} placeholder" + done + pass "fm-brief.sh: a hardened ship brief records the quality contract line and the short quality-gate section" +} + +# --quality has a safe default, so it is optional - but a typo must stop the +# scaffold rather than quietly producing a standard brief for a task firstmate +# resolved as hardened, and a scout, dreamer, or charter must refuse it outright +# rather than accepting and discarding it. +test_quality_is_closed_set_and_refused_where_it_does_not_apply() { + local home out status label args expect + home="$TMP_ROOT/quality-refused-home" + mkdir -p "$home/data" + while IFS='|' read -r label args expect; do + [ -n "$label" ] || continue + # shellcheck disable=SC2086 # args is an intentional word-split arg list + out=$(FM_HOME="$home" "$ROOT/bin/fm-brief.sh" $args 2>&1) + status=$? + [ "$status" -ne 0 ] || fail "$label: expected a non-zero exit" + assert_contains "$out" "$expect" "$label: refusal did not explain why" + done <<'ROWS' +unknown quality value|brief-qref-f1 some-proj --mode no-mistakes --quality nope|--quality must be one of standard, hardened +empty quality value|brief-qref-f2 some-proj --mode no-mistakes --quality|requires a value +quality on a scout brief|brief-qref-f3 some-proj --scout --quality hardened|--quality applies only to ship briefs +quality on a dreamer brief|brief-qref-f4 some-proj --dreamer --quality hardened|--quality applies only to ship briefs +quality on a secondmate charter|brief-qref-f5 --secondmate --no-projects --quality hardened|--quality applies only to ship briefs +ROWS + assert_absent "$home/data/brief-qref-f1/brief.md" "a refused quality value still wrote a brief" + pass "fm-brief.sh: --quality is closed-set validated and refused on scout, dreamer, and charter scaffolds" +} + test_script_parses test_no_heredoc_in_command_substitution test_help_includes_entire_header @@ -985,3 +1076,6 @@ test_status_protocol_shows_documented_decision_key_placement test_scout_and_secondmate_load_decision_hold_policy test_scout_and_secondmate_scaffold test_task_id_reuse_refused_and_preserves_retained_report +test_standard_quality_leaves_the_ship_brief_untouched +test_hardened_brief_records_the_contract_and_the_gate +test_quality_is_closed_set_and_refused_where_it_does_not_apply diff --git a/tests/fm-control-relaunch.test.sh b/tests/fm-control-relaunch.test.sh index 9a7b4285bab..6d2f0737377 100755 --- a/tests/fm-control-relaunch.test.sh +++ b/tests/fm-control-relaunch.test.sh @@ -297,6 +297,54 @@ test_relaunch_preserves_durable_task_metadata() { pass "fm-control relaunch: durable task metadata survives replacement launch publication" } +# The quality posture and its base commit are the two records a hardened task's +# quality loop reads back, so a replacement agent must inherit both unchanged. +# base_sha is the one that is easy to get wrong and hard to notice: each loop +# round commits, so a base recaptured at relaunch would quietly move forward and +# narrow every later measurement to the newest work while still reporting success. +# This drives the worktree's HEAD past the recorded base before relaunching, so a +# recapture would be visible rather than coincidentally equal. +test_relaunch_reuses_the_quality_posture_and_base_commit() { + local dir out rc base moved + dir=$(new_case quality-anchor rl40) + add_ship_task "$dir" rl40 claude + base=$(git -C "$dir/wt" rev-parse HEAD) + { + printf 'quality=hardened\n' + printf 'base_sha=%s\n' "$base" + } >> "$dir/home/state/rl40.meta" + # A hardened task's instructions carry the contract lines the spawn re-checks + # on every launch, so the replacement worker cannot be handed instructions that + # disagree with the task's own record. + { + printf '\n# Definition of done\n' + printf 'Delivery contract: mode=no-mistakes\n' + printf 'Quality contract: quality=hardened\n' + } >> "$dir/home/data/rl40/brief.md" + + # A round of the loop lands on the branch, exactly as it would in real work. + printf 'a killed mutant\n' > "$dir/wt/round-1.txt" + git -C "$dir/wt" add round-1.txt + git -C "$dir/wt" -c user.email=t@example.com -c user.name=t commit --quiet -m "quality round 1" + moved=$(git -C "$dir/wt" rev-parse HEAD) + [ "$moved" != "$base" ] || fail "the fixture failed to move HEAD past the recorded base" + + out=$(run_control "$dir" rl40 relaunch --note "continuing the quality loop"); rc=$? + expect_code 0 "$rc" "a hardened task should relaunch"$'\n'"$out" + [ "$(meta_field "$dir" rl40 quality)" = hardened ] \ + || fail "the quality posture must survive relaunch, got '$(meta_field "$dir" rl40 quality)'" + [ "$(meta_field "$dir" rl40 base_sha)" = "$base" ] \ + || fail "base_sha must be read back, not recaptured: got '$(meta_field "$dir" rl40 base_sha)', expected $base" + [ "$(meta_field "$dir" rl40 base_sha)" != "$moved" ] \ + || fail "base_sha was recaptured at relaunch and now points at the loop's own newest commit" + [ "$(grep -c '^quality=' "$dir/home/state/rl40.meta")" = 1 ] \ + || fail "relaunch left more than one quality= line in the task record" + [ "$(grep -c '^base_sha=' "$dir/home/state/rl40.meta")" = 1 ] \ + || fail "relaunch left more than one base_sha= line in the task record" + [ "$(meta_field "$dir" rl40 mode)" = no-mistakes ] || fail "the delivery mode must survive alongside it" + pass "fm-control relaunch: the quality posture survives and the base commit is read back, never recaptured" +} + test_relaunch_serializes_concurrent_durable_metadata_publication() { local dir control_pid link_pid rc i=0 traceparent prepare ready exported release dir=$(new_case metadata-race rl28) @@ -1284,6 +1332,9 @@ test_spawn_relaunch_refuses_contradicting_flags() { out=$(run_spawn "$dir" rl16 --relaunch --scout); rc=$? expect_code 1 "$rc" "--scout should be refused alongside --relaunch" assert_contains "$out" "recorded kind" "the refusal should name the recorded kind rule" + out=$(run_spawn "$dir" rl16 --relaunch --quality hardened); rc=$? + expect_code 1 "$rc" "--quality should be refused alongside --relaunch" + assert_contains "$out" "recorded quality posture" "the refusal should name the recorded quality rule" out=$(run_spawn "$dir" rl16 "$dir/proj" --relaunch); rc=$? expect_code 1 "$rc" "a project positional should be refused alongside --relaunch" assert_contains "$out" "takes the task id only" "the refusal should name the positional rule" @@ -1314,6 +1365,7 @@ test_spawn_relaunch_refuses_a_pane_outside_the_worktree() { test_same_harness_relaunch_keeps_identity_and_reuses_the_endpoint test_relaunch_preserves_durable_task_metadata +test_relaunch_reuses_the_quality_posture_and_base_commit test_relaunch_serializes_concurrent_durable_metadata_publication test_disabled_relaunch_clears_prior_trace_context test_relaunch_appends_the_progress_note_to_the_instructions diff --git a/tests/fm-task-delivery.test.sh b/tests/fm-task-delivery.test.sh index 34df6fe748a..c9d501ec0db 100755 --- a/tests/fm-task-delivery.test.sh +++ b/tests/fm-task-delivery.test.sh @@ -22,6 +22,18 @@ PROMOTE="$ROOT/bin/fm-promote.sh" PROJECT_MODE="$ROOT/bin/fm-project-mode.sh" TMP_ROOT=$(fm_test_tmproot fm-task-delivery) +# A spawn that gets all the way to metadata also creates /tmp/fm-, which is +# outside TMP_ROOT and therefore outside fm_test_tmproot's cleanup. Track and +# remove each one so this suite leaks nothing on a shared host. +SPAWNED_TASK_TMPS=() +delivery_cleanup() { + local d + for d in "${SPAWNED_TASK_TMPS[@]:-}"; do + [ -z "$d" ] || rm -rf "$d" + done +} +trap delivery_cleanup EXIT + # A home with one registered project, one project directory, and a fake tmux that # refuses, so a spawn that clears the delivery checks still creates nothing. # Echoes "||". @@ -40,12 +52,13 @@ make_home() { # [...] printf '%s\n' "$home|$projects/proj|$fakebin" } -write_brief() { # [] - local home=$1 id=$2 mode=${3:-} +write_brief() { # [] [] + local home=$1 id=$2 mode=${3:-} quality=${4:-} mkdir -p "$home/data/$id" { printf 'You are a crewmate.\n\n# Definition of done\n' [ -z "$mode" ] || printf 'Delivery contract: mode=%s\n' "$mode" + [ -z "$quality" ] || printf 'Quality contract: quality=%s\n' "$quality" } > "$home/data/$id/brief.md" } @@ -272,6 +285,275 @@ EOF pass "fm-project-mode: the conditional policy is accepted, mapped for mechanical callers, and readable raw" } +# The registry's quality posture is the fourth thing a captain can put on a +# project line, and --quality is the only way to read it. Every row here is +# exercised through the real script against a real registry file. +test_project_mode_reads_the_registered_quality_posture() { + local home out label line expect n=0 + home="$TMP_ROOT/project-quality/home" + mkdir -p "$home/data" + while IFS='|' read -r label line expect; do + [ -n "$label" ] || continue + n=$((n + 1)) + printf '%s\n' "$line" > "$home/data/projects.md" + out=$(FM_HOME="$home" "$PROJECT_MODE" --quality qproj 2>/dev/null) + [ "$out" = "$expect" ] || fail "$label: --quality printed '$out', expected '$expect'" + done <<'ROWS' +no annotation at all|- qproj - fixture (added 2026-01-01)|standard +mode only|- qproj [direct-PR] - fixture (added 2026-01-01)|standard +mode and yolo only|- qproj [local-only +yolo] - fixture (added 2026-01-01)|standard +hardened after the mode|- qproj [no-mistakes +hardened] - fixture (added 2026-01-01)|hardened +hardened after mode and yolo|- qproj [direct-PR +yolo +hardened] - fixture (added 2026-01-01)|hardened +hardened between mode and yolo|- qproj [direct-PR +hardened +yolo] - fixture (added 2026-01-01)|hardened +hardened before the mode|- qproj [+hardened local-only +yolo] - fixture (added 2026-01-01)|hardened +hardened on a conditional policy|- qproj [no-mistakes-prod-only +hardened] - fixture (added 2026-01-01)|hardened +an unrecognized flag is ignored, not refused|- qproj [direct-PR +from-the-future] - fixture (added 2026-01-01)|standard +ROWS + # An absent project and an absent registry both resolve to the safe posture + # rather than inheriting the previous row's answer. + out=$(FM_HOME="$home" "$PROJECT_MODE" --quality never-registered 2>/dev/null) + [ "$out" = standard ] || fail "an unregistered project resolved quality '$out', expected standard" + out=$(FM_HOME="$TMP_ROOT/project-quality/no-such-home" "$PROJECT_MODE" --quality qproj 2>/dev/null) + [ "$out" = standard ] || fail "an absent registry resolved quality '$out', expected standard" + pass "fm-project-mode: --quality reads +hardened from any bracket position and defaults to standard" +} + +# The load-bearing registry case. Three callers parse this script's two words +# (bin/fm-fleet-sync.sh, bin/fm-home-seed.sh, bin/fm-spawn.sh), so adding the +# quality posture must leave that stdout exactly as it was: still two words, the +# same two words, for every annotation form including the new one. +test_project_mode_two_word_contract_survives_the_quality_posture() { + local home out label line expect n=0 + home="$TMP_ROOT/project-twoword/home" + mkdir -p "$home/data" + while IFS='|' read -r label line expect; do + [ -n "$label" ] || continue + n=$((n + 1)) + printf '%s\n' "$line" > "$home/data/projects.md" + out=$(FM_HOME="$home" "$PROJECT_MODE" qproj 2>/dev/null) + [ "$out" = "$expect" ] || fail "$label: printed '$out', expected '$expect'" + [ "$(printf '%s' "$out" | wc -w)" -eq 2 ] || fail "$label: stdout was not exactly two words ('$out')" + out=$(FM_HOME="$home" "$PROJECT_MODE" --raw qproj 2>/dev/null) + [ "$(printf '%s' "$out" | wc -w)" -eq 2 ] || fail "$label: --raw stdout was not exactly two words ('$out')" + done <<'ROWS' +no annotation at all|- qproj - fixture (added 2026-01-01)|no-mistakes off +mode only|- qproj [direct-PR] - fixture (added 2026-01-01)|direct-PR off +mode and yolo|- qproj [local-only +yolo] - fixture (added 2026-01-01)|local-only on +yolo only|- qproj [+yolo] - fixture (added 2026-01-01)|no-mistakes on +conditional policy|- qproj [no-mistakes-prod-only] - fixture (added 2026-01-01)|no-mistakes off +conditional policy with yolo|- qproj [no-mistakes-prod-only +yolo] - fixture (added 2026-01-01)|no-mistakes on +unrecognized flag ignored|- qproj [direct-PR +from-the-future] - fixture (added 2026-01-01)|direct-PR off +hardened does not disturb the mode|- qproj [direct-PR +hardened] - fixture (added 2026-01-01)|direct-PR off +hardened does not disturb mode or yolo|- qproj [local-only +yolo +hardened] - fixture (added 2026-01-01)|local-only on +hardened first still resolves the mode behind it|- qproj [+hardened local-only +yolo] - fixture (added 2026-01-01)|local-only on +ROWS + # A typo'd mode keeps warning and keeps falling back, rather than being + # silently rescued by the new flag scan. + printf '%s\n' "- qproj [no-mistakez +hardened] - fixture (added 2026-01-01)" > "$home/data/projects.md" + out=$(FM_HOME="$home" "$PROJECT_MODE" qproj 2>/dev/null) + [ "$out" = "no-mistakes off" ] || fail "a typo'd mode alongside +hardened resolved '$out'" + out=$(FM_HOME="$home" "$PROJECT_MODE" qproj 2>&1 >/dev/null) + assert_contains "$out" "unknown mode" "a typo'd mode alongside +hardened stopped warning" + pass "fm-project-mode: the two-word stdout its three callers parse is unchanged by the quality posture" +} + +# A scout has no quality loop to run and a charter is not a delivery contract, so +# --quality is refused there rather than accepted and quietly ignored. A ship +# spawn accepts it but validates the value, because a typo must never ship a +# hardened task down the standard path. +test_scout_and_secondmate_refuse_the_quality_flag() { + local rec home proj fakebin out status + rec=$(make_home quality-refused) + IFS='|' read -r home proj fakebin <|||". +make_spawning_home() { # + local name=$1 dir home proj wt fakebin + dir="$TMP_ROOT/$name" + home="$dir/home" + proj="$dir/proj" + wt="$dir/wt" + fakebin=$(fm_fakebin "$dir/fake") + cat > "$fakebin/tmux" <<'SH' +#!/usr/bin/env bash +set -u +case "$*" in + *"#{pane_current_path}"*) printf '%s\n' "${FM_FAKE_PANE_PATH:-}"; exit 0 ;; +esac +case "${1:-}" in + display-message) printf 'firstmate\n'; exit 0 ;; + list-windows) exit 0 ;; +esac +exit 0 +SH + chmod +x "$fakebin/tmux" + fm_fake_exit0 "$fakebin" treehouse + mkdir -p "$home/data" "$home/projects" "$home/state" "$home/config" + printf 'claude\n' > "$home/config/crew-harness" + printf '%s\n' "$$" > "$home/state/.lock" + touch "$home/state/.last-watcher-beat" + fm_git_worktree "$proj" "$wt" "wt-$name" + printf '%s\n' "$home|$proj|$wt|$fakebin" +} + +run_spawning() { # + local home=$1 wt=$2 fakebin=$3 + shift 3 + SPAWNED_TASK_TMPS+=("/tmp/fm-$1") + # `env -u` keeps the recorded key set hermetic against an ambient + # FM_TRACE_CONTEXT, which would otherwise add a traceparent= line. + env -u FM_TRACE_CONTEXT \ + FM_ROOT_OVERRIDE='' FM_HOME="$home" \ + FM_STATE_OVERRIDE="$home/state" FM_DATA_OVERRIDE="$home/data" \ + FM_PROJECTS_OVERRIDE="$home/projects" FM_CONFIG_OVERRIDE="$home/config" \ + FM_SPAWN_NO_GUARD=1 FM_BACKEND=tmux FM_FAKE_PANE_PATH="$wt" TMUX="fake,1,0" \ + PATH="$fakebin:$PATH" \ + "$SPAWN" "$@" 2>&1 +} + +meta_value() { # + sed -n "s/^$2=//p" "$1" | tail -n 1 +} + +# The quality posture and the base commit have to reach the task's durable record, +# because that record is what the quality loop reads back later. base_sha must be +# the commit the worktree actually starts from - the whole loop measures diffs +# against it - so this asserts it against a real `git rev-parse HEAD`, not a shape. +# +# The load-bearing half is the standard task: a spawn with no --quality must write +# the record it always wrote, with nothing removed, nothing changed, and only the +# two new additive lines present. +test_spawn_records_the_quality_posture_and_base_commit() { + local rec home proj wt fakebin meta out status base keys + rec=$(make_spawning_home quality-meta) + IFS='|' read -r home proj wt fakebin < Date: Sat, 22 Aug 2026 09:28:47 +0800 Subject: [PATCH 02/24] no-mistakes(review): add quality standing-posture notice, correct fallback header --- bin/fm-project-mode.sh | 8 ++++++-- bin/fm-spawn.sh | 7 +++++++ tests/fm-task-delivery.test.sh | 36 ++++++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+), 2 deletions(-) diff --git a/bin/fm-project-mode.sh b/bin/fm-project-mode.sh index 6a6bedacef4..f701937ed2b 100755 --- a/bin/fm-project-mode.sh +++ b/bin/fm-project-mode.sh @@ -50,8 +50,12 @@ # output path precisely so the two-word stdout contract above stays untouched. # # An unknown/missing project or unknown mode falls back to "no-mistakes off" and warns -# to stderr, so a typo never silently drops the gate; --quality falls back to -# "standard" on the same inputs. +# to stderr, so a typo never silently drops the gate. +# The quality posture resolves independently of that fallback. +# A missing registry file, or a project absent from the registry, does yield "standard". +# An unrecognised mode token resets only the mode and the yolo flag and keeps a +# "+hardened" parsed beside it, because a typo in the mode must not silently drop the +# quality gate too; the unknown-mode warning still goes to stderr. # Usage: fm-project-mode.sh [--raw] [--quality] set -eu diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index b9e9f805425..f07250ad430 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -1793,6 +1793,13 @@ if [ "$KIND" = ship ]; then && [ "$(delivery_rigor_rank "$MODE")" -lt "$(delivery_rigor_rank "$STANDING_MODE")" ]; then echo "notice: $ID ships mode=$MODE while the standing posture for $PROJ_NAME is $STANDING_MODE - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 fi + # The same notice for the quality posture, which is a plain two-value token: there + # is no conditional policy to exclude, so a hardened standing posture shipped as a + # standard task is the only downgrade there is. Advisory only, like the mode notice. + STANDING_QUALITY=$("$FM_ROOT/bin/fm-project-mode.sh" --quality "$PROJ_NAME" 2>/dev/null) || STANDING_QUALITY= + if [ "$STANDING_QUALITY" = hardened ] && [ "$QUALITY" = standard ]; then + echo "notice: $ID ships quality=$QUALITY while the standing posture for $PROJ_NAME is $STANDING_QUALITY - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 + fi fi BRIEF_DIR_REAL=$(cd "$(dirname "$BRIEF")" && pwd -P) diff --git a/tests/fm-task-delivery.test.sh b/tests/fm-task-delivery.test.sh index c9d501ec0db..9131503a21c 100755 --- a/tests/fm-task-delivery.test.sh +++ b/tests/fm-task-delivery.test.sh @@ -554,6 +554,41 @@ EOF pass "fm-spawn: a ship task records quality= and base_sha= additively, and a scout records neither" } +# The registry is the captain's standing quality posture too, so a hardened project +# shipped as a standard task is announced for the same reason a rigor downgrade is: +# allowed on a current captain instruction, never silent. It is advisory only, so the +# spawn still succeeds and still records the standard posture it was handed. +test_spawn_notices_a_quality_downgrade_against_the_registry() { + local rec home proj wt fakebin out status meta + rec=$(make_spawning_home quality-standing) + IFS='|' read -r home proj wt fakebin < "$home/data/projects.md" + write_brief "$home" quality-standing-i1 no-mistakes + out=$(run_spawning "$home" "$wt" "$fakebin" quality-standing-i1 "$proj" --mode no-mistakes --yolo off) + status=$? + expect_code 0 "$status" "the quality notice must not block the spawn"$'\n'"$out" + assert_contains "$out" "ships quality=standard while the standing posture for proj is hardened" \ + "no notice when a hardened project shipped a standard task" + meta="$home/state/quality-standing-i1.meta" + assert_present "$meta" "the announced spawn wrote no task record" + [ "$(meta_value "$meta" quality)" = standard ] \ + || fail "the notice changed the recorded posture to '$(meta_value "$meta" quality)'" + + # 2. The same spawn against a project carrying no +hardened token stays quiet. + printf '%s\n' '- proj [no-mistakes] - fixture (added 2026-01-01)' > "$home/data/projects.md" + write_brief "$home" quality-standing-i2 no-mistakes + out=$(run_spawning "$home" "$wt" "$fakebin" quality-standing-i2 "$proj" --mode no-mistakes --yolo off) + status=$? + expect_code 0 "$status" "a standard project spawn should succeed"$'\n'"$out" + assert_not_contains "$out" "ships quality=" \ + "a project with no registered quality posture printed a quality notice" + pass "fm-spawn: a standard task under a hardened standing posture is announced, never blocked" +} + test_ship_spawn_requires_a_valid_delivery_contract test_scout_and_secondmate_refuse_delivery_flags test_spawn_refuses_a_brief_mode_mismatch @@ -566,4 +601,5 @@ test_project_mode_two_word_contract_survives_the_quality_posture test_scout_and_secondmate_refuse_the_quality_flag test_spawn_refuses_a_brief_quality_mismatch test_spawn_records_the_quality_posture_and_base_commit +test_spawn_notices_a_quality_downgrade_against_the_registry echo "# all fm-task-delivery tests passed" From aaa22727177f7015b1106aac448b7d74f5dd1af1 Mon Sep 17 00:00:00 2001 From: PP <121104417+BohnBawerick@users.noreply.github.com> Date: Sat, 22 Aug 2026 09:41:44 +0800 Subject: [PATCH 03/24] no-mistakes(review): fix test suite teardown, header, typo-fallback coverage --- tests/fm-task-delivery.test.sh | 45 ++++++++++++++++++++++++++++------ 1 file changed, 37 insertions(+), 8 deletions(-) diff --git a/tests/fm-task-delivery.test.sh b/tests/fm-task-delivery.test.sh index 9131503a21c..4335dd35252 100755 --- a/tests/fm-task-delivery.test.sh +++ b/tests/fm-task-delivery.test.sh @@ -9,9 +9,14 @@ # spawns carry no delivery posture at all. The registry keeps only the captain's # standing posture, for the mechanical consumers and for one advisory notice. # -# Every spawn case here stops before any endpoint exists: the delivery checks run -# ahead of backend creation, and a fake `tmux` that exits non-zero backstops the -# cases that are meant to get past them, so no window or worktree is ever created. +# The delivery-check cases stop before any endpoint exists: those checks run ahead +# of backend creation, and a fake `tmux` that exits non-zero backstops the cases +# that are meant to get past them, so no window or worktree is ever created. The +# metadata and standing-posture cases are the opposite on purpose: make_spawning_home +# builds a real git worktree and a fake `tmux` that exits 0 and answers the +# pane-path query, so run_spawning carries a ship spawn all the way to its durable +# record. Those cases really do create things, including /tmp/fm- outside +# TMP_ROOT, so they own the teardown in delivery_cleanup below. set -u # shellcheck source=tests/lib.sh @@ -25,14 +30,33 @@ TMP_ROOT=$(fm_test_tmproot fm-task-delivery) # A spawn that gets all the way to metadata also creates /tmp/fm-, which is # outside TMP_ROOT and therefore outside fm_test_tmproot's cleanup. Track and # remove each one so this suite leaks nothing on a shared host. -SPAWNED_TASK_TMPS=() +# +# The tracking goes through a `$$`-keyed file, not an array, for the reason +# tests/lib.sh gives under "self-cleaning temp root": every spawn here is invoked +# as `out=$(run_spawning ...)`, which forks a subshell, so an array append made +# inside it dies with that subshell and never reaches this shell. `$$` stays the +# invoking shell's PID across that boundary, so the file does reach cleanup. +# +# This trap replaces the shared EXIT trap tests/lib.sh arms at source time, so it +# ends by calling fm_test_cleanup itself: TMP_ROOT and lib.sh's own registry still +# go, and they go on the failing path too, because fail() exits. +SPAWNED_TASK_REGISTRY=$(mktemp "${TMPDIR:-/tmp}/.fm-task-delivery-tmps.$$.XXXXXX") +track_spawned_task_tmp() { # + printf '%s\n' "$1" >> "$SPAWNED_TASK_REGISTRY" 2>/dev/null || true +} delivery_cleanup() { local d - for d in "${SPAWNED_TASK_TMPS[@]:-}"; do - [ -z "$d" ] || rm -rf "$d" - done + if [ -f "$SPAWNED_TASK_REGISTRY" ]; then + while IFS= read -r d; do + [ -z "$d" ] || rm -rf "$d" + done < "$SPAWNED_TASK_REGISTRY" + rm -f "$SPAWNED_TASK_REGISTRY" + fi + fm_test_cleanup } trap delivery_cleanup EXIT +trap 'delivery_cleanup; exit 130' INT +trap 'delivery_cleanup; exit 143' TERM # A home with one registered project, one project directory, and a fake tmux that # refuses, so a spawn that clears the delivery checks still creates nothing. @@ -354,6 +378,11 @@ ROWS [ "$out" = "no-mistakes off" ] || fail "a typo'd mode alongside +hardened resolved '$out'" out=$(FM_HOME="$home" "$PROJECT_MODE" qproj 2>&1 >/dev/null) assert_contains "$out" "unknown mode" "a typo'd mode alongside +hardened stopped warning" + # The quality posture resolves on its own, so the mode fallback does not take the + # +hardened down with it: a typo in the mode must not silently drop the gate. + out=$(FM_HOME="$home" "$PROJECT_MODE" --quality qproj 2>/dev/null) + [ "$out" = hardened ] \ + || fail "a typo'd mode dropped the registered quality posture to '$out', expected hardened" pass "fm-project-mode: the two-word stdout its three callers parse is unchanged by the quality posture" } @@ -469,7 +498,7 @@ SH run_spawning() { # local home=$1 wt=$2 fakebin=$3 shift 3 - SPAWNED_TASK_TMPS+=("/tmp/fm-$1") + track_spawned_task_tmp "/tmp/fm-$1" # `env -u` keeps the recorded key set hermetic against an ambient # FM_TRACE_CONTEXT, which would otherwise add a traceparent= line. env -u FM_TRACE_CONTEXT \ From 32f566d8559e130b4fb74daf6136cc87d9eed522 Mon Sep 17 00:00:00 2001 From: PP <121104417+BohnBawerick@users.noreply.github.com> Date: Sat, 22 Aug 2026 09:59:27 +0800 Subject: [PATCH 04/24] no-mistakes(review): add promote quality notice, fix brief help, pin base capture --- bin/fm-brief.sh | 7 ++-- bin/fm-promote.sh | 24 +++++++++++- tests/fm-task-delivery.test.sh | 70 ++++++++++++++++++++++++++++++++-- 3 files changed, 93 insertions(+), 8 deletions(-) diff --git a/bin/fm-brief.sh b/bin/fm-brief.sh index 00cd3ab3002..3941dd259a8 100755 --- a/bin/fm-brief.sh +++ b/bin/fm-brief.sh @@ -59,10 +59,11 @@ # brief carries neither, so an absent line means standard and a standard brief stays # byte-identical to what this scaffold produced before --quality existed. bin/fm-spawn.sh # checks that line against its own --quality exactly as it checks the mode line. -# --quality is refused on scout and secondmate scaffolds for the same reason --mode is. +# --quality is refused on scout, dreamer and secondmate scaffolds, for the same reason +# --mode is. # Ship briefs begin with a worktree-isolation assertion before the branch step. -# --mode is refused on scout and secondmate scaffolds: a scout's deliverable is a -# report rather than a merge, and a charter is not a delivery contract. +# --mode is refused on scout, dreamer and secondmate scaffolds: a scout or dreamer +# delivers a report rather than a merge, and a charter is not a delivery contract. # There is no --yolo flag here. The worker never owns approval decisions, so yolo is # a spawn-time and firstmate-side input only (AGENTS.md section 7). # Every scaffold's status protocol distinguishes the configured diff --git a/bin/fm-promote.sh b/bin/fm-promote.sh index 0ed1fd06161..868feb6f543 100755 --- a/bin/fm-promote.sh +++ b/bin/fm-promote.sh @@ -9,8 +9,15 @@ # A scout records no delivery posture, so promotion is where this task's delivery # contract is decided: --mode and --yolo are REQUIRED and written into the meta # alongside the kind= flip. Firstmate resolves both at promotion time, having just -# read the scout's report (AGENTS.md section 7); data/projects.md holds the -# captain's standing posture as context, and this script never looks it up. +# read the scout's report (AGENTS.md section 7). data/projects.md holds the captain's +# standing posture, and this script reads only its quality half, only to print one +# advisory notice on stderr after a successful promotion; the delivery mode stays the +# caller's explicit decision. +# A promoted task deliberately records no quality= and no base_sha=. The base commit +# cannot be captured here, because the promoted worker resets to a clean +# default-branch base only afterwards, and a hardened record with no anchor would look +# complete to the quality loop while being unanchored. Both keys belong to the task +# that owns the base-capture question (bin/fm-quality.sh). # no-mistakes-prod-only is a registry policy rather than a task mode and is refused. # Usage: fm-promote.sh --mode --yolo set -eu @@ -120,6 +127,19 @@ TMP= fm_lock_release "$META_LOCK" META_LOCK_HELD=0 +# The quality sibling of the standing-posture notice in bin/fm-spawn.sh: advisory +# only, printed after the record is already rewritten so it can never affect the +# promotion. A record with no project=, a missing registry, or a failed lookup simply +# skips it, exactly as the spawn notice tolerates an empty standing mode. +PROMOTED_PROJECT=$(sed -n 's/^project=//p' "$META" | tail -n 1) +PROMOTED_PROJECT=${PROMOTED_PROJECT##*/} +if [ -n "$PROMOTED_PROJECT" ]; then + STANDING_QUALITY=$("$FM_ROOT/bin/fm-project-mode.sh" --quality "$PROMOTED_PROJECT" 2>/dev/null) || STANDING_QUALITY= + if [ "$STANDING_QUALITY" = hardened ]; then + echo "notice: $ID promotes carrying no quality posture while the standing posture for $PROMOTED_PROJECT is hardened - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 + fi +fi + HOME_Q=$(printf '%q' "$FM_HOME") echo "promoted $ID to ship mode=$MODE yolo=$YOLO (teardown protection restored)" echo "next: FM_HOME=$HOME_Q bin/fm-send.sh fm-$ID ''" diff --git a/tests/fm-task-delivery.test.sh b/tests/fm-task-delivery.test.sh index 4335dd35252..04f3b944fba 100755 --- a/tests/fm-task-delivery.test.sh +++ b/tests/fm-task-delivery.test.sh @@ -275,6 +275,57 @@ test_promote_requires_and_records_the_delivery_contract() { pass "fm-promote: promotion requires the delivery contract and records it exactly once" } +# A promoted task carries no quality posture at all, so the registry's standing +# posture is announced rather than silently lost. Advisory only: the promotion still +# happens, and a record with no project= to look up promotes quietly. +test_promote_notices_the_standing_quality_posture() { + local home meta out status + home="$TMP_ROOT/promote-quality/home" + mkdir -p "$home/state" "$home/data" + + run_promote() { # + FM_HOME="$home" FM_STATE_OVERRIDE="$home/state" FM_DATA_OVERRIDE="$home/data" \ + "$PROMOTE" "$1" --mode direct-PR --yolo on 2>&1 + } + + # 1. A hardened project: the notice fires and the promotion still lands. + printf '%s\n' '- proj [no-mistakes +hardened] - fixture (added 2026-01-01)' > "$home/data/projects.md" + meta="$home/state/promote-q1.meta" + printf 'window=fm-promote-q1\nkind=scout\nworktree=/tmp/wt\nproject=%s/projects/proj\n' "$home" > "$meta" + out=$(run_promote promote-q1) + status=$? + expect_code 0 "$status" "the quality notice must not block a promotion"$'\n'"$out" + assert_contains "$out" "the standing posture for proj is hardened" \ + "no notice when a hardened project's scout was promoted" + assert_grep 'kind=ship' "$meta" "the announced promotion did not restore ship teardown protection" + assert_grep 'mode=direct-PR' "$meta" "the announced promotion did not record the decided delivery mode" + assert_grep 'yolo=on' "$meta" "the announced promotion did not record the decided approval posture" + assert_no_grep 'quality=' "$meta" "a promoted task recorded a quality posture it cannot anchor" + assert_no_grep 'base_sha=' "$meta" "a promoted task recorded a base commit it cannot capture" + + # 2. The same promotion against a project with no +hardened token stays quiet. + printf '%s\n' '- proj [no-mistakes] - fixture (added 2026-01-01)' > "$home/data/projects.md" + meta="$home/state/promote-q2.meta" + printf 'window=fm-promote-q2\nkind=scout\nworktree=/tmp/wt\nproject=%s/projects/proj\n' "$home" > "$meta" + out=$(run_promote promote-q2) + status=$? + expect_code 0 "$status" "a standard project's promotion should succeed"$'\n'"$out" + assert_not_contains "$out" "standing posture" \ + "a project with no registered quality posture printed a notice" + + # 3. A record with no project= line has nothing to look up and promotes silently. + printf '%s\n' '- proj [no-mistakes +hardened] - fixture (added 2026-01-01)' > "$home/data/projects.md" + meta="$home/state/promote-q3.meta" + printf 'window=fm-promote-q3\nkind=scout\nworktree=/tmp/wt\n' > "$meta" + out=$(run_promote promote-q3) + status=$? + expect_code 0 "$status" "a promotion with no project= should still succeed"$'\n'"$out" + assert_not_contains "$out" "standing posture" \ + "a record with no project= resolved a standing posture from somewhere" + assert_grep 'kind=ship' "$meta" "a promotion with no project= did not rewrite the record" + pass "fm-promote: a hardened standing posture is announced on promotion, never blocked" +} + # The registry parser survives for the mechanical consumers only. It accepts the # conditional policy, maps it to its most rigorous leg for them, and exposes the # raw annotation for the one caller that must tell a policy from a flat mode. @@ -523,12 +574,22 @@ meta_value() { # # the record it always wrote, with nothing removed, nothing changed, and only the # two new additive lines present. test_spawn_records_the_quality_posture_and_base_commit() { - local rec home proj wt fakebin meta out status base keys + local rec home proj wt fakebin meta out status base prespawn keys rec=$(make_spawning_home quality-meta) IFS='|' read -r home proj wt fakebin < Date: Sat, 22 Aug 2026 10:14:52 +0800 Subject: [PATCH 05/24] no-mistakes(review): document hardened registration, refuse it on conditional policy --- .agents/skills/project-management/SKILL.md | 6 +++ bin/fm-project-mode.sh | 11 +++++ tests/fm-task-delivery.test.sh | 49 +++++++++++++++++++++- 3 files changed, 65 insertions(+), 1 deletion(-) diff --git a/.agents/skills/project-management/SKILL.md b/.agents/skills/project-management/SKILL.md index 8feb522bd0c..9690b7d695a 100644 --- a/.agents/skills/project-management/SKILL.md +++ b/.agents/skills/project-management/SKILL.md @@ -42,12 +42,18 @@ Choose that posture when adding or creating the project: - `direct-PR` pushes and opens a PR without the no-mistakes pipeline. - `local-only` has no required remote or PR and lands only through the approved local fast-forward path. - `no-mistakes-prod-only` is a conditional policy rather than one flat mode: genuinely internal-only tooling, automation, contributor or operator process, and release or submission work ships `direct-PR`, while product-facing, mixed, and uncertain work ships `no-mistakes`. +- `+hardened` is the highest-rigor choice on this list, adding the quality gate that runs before validation; it rides alongside one of the flat modes above rather than replacing it, so a hardened project is registered as `[no-mistakes +hardened]`, `[direct-PR +hardened]`, or `[local-only +hardened]`. `no-mistakes-prod-only` is the default for a newly added or created remote-backed project when the captain specifies nothing, and a project with no remote defaults to `local-only`. State that resolved default while confirming the source, local name, and posture instead of asking the captain to choose from scratch, and record a flat mode instead whenever they ask for one. Existing registry entries keep the meaning they already have and are never migrated or reinterpreted, so a legacy entry with no bracket stays `no-mistakes`. Registering a conditional policy is a one-time choice and never requires classifying any change; the per-task surface classification happens at each task's intake, and internal-only is never inferred from file location or project name. +`+hardened` is off for every project unless the captain asks for it, so a project registered without it is `standard`. +Refuse `+hardened` together with `no-mistakes-prod-only` and tell the captain to pick a flat delivery mode instead. +A conditional policy decides per task, so a quality standard that covers only part of a project is a posture nobody can state in one sentence. +`AGENTS.md` section 7 owns how each task's quality resolves at intake, and `bin/fm-project-mode.sh --quality` owns how the registered token is read. + The optional `+yolo` posture changes routine approval authority but does not change the delivery mode. Default it off for every project and every posture, and enable it only on the captain's explicit instruction. `AGENTS.md` section 7 owns the complete authority boundary and exceptions when it is on. diff --git a/bin/fm-project-mode.sh b/bin/fm-project-mode.sh index f701937ed2b..f739a50d17a 100755 --- a/bin/fm-project-mode.sh +++ b/bin/fm-project-mode.sh @@ -56,6 +56,13 @@ # An unrecognised mode token resets only the mode and the yolo flag and keeps a # "+hardened" parsed beside it, because a typo in the mode must not silently drop the # quality gate too; the unknown-mode warning still goes to stderr. +# "+hardened" beside "no-mistakes-prod-only" is the opposite case and drops to +# "standard" with its own stderr warning: a hardened project must pick a flat delivery +# mode, because a conditional policy decides per task and a quality standard covering +# only part of a project is not a statable posture +# (.agents/skills/project-management/SKILL.md "Delivery posture"). Unlike the typo, +# that combination parses cleanly and was ruled out on purpose. The mode still resolves +# to no-mistakes-prod-only, the two-word stdout is unchanged, and the exit stays 0. # Usage: fm-project-mode.sh [--raw] [--quality] set -eu @@ -127,6 +134,10 @@ case "$mode" in esac case "$yolo" in on|off) ;; *) yolo=off ;; esac case "$quality" in standard|hardened) ;; *) quality=standard ;; esac +if [ "$mode" = no-mistakes-prod-only ] && [ "$quality" = hardened ]; then + echo "warn: +hardened is refused alongside the conditional policy no-mistakes-prod-only for $NAME; a hardened project must pick a flat delivery mode (no-mistakes, direct-PR or local-only), so defaulting quality to standard" >&2 + quality=standard +fi # A conditional policy is not a task mode. Mechanical callers get its most # rigorous leg; --raw callers get the annotation itself (see the header). if [ "$RAW" -eq 0 ] && [ "$mode" = no-mistakes-prod-only ]; then diff --git a/tests/fm-task-delivery.test.sh b/tests/fm-task-delivery.test.sh index 04f3b944fba..2012d32971f 100755 --- a/tests/fm-task-delivery.test.sh +++ b/tests/fm-task-delivery.test.sh @@ -381,7 +381,7 @@ hardened after the mode|- qproj [no-mistakes +hardened] - fixture (added 2026-01 hardened after mode and yolo|- qproj [direct-PR +yolo +hardened] - fixture (added 2026-01-01)|hardened hardened between mode and yolo|- qproj [direct-PR +hardened +yolo] - fixture (added 2026-01-01)|hardened hardened before the mode|- qproj [+hardened local-only +yolo] - fixture (added 2026-01-01)|hardened -hardened on a conditional policy|- qproj [no-mistakes-prod-only +hardened] - fixture (added 2026-01-01)|hardened +hardened on a conditional policy is refused and drops to standard|- qproj [no-mistakes-prod-only +hardened] - fixture (added 2026-01-01)|standard an unrecognized flag is ignored, not refused|- qproj [direct-PR +from-the-future] - fixture (added 2026-01-01)|standard ROWS # An absent project and an absent registry both resolve to the safe posture @@ -393,6 +393,52 @@ ROWS pass "fm-project-mode: --quality reads +hardened from any bracket position and defaults to standard" } +# The registry is the only way to turn the quality gate on, so this reader is where +# the registration rule is backed mechanically +# (.agents/skills/project-management/SKILL.md "Delivery posture"). +hardened rides a +# flat mode; alongside the conditional policy it is refused, because a policy that +# decides per task cannot carry one statable quality posture. The refusal follows the +# unknown-mode precedent: warn on stderr, resolve to the safe value, leave the +# two-word stdout its three callers parse alone, and exit 0. +test_project_mode_refuses_hardened_on_the_conditional_policy() { + local home out err status label line quality words n=0 + home="$TMP_ROOT/project-hardened-policy/home" + mkdir -p "$home/data" + while IFS='|' read -r label line quality words; do + [ -n "$label" ] || continue + n=$((n + 1)) + printf '%s\n' "$line" > "$home/data/projects.md" + out=$(FM_HOME="$home" "$PROJECT_MODE" --quality qproj 2>/dev/null) + status=$? + expect_code 0 "$status" "$label: --quality exited non-zero" + [ "$out" = "$quality" ] || fail "$label: --quality printed '$out', expected '$quality'" + out=$(FM_HOME="$home" "$PROJECT_MODE" qproj 2>/dev/null) + [ "$out" = "$words" ] || fail "$label: the two-word stdout printed '$out', expected '$words'" + err=$(FM_HOME="$home" "$PROJECT_MODE" --quality qproj 2>&1 >/dev/null) + case "$label" in + refused*) + assert_contains "$err" "+hardened is refused" "$label: the refused combination printed no warning" + assert_contains "$err" "flat delivery mode" "$label: the warning did not say how to fix the registry line" ;; + *) + assert_not_contains "$err" "+hardened is refused" "$label: a legitimate registry line was warned about" ;; + esac + done <<'ROWS' +hardened rides no-mistakes|- qproj [no-mistakes +hardened] - fixture (added 2026-01-01)|hardened|no-mistakes off +hardened rides direct-PR|- qproj [direct-PR +hardened] - fixture (added 2026-01-01)|hardened|direct-PR off +hardened rides local-only|- qproj [local-only +hardened] - fixture (added 2026-01-01)|hardened|local-only off +hardened rides a flat mode with yolo|- qproj [direct-PR +yolo +hardened] - fixture (added 2026-01-01)|hardened|direct-PR on +refused alongside the conditional policy|- qproj [no-mistakes-prod-only +hardened] - fixture (added 2026-01-01)|standard|no-mistakes off +refused alongside the conditional policy with yolo|- qproj [no-mistakes-prod-only +yolo +hardened] - fixture (added 2026-01-01)|standard|no-mistakes on +the conditional policy without hardened stays quiet|- qproj [no-mistakes-prod-only] - fixture (added 2026-01-01)|standard|no-mistakes off +ROWS + # --raw still reports the registered annotation: only the quality posture drops. + printf '%s\n' '- qproj [no-mistakes-prod-only +hardened] - fixture (added 2026-01-01)' > "$home/data/projects.md" + out=$(FM_HOME="$home" "$PROJECT_MODE" --raw qproj 2>/dev/null) + [ "$out" = "no-mistakes-prod-only off" ] \ + || fail "the refusal changed the raw annotation to '$out', expected 'no-mistakes-prod-only off'" + pass "fm-project-mode: +hardened rides a flat mode and is refused on the conditional policy" +} + # The load-bearing registry case. Three callers parse this script's two words # (bin/fm-fleet-sync.sh, bin/fm-home-seed.sh, bin/fm-spawn.sh), so adding the # quality posture must leave that stdout exactly as it was: still two words, the @@ -690,6 +736,7 @@ test_promote_requires_and_records_the_delivery_contract test_promote_notices_the_standing_quality_posture test_project_mode_maps_the_conditional_policy test_project_mode_reads_the_registered_quality_posture +test_project_mode_refuses_hardened_on_the_conditional_policy test_project_mode_two_word_contract_survives_the_quality_posture test_scout_and_secondmate_refuse_the_quality_flag test_spawn_refuses_a_brief_quality_mismatch From 18758793385448283152954733faa31afc2f952a Mon Sep 17 00:00:00 2001 From: PP <121104417+BohnBawerick@users.noreply.github.com> Date: Sat, 22 Aug 2026 11:01:46 +0800 Subject: [PATCH 06/24] no-mistakes(review): document hardened token in architecture and README grammar --- README.md | 2 +- docs/architecture.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8ed5226b171..4446ed5a39f 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ Launching a supported harness inside it instantiates your first mate - and makes - **A visible crew** - every crewmate works in its own tmux window, experimental herdr/zellij tab, cmux workspace, or Orca terminal you can watch or type into; the first mate reconciles. - **Disposable worktrees** - each task runs in a clean [treehouse](https://github.com/kunchenguid/treehouse) git worktree, or an Orca-managed worktree when `backend=orca`, so parallel work on one repo never collides. - **Two task shapes** - ship tasks deliver authorized changes; scout tasks leave standalone investigation reports when the intake contract warrants separate research. -- **Explicit project modes** - each project ships via `no-mistakes`, `direct-PR`, or `local-only`, with an optional `+yolo` autonomy flag. +- **Explicit project modes** - each project ships via `no-mistakes`, `direct-PR`, `local-only`, or one of those plus `+hardened` for the highest-rigor quality gate, with an optional `+yolo` autonomy flag. - **Optional secondmates** - opt in to persistent second mates that run from isolated firstmate homes with their own `FM_HOME`, state, projects, and session lock, either locally or as a whole home on an SSH-reachable host, with guarded updates and recovery that never turns an unavailable remote route into a local replacement. - **Event-driven, zero-token supervision** - a bash watcher sleeps on the fleet and wakes the first mate only when something needs you; verified primary harnesses also get a turn-end backstop that blocks or follows up on a blind stop when work is under way and supervision is not live. - **Optional Relay** - opt in with one local `.env` pairing token so firstmate can answer your public mentions on X and Discord alike, act on normal reversible mention requests through the same lifecycle as chat requests, acknowledge spawned work, and post up to three public-safe completion follow-ups within seven days for genuine milestones and the final outcome without changing non-Relay behavior; a final reply promised in a thread becomes durable state that is reconciled from disk, so a restart or a compacted conversation cannot lose it; dry-run preview records would-be replies and dismissals locally before go-live. diff --git a/docs/architecture.md b/docs/architecture.md index 520c881893f..8f4f018e167 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -250,7 +250,7 @@ The `data/secondmates.md` line contract is owned by the [`secondmate-provisionin `no-mistakes` tasks run the full validation pipeline, `direct-PR` tasks open PRs without that pipeline, and `local-only` tasks stay local until firstmate performs an approved fast-forward merge. Each task's mode and `yolo` posture are firstmate's decision at intake and are passed explicitly to `bin/fm-brief.sh`, `bin/fm-spawn.sh`, and `bin/fm-promote.sh`, which refuse a ship task that does not carry them. A ship brief records its mode as a fixed machine-readable line and the spawn refuses to launch on a different one, so the worker's instructions and the recorded task delivery cannot diverge. -`data/projects.md` records each project's standing posture and optional `+yolo` flag as the captain's default and as context for that decision, including the conditional `no-mistakes-prod-only` policy; a ship spawn that drops below the registered rigor prints a deviation notice and continues. +`data/projects.md` records each project's standing posture, its optional `+hardened` quality posture, and its optional `+yolo` flag as the captain's default and as context for that decision, including the conditional `no-mistakes-prod-only` policy, which `+hardened` may not ride; a ship spawn or a promotion that drops below either registered posture prints a deviation notice and continues. `bin/fm-project-mode.sh` remains the one registry parser for the mechanical consumers that have no task in hand: fleet sync's `local-only` skip and home seeding's refusal and no-mistakes initialization. When a selected delivery path calls for a diff, `bin/fm-review-diff.sh` refreshes the authoritative base and, when task meta records `pr=`, always fetches and compares against `refs/pull//head` by default (recorded `pr_head=` is only an offline fallback) before falling back to the local branch with a warning. Where a no-mistakes pipeline stores evidence in the repo, it publishes that PR-viewable validation evidence to an orphan evidence branch that shares no history with code branches, so it never enters the crew branch or the default branch. From a7ca54914e6202b06cf776267b8b71c1763552aa Mon Sep 17 00:00:00 2001 From: PP <121104417+BohnBawerick@users.noreply.github.com> Date: Sat, 22 Aug 2026 11:25:20 +0800 Subject: [PATCH 07/24] no-mistakes(document): document hardened quality contract in architecture and scripts inventory --- docs/architecture.md | 4 ++-- docs/scripts.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 8f4f018e167..b8b8b56de79 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -249,9 +249,9 @@ The `data/secondmates.md` line contract is owned by the [`secondmate-provisionin `no-mistakes` tasks run the full validation pipeline, `direct-PR` tasks open PRs without that pipeline, and `local-only` tasks stay local until firstmate performs an approved fast-forward merge. Each task's mode and `yolo` posture are firstmate's decision at intake and are passed explicitly to `bin/fm-brief.sh`, `bin/fm-spawn.sh`, and `bin/fm-promote.sh`, which refuse a ship task that does not carry them. -A ship brief records its mode as a fixed machine-readable line and the spawn refuses to launch on a different one, so the worker's instructions and the recorded task delivery cannot diverge. +A ship brief records its mode as a fixed machine-readable line, and a hardened ship brief records its quality posture as a sibling line; the spawn refuses to launch on a value that disagrees with either, so the worker's instructions and the recorded task contract cannot diverge. `data/projects.md` records each project's standing posture, its optional `+hardened` quality posture, and its optional `+yolo` flag as the captain's default and as context for that decision, including the conditional `no-mistakes-prod-only` policy, which `+hardened` may not ride; a ship spawn or a promotion that drops below either registered posture prints a deviation notice and continues. -`bin/fm-project-mode.sh` remains the one registry parser for the mechanical consumers that have no task in hand: fleet sync's `local-only` skip and home seeding's refusal and no-mistakes initialization. +`bin/fm-project-mode.sh` remains the one registry parser, both for the standing-posture notices above and for the mechanical consumers that have no task in hand: fleet sync's `local-only` skip and home seeding's refusal and no-mistakes initialization. When a selected delivery path calls for a diff, `bin/fm-review-diff.sh` refreshes the authoritative base and, when task meta records `pr=`, always fetches and compares against `refs/pull//head` by default (recorded `pr_head=` is only an offline fallback) before falling back to the local branch with a warning. Where a no-mistakes pipeline stores evidence in the repo, it publishes that PR-viewable validation evidence to an orphan evidence branch that shares no history with code branches, so it never enters the crew branch or the default branch. This repo uses that setting, and its own `.no-mistakes/` directory remains local state that stays gitignored and is rejected by CI if tracked; [`configuration.md`](configuration.md) owns the setting. diff --git a/docs/scripts.md b/docs/scripts.md index 11e06955f26..5467dec0400 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -62,7 +62,7 @@ The shared no-mistakes gate refusal for fleet lifecycle entrypoints is summarize | `backends/orca.sh` | Experimental Orca backend adapter owning both worktree and terminal | | `backends/cmux.sh` | Experimental cmux session-provider adapter | | `fm-config-push.sh` | Push declared inherited local material to live local or remote secondmates and send the placement-specific config reread when changed | -| `fm-project-mode.sh` | Resolve a project's registered delivery posture from `data/projects.md` for fleet sync and home seeding | +| `fm-project-mode.sh` | Resolve a project's registered delivery and quality postures from `data/projects.md` | | `fm-merge-local.sh` | Fast-forward a `local-only` project or Firstmate's own repository local default branch after approval | | `fm-review-diff.sh` | Review a crewmate branch or resolved PR head against the authoritative base | | `fm-marker-lib.sh` | Compatibility entry point for the from-firstmate carrier owned by `fm-operational-input.sh` | From bb124a1bf72a2fd803ab91c9ce2c633f5d95b716 Mon Sep 17 00:00:00 2001 From: PP <121104417+BohnBawerick@users.noreply.github.com> Date: Sat, 22 Aug 2026 12:55:54 +0800 Subject: [PATCH 08/24] feat: revise D2 quality receipts and add a wall-clock bound The stage 0a pilot showed the receipt cannot express real findings and that a missing head_sha makes a drifted base report not-applicable and exit 0. This revises unpublished schema v1 in place: require head_sha, duration_ms, engine, threshold, and a stable finding id; replace survivors[] with per-phase findings[]; and make verify one envelope with phases[]. bounds.budget_minutes is the missing wall-clock bound. --- bin/fm-quality-receipt.sh | 366 +++++++++++++++++ bin/fm-test-run.sh | 5 +- docs/architecture.md | 1 + docs/documentation-audiences.json | 4 + docs/quality-gate.md | 150 +++++++ docs/quality-receipt.schema.json | 210 ++++++++++ docs/scripts.md | 1 + tests/fm-quality-receipt.test.sh | 661 ++++++++++++++++++++++++++++++ 8 files changed, 1397 insertions(+), 1 deletion(-) create mode 100755 bin/fm-quality-receipt.sh create mode 100644 docs/quality-gate.md create mode 100644 docs/quality-receipt.schema.json create mode 100755 tests/fm-quality-receipt.test.sh diff --git a/bin/fm-quality-receipt.sh b/bin/fm-quality-receipt.sh new file mode 100755 index 00000000000..e5cdf33d34d --- /dev/null +++ b/bin/fm-quality-receipt.sh @@ -0,0 +1,366 @@ +#!/usr/bin/env bash +# fm-quality-receipt.sh - validate a quality-gate receipt against the D2 schema. +# +# The committed schema at docs/quality-receipt.schema.json is the owner of the +# JSON shape. This script is the check. docs/quality-gate.md owns the rationale, +# the D1 bounds including budget_minutes, and the verify-envelope decision. +# +# Usage: +# fm-quality-receipt.sh validate [--check-head ] [|-] +# fm-quality-receipt.sh schema +# fm-quality-receipt.sh -h | --help +# +# validate reads one JSON document from or stdin. +# Exit 0 on a valid receipt, 1 on an invalid one, 2 on usage or tool errors. +# +# Post-schema rules, because JSON Schema cannot state them: +# - finding ids are unique inside each findings array +# - each verify child's base_sha and head_sha equal the envelope's +# --check-head then requires head_sha to resolve to that tree's HEAD. +# FM_QUALITY_RECEIPT_SCHEMA overrides the schema path (test seam). +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +SCHEMA="${FM_QUALITY_RECEIPT_SCHEMA:-$FM_ROOT/docs/quality-receipt.schema.json}" +SELF="$SCRIPT_DIR/fm-quality-receipt.sh" + +fm_quality_receipt_usage() { + sed -n '2,21{s/^# \{0,1\}//;p;}' "$SELF" +} + +if ! command -v python3 >/dev/null 2>&1; then + printf 'fm-quality-receipt: python3 is required\n' >&2 + exit 2 +fi + +CMD="" +CHECK_HEAD="" +FILE="" +while [ "$#" -gt 0 ]; do + case "$1" in + -h|--help) + fm_quality_receipt_usage + exit 0 + ;; + schema) + [ -z "$CMD" ] || { + printf 'fm-quality-receipt: unexpected extra command %s\n' "$1" >&2 + exit 2 + } + CMD=schema + shift + ;; + validate) + [ -z "$CMD" ] || { + printf 'fm-quality-receipt: unexpected extra command %s\n' "$1" >&2 + exit 2 + } + CMD=validate + shift + ;; + --check-head) + [ "$#" -ge 2 ] || { + printf 'fm-quality-receipt: --check-head requires a git dir\n' >&2 + exit 2 + } + CHECK_HEAD=$2 + shift 2 + ;; + --) + shift + break + ;; + -) + [ -z "$FILE" ] || { + printf 'fm-quality-receipt: unexpected extra argument %s\n' "$1" >&2 + exit 2 + } + FILE=- + shift + ;; + -*) + printf 'fm-quality-receipt: unknown option %s\n' "$1" >&2 + exit 2 + ;; + *) + [ -z "$FILE" ] || { + printf 'fm-quality-receipt: unexpected extra argument %s\n' "$1" >&2 + exit 2 + } + FILE=$1 + shift + ;; + esac +done + +[ -z "$CMD" ] && { + fm_quality_receipt_usage >&2 + exit 2 +} + +if [ "$CMD" = schema ]; then + [ -z "$CHECK_HEAD" ] && [ -z "$FILE" ] || { + printf 'fm-quality-receipt: schema takes no extra arguments\n' >&2 + exit 2 + } + [ -f "$SCHEMA" ] || { + printf 'fm-quality-receipt: schema file missing: %s\n' "$SCHEMA" >&2 + exit 2 + } + cat "$SCHEMA" + exit 0 +fi + +[ "$CMD" = validate ] || { + printf 'fm-quality-receipt: unknown command %s\n' "$CMD" >&2 + exit 2 +} + +[ -f "$SCHEMA" ] || { + printf 'fm-quality-receipt: schema file missing: %s\n' "$SCHEMA" >&2 + exit 2 +} + +[ -n "$FILE" ] || FILE=- + +exec python3 - "$SCHEMA" "$CHECK_HEAD" "$FILE" <<'PY' +from __future__ import annotations + +import json +import re +import subprocess +import sys + + +class SchemaError(Exception): + def __init__(self, path: str, message: str) -> None: + super().__init__(f"{path}: {message}") + self.path = path + self.message = message + + +def is_int(value: object) -> bool: + return isinstance(value, int) and not isinstance(value, bool) + + +def is_number(value: object) -> bool: + return is_int(value) or isinstance(value, float) + + +def unescape(part: str) -> str: + return part.replace("~1", "/").replace("~0", "~") + + +def resolve(ref: str, root: dict) -> dict: + if not ref.startswith("#/"): + raise SchemaError("$", f"unsupported $ref {ref}") + cur: object = root + for part in ref[2:].split("/"): + if not isinstance(cur, dict): + raise SchemaError("$", f"broken $ref {ref}") + key = unescape(part) + if key not in cur: + raise SchemaError("$", f"broken $ref {ref}") + cur = cur[key] + if not isinstance(cur, dict) and not isinstance(cur, bool): + raise SchemaError("$", f"broken $ref {ref}") + return cur # type: ignore[return-value] + + +def matches(instance: object, schema: object, root: dict) -> bool: + try: + validate(instance, schema, root, "$") + except SchemaError: + return False + return True + + +def validate(instance: object, schema: object, root: dict, path: str) -> None: + if schema is True: + return + if schema is False: + raise SchemaError(path, "not allowed") + if not isinstance(schema, dict): + raise SchemaError(path, "invalid schema") + if "$ref" in schema: + validate(instance, resolve(schema["$ref"], root), root, path) + return + if "allOf" in schema: + for sub in schema["allOf"]: + validate(instance, sub, root, path) + if "if" in schema: + if matches(instance, schema["if"], root): + if "then" in schema: + validate(instance, schema["then"], root, path) + elif "else" in schema: + validate(instance, schema["else"], root, path) + if "not" in schema: + if matches(instance, schema["not"], root): + raise SchemaError(path, "matched a forbidden schema") + expected_type = schema.get("type") + if expected_type == "object": + if not isinstance(instance, dict): + raise SchemaError(path, "expected object") + elif expected_type == "array": + if not isinstance(instance, list): + raise SchemaError(path, "expected array") + elif expected_type == "string": + if not isinstance(instance, str): + raise SchemaError(path, "expected string") + elif expected_type == "integer": + if not is_int(instance): + raise SchemaError(path, "expected integer") + elif expected_type == "number": + if not is_number(instance): + raise SchemaError(path, "expected number") + elif expected_type is not None: + raise SchemaError(path, f"unsupported type {expected_type}") + if "const" in schema and instance != schema["const"]: + raise SchemaError(path, f"expected {schema['const']!r}") + if "enum" in schema and instance not in schema["enum"]: + raise SchemaError(path, f"expected one of {schema['enum']!r}") + if "pattern" in schema: + if not isinstance(instance, str) or re.search(schema["pattern"], instance) is None: + raise SchemaError(path, f"expected to match {schema['pattern']}") + if "minLength" in schema: + if not isinstance(instance, str) or len(instance) < schema["minLength"]: + raise SchemaError(path, f"shorter than {schema['minLength']}") + if "minimum" in schema and is_number(instance) and instance < schema["minimum"]: + raise SchemaError(path, f"below minimum {schema['minimum']}") + if isinstance(instance, list): + if "minItems" in schema and len(instance) < schema["minItems"]: + raise SchemaError(path, f"fewer than {schema['minItems']} items") + item_schema = schema.get("items") + if item_schema is not None: + for i, item in enumerate(instance): + validate(item, item_schema, root, f"{path}/{i}") + if isinstance(instance, dict): + if "minProperties" in schema and len(instance) < schema["minProperties"]: + raise SchemaError(path, f"fewer than {schema['minProperties']} properties") + props = schema.get("properties", {}) + required = schema.get("required", []) + for key in required: + if key not in instance: + raise SchemaError(f"{path}/{key}", "required") + additional = schema.get("additionalProperties", True) + for key, value in instance.items(): + child = f"{path}/{key}" + if key in props: + validate(value, props[key], root, child) + elif additional is False: + raise SchemaError(child, "additional property") + elif additional is not True: + validate(value, additional, root, child) + + +def unique_finding_ids(node: object, path: str) -> None: + if isinstance(node, list): + for i, item in enumerate(node): + unique_finding_ids(item, f"{path}/{i}") + return + if not isinstance(node, dict): + return + findings = node.get("findings") + if isinstance(findings, list): + seen: dict[str, int] = {} + for i, item in enumerate(findings): + if not isinstance(item, dict): + continue + finding_id = item.get("id") + if not isinstance(finding_id, str): + continue + if finding_id in seen: + raise SchemaError( + f"{path}/findings/{i}/id", + f"duplicate id {finding_id!r} (also {path}/findings/{seen[finding_id]}/id)", + ) + seen[finding_id] = i + for key, value in node.items(): + unique_finding_ids(value, f"{path}/{key}") + + +def verify_child_shas(receipt: object) -> None: + if not isinstance(receipt, dict): + return + if receipt.get("phase") != "verify": + return + phases = receipt.get("phases") + if not isinstance(phases, list): + return + for i, child in enumerate(phases): + if not isinstance(child, dict): + continue + for field in ("base_sha", "head_sha"): + if child.get(field) != receipt.get(field): + raise SchemaError( + f"$/phases/{i}/{field}", + f"must equal envelope {field}", + ) + + +def git_rev_parse(git_dir: str, *args: str) -> str: + proc = subprocess.run( + ["git", "-C", git_dir, "rev-parse", *args], + check=False, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + if proc.returncode != 0: + detail = proc.stderr.strip() or "git rev-parse failed" + raise SchemaError("$/head_sha", detail) + return proc.stdout.strip() + + +def check_head(receipt: object, git_dir: str) -> None: + if not isinstance(receipt, dict): + raise SchemaError("$", "expected object") + head_sha = receipt.get("head_sha") + if not isinstance(head_sha, str): + raise SchemaError("$/head_sha", "required") + actual = git_rev_parse(git_dir, "HEAD") + try: + resolved = git_rev_parse(git_dir, "--verify", f"{head_sha}^{{commit}}") + except SchemaError as exc: + raise SchemaError("$/head_sha", f"does not resolve in {git_dir}: {exc.message}") from exc + if resolved != actual: + raise SchemaError( + "$/head_sha", + f"{resolved} is not HEAD {actual}", + ) + + +def main() -> int: + schema_path, check_head_dir, source = sys.argv[1], sys.argv[2], sys.argv[3] + try: + with open(schema_path, encoding="utf-8") as handle: + schema = json.load(handle) + except (OSError, json.JSONDecodeError) as exc: + print(f"fm-quality-receipt: cannot read schema: {exc}", file=sys.stderr) + return 2 + try: + if source == "-": + raw = sys.stdin.read() + else: + with open(source, encoding="utf-8") as handle: + raw = handle.read() + receipt = json.loads(raw) + except (OSError, json.JSONDecodeError) as exc: + print(f"fm-quality-receipt: cannot read receipt: {exc}", file=sys.stderr) + return 1 + try: + validate(receipt, schema, schema, "$") + unique_finding_ids(receipt, "$") + verify_child_shas(receipt) + if check_head_dir: + check_head(receipt, check_head_dir) + except SchemaError as exc: + print(f"fm-quality-receipt: {exc}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) +PY diff --git a/bin/fm-test-run.sh b/bin/fm-test-run.sh index d96f31a4e3b..288b61430ad 100755 --- a/bin/fm-test-run.sh +++ b/bin/fm-test-run.sh @@ -374,6 +374,7 @@ family_for_basename() { fm-kimi-harness.test.sh|fm-muse-harness.test.sh|fm-herdr-lab.test.sh|fm-lint.test.sh|\ fm-lint-workflows.test.sh|\ fm-operational-input.test.sh|fm-pi-primary-types.test.sh|\ + fm-quality-receipt.test.sh|\ fm-send-popup-settle.test.sh|fm-send-settle.test.sh|\ fm-subagent-pretool-check.test.sh|\ fm-supervision-instructions.test.sh|fm-task-delivery.test.sh|\ @@ -1203,6 +1204,7 @@ families_for_changed_path() { bin/fm-tmux-lib.sh|bin/fm-marker-lib.sh|bin/fm-operational-input.sh|bin/fm-tasks-axi-lib.sh|\ bin/fm-vendor-auth-probe.sh|\ bin/fm-primary-scope-lib.sh|bin/fm-project-mode.sh|bin/fm-promote.sh|\ + bin/fm-quality-receipt.sh|\ bin/fm-ff-lib.sh|bin/fm-gotmp*|bin/*pretool*) printf '%s\n' pure-contract-unit ;; @@ -1222,7 +1224,8 @@ families_for_changed_path() { printf '%s\n' pure-contract-unit ;; .github/*|.tasks.toml|AGENTS.md|CLAUDE.md|CONTRIBUTING.md|\ - docs/configuration.md|docs/supervision-protocols/*) + docs/configuration.md|docs/supervision-protocols/*|\ + docs/quality-gate.md|docs/quality-receipt.schema.json) printf '%s\n' pure-contract-unit ;; tests/lib.sh|tests/*-helpers.sh) diff --git a/docs/architecture.md b/docs/architecture.md index b8b8b56de79..445402e6047 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -251,6 +251,7 @@ The `data/secondmates.md` line contract is owned by the [`secondmate-provisionin Each task's mode and `yolo` posture are firstmate's decision at intake and are passed explicitly to `bin/fm-brief.sh`, `bin/fm-spawn.sh`, and `bin/fm-promote.sh`, which refuse a ship task that does not carry them. A ship brief records its mode as a fixed machine-readable line, and a hardened ship brief records its quality posture as a sibling line; the spawn refuses to launch on a value that disagrees with either, so the worker's instructions and the recorded task contract cannot diverge. `data/projects.md` records each project's standing posture, its optional `+hardened` quality posture, and its optional `+yolo` flag as the captain's default and as context for that decision, including the conditional `no-mistakes-prod-only` policy, which `+hardened` may not ride; a ship spawn or a promotion that drops below either registered posture prints a deviation notice and continues. +The project-owned quality-gate contract and the receipt a hardened run must emit are owned by [`quality-gate.md`](quality-gate.md). `bin/fm-project-mode.sh` remains the one registry parser, both for the standing-posture notices above and for the mechanical consumers that have no task in hand: fleet sync's `local-only` skip and home seeding's refusal and no-mistakes initialization. When a selected delivery path calls for a diff, `bin/fm-review-diff.sh` refreshes the authoritative base and, when task meta records `pr=`, always fetches and compares against `refs/pull//head` by default (recorded `pr_head=` is only an offline fallback) before falling back to the local branch with a warning. Where a no-mistakes pipeline stores evidence in the repo, it publishes that PR-viewable validation evidence to an orphan evidence branch that shares no history with code branches, so it never enters the crew branch or the default branch. diff --git a/docs/documentation-audiences.json b/docs/documentation-audiences.json index 9597279923e..66cd9b89817 100644 --- a/docs/documentation-audiences.json +++ b/docs/documentation-audiences.json @@ -288,6 +288,10 @@ "path": "docs/orca-backend.md", "audience": "operator-current" }, + { + "path": "docs/quality-gate.md", + "audience": "maintainer-architecture" + }, { "path": "docs/remote-secondmates.md", "audience": "operator-current" diff --git a/docs/quality-gate.md b/docs/quality-gate.md new file mode 100644 index 00000000000..b22f5977e9c --- /dev/null +++ b/docs/quality-gate.md @@ -0,0 +1,150 @@ +# Quality gate contract and receipt + +The project-owned quality-gate file and the receipt its commands print. +This is the moved owner for those two contracts from the parked Stage 0 spec, revised after the Stage 0a pilot on quota-axi. + +This page describes a capability. +Firstmate itself is not a project the bar is applied to, and this repository does not ship a `.quality-gate.yaml`. + +`bin/fm-quality.sh`, the loop controller, is not in this revision. +It consumes the receipt schema defined here, so the schema had to land first. + +## D1. `.quality-gate.yaml` + +Lives at the root of a hardened project, committed. +The name is vendor-neutral on purpose. +Nothing in the file mentions firstmate, so the file and its CI job survive if firstmate is never used on that repo again. + +```yaml +# .quality-gate.yaml +version: 1 + +# The one command CI runs. Exit 0 or 1. Prints one D2 verify receipt on stdout. +verify: "make quality" + +# The ordinary test suite. The loop runs this after every round's edits +# and reverts the round if it goes red. +test: "pnpm test" + +# The two phases, for the pre-flight loop only. CI does not read these. +clean: + command: "pnpm run quality:complexity" + threshold: + crap_max: 15 + +harden: + command: "pnpm run quality:mutation" + threshold: + kill_rate_min: 0.80 + exclude: + - "src/generated/**" + +bounds: + max_iterations: 4 + no_progress_limit: 2 + budget_usd: 8 + budget_minutes: 20 +``` + +The two threshold numbers in that example are the original design values. +They are captain decisions, not part of this revision, and this page does not change them. + +Field rules: + +- `version` is required and is refused if unknown, so a future schema change fails loudly rather than being half-read. +- `verify` is required. Everything else is optional, and a missing phase means that phase reports `not-applicable`. +- Thresholds are per project. There is no universal default, and `fm-quality.sh` must not invent one. +- `bounds` has the defaults listed above, applied when the key is absent. +- Commands run through the platform shell from the repo root, the same convention no-mistakes uses for `commands.*`. + +### `bounds.budget_minutes` + +The parked spec gave `max_iterations`, `no_progress_limit`, and `budget_usd` only. +Stage 0a showed the cost that actually decides affordability is measurement wall clock, and it is spent before any harness call exists to enforce spend against. +A two-hour measurement costs nothing in tokens. + +`budget_minutes` is the missing bound. +The default of 20 is the Stage 0a-affordable window: four vitest-runner measurements on the diffs that were timed. +A project whose engine is slower must set a higher number rather than overrunning a bound that was never written down. +`budget_usd` stays, because the agent-turn spend is a different resource and is still unmeasured. + +## D2. The receipt + +Every phase command prints one JSON object matching [`quality-receipt.schema.json`](quality-receipt.schema.json). +`bin/fm-quality-receipt.sh` is the check, and `bin/fm-quality-receipt.sh schema` reprints that file. + +`schema_version` stays `1`. +This revises unpublished v1 in place. +No production receipts exist yet. +The Stage 0a pilot objects that passed the old checker are not valid against this revision, and that is intentional. + +What breaks, and why: + +- `head_sha` is now required. + Without it a drifted `base_sha` reports `not-applicable` and exits 0, which is the silent miss the design called easiest to get wrong and hardest to notice. + A docs-only change still has a distinct head and base. + A receipt that cannot show both cannot tell those apart from an anchor that drifted onto `HEAD`. +- `survivors[]` is now `findings[]`. + A complexity offender is not a survivor, has no mutant, and is not `killable`. + The required classification enum was forcing a lie. +- Each finding has a stable `id`. + `file` plus `line` collide. + The pilot saw two distinct surviving mutants on one line, and three on another. + "No progress" has to mean the same findings, not the same count. +- `detail` replaces `mutant`, because the field is not mutation-specific. +- `engine` (name and version) and `threshold` (the numbers this outcome was judged against) are required on `clean` and `harden`. + The same code scored 2 to 37 points apart on two runners. + A receipt with no engine identity is not comparable to any other receipt. +- `duration_ms` is a required top-level integer. + Wall clock is the resource the bound is for, so it is not an optional key inside `metrics`. +- `phase: "verify"` is no longer a flat object with mixed findings. + +`outcome` is still the one field a caller reads to decide anything. +`metrics` is still an open map of numbers, because a Python project and a TypeScript project will not report the same keys. + +### Classification + +`clean` findings use `over-threshold`. +`harden` findings use `killable`, `equivalent`, `unreachable`, `unsupported`, or `defect`. +A clean finding classified `killable` is invalid, which is the point of splitting the vocabulary. + +Harden `id` values come from the engine's own stable mutant id when it has one. +Clean `id` values are a per-function identity the phase command controls, typically `file:line:name`. +Ids are unique inside one `findings` array. +They are not unique across the two phases of a verify receipt, because the two engines do not share an id space. + +### Verify emits one object, with `phases[]` + +`verify` prints one envelope object, not two objects, and not a flattened mix. + +One command is still the CI contract. +It exits 0 or 1, prints one JSON document on stdout, and exposes one `outcome` to branch on. +Two raw objects would preserve per-phase mapping and lose that single outcome. +One flat object with mixed `findings[]` preserves the single outcome and loses the mapping. +That is what the pilot had to do, prefixing metric keys by hand and filing complexity rows next to mutants under the same `killable` label. + +The envelope therefore carries `phase: "verify"`, the folded `outcome`, the same `base_sha` and `head_sha`, the wall-clock `duration_ms` of the wrapper, and a `phases[]` array of complete `clean` and `harden` receipts. +It does not carry `findings`, `engine`, or `threshold` of its own: those belong on the child that produced them. + +Phase commands still emit one `clean` or `harden` object each. +Only the verify wrapper builds the envelope. +Each child's `base_sha` and `head_sha` must equal the envelope's, so a wrapper cannot glue receipts from two trees. + +The folded `outcome` is a D4 rule, not a schema constraint. +`blocked` outranks `defect-found`, which outranks `exhausted` and `stuck`, which outrank `pass`. +`not-applicable` is the envelope outcome only when every child is `not-applicable`. + +## Checking a receipt + +```sh +bin/fm-quality-receipt.sh validate +bin/fm-quality-receipt.sh validate --check-head +bin/fm-quality-receipt.sh schema +``` + +`--check-head` resolves `head_sha` in that tree and requires it to be that tree's `HEAD`. +A receipt that stuffed a constant in `head_sha` fails as soon as `HEAD` moves. +The Stage 0a fail-open, a `not-applicable` object with no `head_sha` at all, fails even without that flag. + +`FM_QUALITY_RECEIPT_SCHEMA` may point the validator at a different schema file. +That seam exists so tests can prove the committed schema is the owner rather than a list of constants inside the script. diff --git a/docs/quality-receipt.schema.json b/docs/quality-receipt.schema.json new file mode 100644 index 00000000000..70c0129bd13 --- /dev/null +++ b/docs/quality-receipt.schema.json @@ -0,0 +1,210 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "firstmate quality receipt", + "type": "object", + "required": [ + "schema_version", + "phase", + "outcome", + "base_sha", + "head_sha", + "duration_ms" + ], + "additionalProperties": false, + "properties": { + "schema_version": { "const": 1 }, + "phase": { "enum": ["clean", "harden", "verify"] }, + "outcome": { "$ref": "#/$defs/outcome" }, + "base_sha": { "$ref": "#/$defs/sha" }, + "head_sha": { "$ref": "#/$defs/sha" }, + "duration_ms": { "type": "integer", "minimum": 0 }, + "engine": { "$ref": "#/$defs/engine" }, + "threshold": { "$ref": "#/$defs/threshold" }, + "metrics": { "$ref": "#/$defs/metrics" }, + "findings": { + "type": "array", + "items": { "$ref": "#/$defs/finding" } + }, + "phases": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/phase_result" } + }, + "exclusions": { + "type": "array", + "items": { "type": "string" } + }, + "notes": { "type": "string" } + }, + "allOf": [ + { + "if": { + "properties": { "phase": { "const": "verify" } }, + "required": ["phase"] + }, + "then": { + "required": ["phases"], + "properties": { + "phases": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/phase_result" } + }, + "findings": false, + "engine": false, + "threshold": false + } + }, + "else": { + "$ref": "#/$defs/phase_result" + } + } + ], + "$defs": { + "sha": { + "type": "string", + "pattern": "^[0-9a-f]{7,40}$" + }, + "outcome": { + "enum": [ + "pass", + "blocked", + "not-applicable", + "exhausted", + "stuck", + "defect-found" + ] + }, + "engine": { + "type": "object", + "required": ["name", "version"], + "additionalProperties": false, + "properties": { + "name": { "type": "string", "minLength": 1 }, + "version": { "type": "string", "minLength": 1 } + } + }, + "threshold": { + "type": "object", + "minProperties": 1, + "additionalProperties": { "type": "number" } + }, + "metrics": { + "type": "object", + "additionalProperties": { "type": "number" } + }, + "finding": { + "type": "object", + "required": ["id", "file", "classification"], + "additionalProperties": false, + "properties": { + "id": { "type": "string", "minLength": 1 }, + "file": { "type": "string", "minLength": 1 }, + "line": { "type": "integer", "minimum": 1 }, + "detail": { "type": "string" }, + "classification": { + "enum": [ + "killable", + "equivalent", + "unreachable", + "unsupported", + "defect", + "over-threshold" + ] + }, + "note": { "type": "string" } + } + }, + "phase_result": { + "type": "object", + "required": [ + "schema_version", + "phase", + "outcome", + "base_sha", + "head_sha", + "duration_ms", + "engine", + "threshold", + "findings" + ], + "additionalProperties": false, + "properties": { + "schema_version": { "const": 1 }, + "phase": { "enum": ["clean", "harden"] }, + "outcome": { "$ref": "#/$defs/outcome" }, + "base_sha": { "$ref": "#/$defs/sha" }, + "head_sha": { "$ref": "#/$defs/sha" }, + "duration_ms": { "type": "integer", "minimum": 0 }, + "engine": { "$ref": "#/$defs/engine" }, + "threshold": { "$ref": "#/$defs/threshold" }, + "metrics": { "$ref": "#/$defs/metrics" }, + "findings": { + "type": "array", + "items": { "$ref": "#/$defs/finding" } + }, + "exclusions": { + "type": "array", + "items": { "type": "string" } + }, + "notes": { "type": "string" } + }, + "allOf": [ + { + "if": { + "properties": { "phase": { "const": "clean" } }, + "required": ["phase"] + }, + "then": { + "properties": { + "findings": { + "type": "array", + "items": { + "allOf": [ + { "$ref": "#/$defs/finding" }, + { + "properties": { + "classification": { "const": "over-threshold" } + } + } + ] + } + } + } + } + }, + { + "if": { + "properties": { "phase": { "const": "harden" } }, + "required": ["phase"] + }, + "then": { + "properties": { + "findings": { + "type": "array", + "items": { + "allOf": [ + { "$ref": "#/$defs/finding" }, + { + "properties": { + "classification": { + "enum": [ + "killable", + "equivalent", + "unreachable", + "unsupported", + "defect" + ] + } + } + } + ] + } + } + } + } + } + ] + } + } +} diff --git a/docs/scripts.md b/docs/scripts.md index 5467dec0400..8cf4fde5e63 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -63,6 +63,7 @@ The shared no-mistakes gate refusal for fleet lifecycle entrypoints is summarize | `backends/cmux.sh` | Experimental cmux session-provider adapter | | `fm-config-push.sh` | Push declared inherited local material to live local or remote secondmates and send the placement-specific config reread when changed | | `fm-project-mode.sh` | Resolve a project's registered delivery and quality postures from `data/projects.md` | +| `fm-quality-receipt.sh` | Validate a quality-gate receipt against the D2 schema, or print that schema | | `fm-merge-local.sh` | Fast-forward a `local-only` project or Firstmate's own repository local default branch after approval | | `fm-review-diff.sh` | Review a crewmate branch or resolved PR head against the authoritative base | | `fm-marker-lib.sh` | Compatibility entry point for the from-firstmate carrier owned by `fm-operational-input.sh` | diff --git a/tests/fm-quality-receipt.test.sh b/tests/fm-quality-receipt.test.sh new file mode 100755 index 00000000000..a939b989e97 --- /dev/null +++ b/tests/fm-quality-receipt.test.sh @@ -0,0 +1,661 @@ +#!/usr/bin/env bash +# Behavior tests for bin/fm-quality-receipt.sh and the D2 receipt schema. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +TMP_ROOT=$(fm_test_tmproot fm-quality-receipt) +RECEIPT="$ROOT/bin/fm-quality-receipt.sh" + +validate() { + "$RECEIPT" validate "$@" +} + +# Build a JSON receipt from a python literal on stdin. +write_receipt() { + python3 -c 'import json,sys; sys.stdout.write(json.dumps(eval(sys.stdin.read())))' >"$1" \ + || fail "could not write receipt $1" +} + +BASE=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +HEAD=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + +# A valid clean pass. Callers override fields by name. +clean_pass_py() { + cat <