Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/stuck-crewmate-recovery/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The target window's harness is recorded as `harness=` in `state/<id>.meta`.
This procedure covers ordinary `kind=ship` and `kind=scout` direct reports.
Load `secondmate-provisioning` instead for `kind=secondmate` recovery.

For a REMOTE secondmate, `fm-crew-state`'s `unknown`/`worktree gone` and `fm-send`'s `remote send failed`/`delivery unconfirmed` verdicts are unreliable and routinely false-negative; do not conclude the mate is dead or the send failed from those alone, confirm against the actual remote pane first.
For a REMOTE secondmate, `fm-crew-state` and `fm-peek` read the actual remote endpoint over `fm-on.sh`, and `fm-send` reports a delivered-with-pending-confirmation steer as delivered (their headers own the contracts); an `unknown-remote` read or unreachable-host failure means the remote state could not be read, never that the mate is dead or the send failed.
Recover a genuinely stuck remote mate only through `bin/fm-spawn.sh <id> --secondmate`, never raw herdr pane close/kill surgery, which strands the endpoint binding.

Treat the digest's endpoint result as a presence signal, not proof that the task's work or validation run is gone.
Expand Down
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,9 @@ After an autonomous merge, give the captain a one-line full-URL or local-main ou

### Validate

For a no-mistakes ship, trigger validation on the same worker after its implementation commit, using the harness invocation owned by `harness-adapters`.
For a no-mistakes ship, the same worker starts its own validation run immediately after the implementation commit and reports `done:` only with a PR.
The worker appends a nonterminal `working:` line when that run starts, and appends `failed:` or `blocked:` if the run dies mid-pipeline, so firstmate still learns start and failure without a handoff `done:`.
Firstmate does not send a start trigger after the implementation commit.
The task worker that starts a no-mistakes run drives the pipeline and owns every `no-mistakes axi run` and `no-mistakes axi respond` call through the next gate or outcome.
Firstmate never invokes `no-mistakes axi respond` for a crew-owned run.
Once validation starts, prefer routing new requirements to follow-up work rather than expanding the current task, unless a new requirement completely invalidates the work being validated; however, the smallest downstream changes needed to keep already accepted product or engineering behavior correct, add behavioral tests where an executable contract exists, or keep documentation accurate remain within the current task even when they touch files not named at intake, and corrections required to satisfy already accepted intent are not new requirements.
Expand Down
19 changes: 12 additions & 7 deletions bin/fm-brief.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@
# For ship tasks, --mode is REQUIRED and shapes the definition of done. Firstmate
# resolves it per task at intake (AGENTS.md section 7); data/projects.md holds the
# captain's standing posture as context, and this script never reads it:
# no-mistakes implement -> /no-mistakes pipeline -> PR -> configured merge authority
# no-mistakes implement -> worker starts the no-mistakes pipeline (the `no-mistakes axi`
# CLI, not a skill) immediately after the implementation commit -> PR ->
# configured merge authority
# direct-PR implement -> push + open PR via gh-axi (no pipeline) -> configured merge authority
# local-only implement on branch, stop and report "ready in branch" (no push/PR);
# the configured merge authority approves, firstmate merges to local main
Expand Down Expand Up @@ -362,7 +364,7 @@ Delivery contract: mode=direct-PR
This task ships **direct-PR**: you raise the PR yourself, without the no-mistakes pipeline.
The task is complete only when committed on your branch.
When it is implemented and committed, push your branch and open a PR with \`gh-axi\`, then append \`done: PR {url}\` to the status file and stop.
Do NOT run /no-mistakes. The configured merge authority decides whether to merge the PR; firstmate relays the outcome.
Do NOT run the no-mistakes pipeline. The configured merge authority decides whether to merge the PR; firstmate relays the outcome.
EOF
;;
local-only)
Expand All @@ -385,12 +387,13 @@ EOF
IFS= read -r -d '' DOD <<EOF || true
# Definition of done
Delivery contract: mode=no-mistakes
The task is complete only when committed on your branch.
When you believe it is complete, append \`done: {summary}\` to the status file and stop.
Firstmate will then instruct you to run /no-mistakes to validate and ship a PR.
This mode is complete only when the no-mistakes pipeline has shipped a PR whose checks are green.
When implementation is committed on your branch, start the no-mistakes pipeline yourself immediately.
Append \`working: starting no-mistakes validation\` to the status file, then run the \`no-mistakes\` CLI on your \`PATH\`: \`no-mistakes axi run --intent "<...>"\` to start, and \`no-mistakes axi respond\` for each gate.
Do not append \`done:\` until there is a PR.

You drive no-mistakes by responding to its gates, not by implementing fixes.
Follow the guidance no-mistakes itself provides for the mechanics: it loads when you invoke /no-mistakes, and \`no-mistakes axi run --help\` plus the \`help\` lines in each \`axi\` response are authoritative and version-matched to the installed binary.
Follow the guidance no-mistakes itself provides for the mechanics: \`no-mistakes axi run --help\` plus the \`help\` lines in each \`axi\` response are authoritative and version-matched to the installed binary.
When starting no-mistakes, make \`--intent\` preserve all relevant content from this brief's \`# Task\` section plus every later accepted Firstmate requirement, clarification, constraint, exclusion, and supersession, carrying only each requirement's current accepted form; retain direct requirements instead of substituting a diff summary, and exclude generic operational, status, delivery, and other scaffold boilerplate unless it is task-specific.
Do not hand-edit, commit, or fix findings yourself while a run is active - the pipeline applies every fix.

Expand All @@ -400,7 +403,9 @@ Two firstmate-specific rules layer on top of that guidance:
When the decision comes back, feed it to the gate with \`no-mistakes axi respond\` and let the pipeline apply it - do not route the question to "the user" or implement the fix yourself.
- Avoid \`--yes\`: it would silently bypass firstmate's authority check and any required captain escalation.

After /no-mistakes reports CI green (the CI-ready return point - do not wait for it to keep monitoring in the background until merge), append \`done: PR {url} checks green\` and stop. You are finished.
If you cannot start or continue the run, append \`blocked: {the exact error}\` and stop, never \`done:\`.
If the run dies mid-pipeline, append \`failed: {the exact error}\` and stop, never stay silent.
After the run reports CI green (the CI-ready return point - do not wait for it to keep monitoring in the background until merge), append \`done: PR {url} checks green\` and stop. You are finished.
EOF
;;
esac
Expand Down
57 changes: 53 additions & 4 deletions bin/fm-crew-state.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,17 @@
# fixed mapping logic, no heuristics and no LLM. Output is one stable, parseable,
# token-tight line firstmate can read every heartbeat:
#
# state: <working|parked|done|blocked|paused|failed|unknown> · source: <run-step|pane|status-log|none> · <detail>
# state: <working|parked|done|blocked|paused|failed|unknown> · source: <run-step|pane|status-log|remote-endpoint|none> · <detail>
#
# Logic, in order:
# 1. Resolve worktree + backend target + kind from state/<id>.meta.
# 1. Resolve worktree + backend target + kind from state/<id>.meta. A meta
# recording remote_host= is a remote secondmate: its worktree and endpoint
# live on that host, so the local worktree and pane reads are skipped and
# the remote host is asked for the endpoint's recovery-grade state
# (fm-on.sh + fm-remote-secondmate-control.sh state). alive falls through
# to the routed status log; dead/missing report the remote verdict; an
# unreachable or unreadable remote reports unknown-remote, never a false
# gone/dead.
# 2. Matching no-mistakes run for this crew's branch AND current code identity,
# active or terminal (from `axi status`, or the coarse `no-mistakes runs`
# fallback)? Branch name alone is not enough: a historical run on a reused
Expand Down Expand Up @@ -101,10 +108,13 @@ meta_value() { # <key>
WT=$(meta_value worktree)
KIND=$(meta_value kind)
HARNESS=$(meta_value harness)
REMOTE_HOST=$(meta_value remote_host)
[ -n "$KIND" ] || KIND=ship

# A torn-down (or never-created) worktree has no current state to read.
if [ -z "$WT" ] || [ ! -d "$WT" ]; then
# A torn-down (or never-created) worktree has no current state to read. A
# remote secondmate's recorded worktree is a path on ITS host, so the local
# probe proves nothing for it - the remote arm below reads the true source.
if [ -z "$REMOTE_HOST" ] && { [ -z "$WT" ] || [ ! -d "$WT" ]; }; then
emit unknown none "worktree gone (torn down?)"
fi

Expand Down Expand Up @@ -138,6 +148,45 @@ map_log_state() { # <line>
LOG_LINE=$(log_last_line || true)
LOG_VERB=$(status_line_verb "$LOG_LINE")

# --- remote secondmate: the true source is the remote endpoint ---------------
# A remote mate's recorded worktree and backend target live on its own host, so
# the local worktree probe above and the local pane reads below would misreport
# a healthy remote mate as gone or dead. Ask the remote host for the endpoint's
# recovery-grade state over the same fm-on.sh transport fm-send uses, then read
# current activity from the routed status log exactly as for a local
# secondmate (an idle endpoint is healthy for a secondmate either way). An
# unreachable host or unreadable endpoint is reported as unknown-remote -
# explicitly NOT proof of death - so a transport blip never reads as a torn
# down or dead mate; only the remote host's own dead/missing verdict may say
# the endpoint is actually gone.
if [ -n "$REMOTE_HOST" ]; then
if ! REMOTE_STATE=$(FM_HOME="$FM_HOME" "$SCRIPT_DIR/fm-on.sh" "$ID" \
fm-remote-secondmate-control.sh state "$ID" < /dev/null 2>/dev/null); then
REMOTE_STATE=
fi
REMOTE_STATE=$(printf '%s\n' "$REMOTE_STATE" | tail -1)
case "$REMOTE_STATE" in
alive)
if [ -n "$LOG_VERB" ]; then
LOG_STATE=$(map_log_state "$LOG_LINE")
if [ "$LOG_STATE" != unknown ]; then
emit "$LOG_STATE" status-log "$(status_line_note "$LOG_LINE")${SEP}remote endpoint alive on $REMOTE_HOST"
fi
fi
emit unknown remote-endpoint "alive on $REMOTE_HOST (an idle secondmate is healthy)"
;;
dead|missing)
emit unknown remote-endpoint "remote endpoint $REMOTE_STATE on $REMOTE_HOST"
;;
'')
emit unknown remote-endpoint "unknown-remote: $REMOTE_HOST unreachable or endpoint unreadable (not proof of death)"
;;
*)
emit unknown remote-endpoint "unknown-remote: endpoint state '$REMOTE_STATE' on $REMOTE_HOST (not proof of death)"
;;
esac
fi

# pane_readable is consulted ONLY in the no-run fallback below. The run-step path
# stays authoritative regardless of pane liveness - judge by the run-step, not the
# shell - so a finished crew whose endpoint has closed still reports its run-step
Expand Down
23 changes: 22 additions & 1 deletion bin/fm-peek.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
# Usage: fm-peek.sh <target> [lines=40]
# <target> may be an exact task id, a legacy fm-<id> task label resolved
# through this home's state/<id>.meta, or an explicit backend target.
# A selector whose meta records remote_host= is a remote secondmate: its pane
# lives on that host, so the capture routes over fm-on.sh to the host-local
# capture (fm-remote-secondmate-control.sh), clamped to that command's
# 100-line cap. An unreachable host or unreadable endpoint fails loudly naming
# the host; the local backend adapters are never asked to read a remote target.
set -eu

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
Expand All @@ -16,9 +21,25 @@ STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}"
"$SCRIPT_DIR/fm-guard.sh" || true

RAW_TARGET=$1
T=$(fm_backend_resolve_selector "$RAW_TARGET" "$STATE")
N=${2:-40}

REMOTE_META=$(fm_backend_meta_for_selector "$RAW_TARGET" "$STATE" 2>/dev/null || true)
if [ -n "$REMOTE_META" ] && [ -n "$(fm_meta_get "$REMOTE_META" remote_host)" ]; then
REMOTE_ID=${REMOTE_META##*/}
REMOTE_ID=${REMOTE_ID%.meta}
REMOTE_HOST=$(fm_meta_get "$REMOTE_META" remote_host)
case "$N" in ''|*[!0-9]*|0) N=40 ;; esac
[ "$N" -le 100 ] || N=100
if ! FM_HOME="$FM_HOME" "$SCRIPT_DIR/fm-on.sh" "$REMOTE_ID" \
fm-remote-secondmate-control.sh capture "$REMOTE_ID" "$N" < /dev/null; then
echo "error: could not read the remote pane of $REMOTE_ID on $REMOTE_HOST (host unreachable or endpoint unreadable; the mate is not thereby dead)" >&2
exit 1
fi
exit 0
fi

T=$(fm_backend_resolve_selector "$RAW_TARGET" "$STATE")

BACKEND=$(fm_backend_of_selector "$RAW_TARGET" "$T" "$STATE")
EXPECTED_LABEL=$(fm_backend_expected_label_of_selector "$RAW_TARGET" "$STATE")

Expand Down
6 changes: 6 additions & 0 deletions bin/fm-remote-secondmate-control.sh
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,12 @@ cmd_send() {
validate_id "$id"
validate_home "$id"
remote_endpoint_require "$id"
# fm-send's exit status is the delivery verdict the parent home acts on
# (0 = confirmed, 3 = delivered with the submit read-back unconfirmed, other
# nonzero = failed; see bin/fm-send.sh's header). The job worker, entrypoint,
# and ssh all preserve it, so no mapping may happen here: flattening exit 3
# into a generic failure is exactly the false-negative the parent's remote
# send path exists to avoid.
FM_HOME="$TARGET_HOME" FM_ROOT_OVERRIDE="$FM_ROOT" FM_STATE_OVERRIDE="$TARGET_HOME/state" \
"$SCRIPT_DIR/fm-send.sh" "$REMOTE_ENDPOINT_TARGET" "$message"
}
Expand Down
Loading
Loading