Skip to content

fix(mt#5060): Watch the daemon's whole src/ tree and admit .json modules in the tray watcher - #3717

Merged
edobry merged 2 commits into
mainfrom
task/mt-5060
Sep 11, 2026
Merged

edobry merged 2 commits into
mainfrom
task/mt-5060

Conversation

@minsky-ai

@minsky-ai minsky-ai Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

The tray's backend watcher covered src/cockpit plus discovered packages/*/src. The daemon is bun run src/cli.ts, whose static in-repo closure (measured with a Bun.build recording plugin, 2026-09-11) is 1,194 files — 465 under src/, of which only 143 are under src/cockpit. The other 322 (src/adapters 216, src/commands 47, src/mcp 39, src/utils 10, src/domain 3, src/generated 2, …) were unwatched. Independently, is_backend_module_file was .ts/.mts/.cts-only, so the two generated .json imports (src/generated/interceptor-catalog.json, completion-manifest.json) were rejected even where a root covered them.

Reproduced from the system's own records rather than by touching the principal's daemon: the daemon boot log (~/.local/state/minsky/logs/cockpit-daemon*.log) against the local main reflog and each pull's file set. On 2026-09-10, 6 of 6 pulls that touched a watched root restarted the daemon within 7s; 0 of 4 pulls that touched closure files only outside the watched roots did. The daemon booted at 07:30:53Z and served that tree until 00:18:38Z the next day — through a src/domain/calibration change at 08:27 and two interceptor-catalog.json changes at 15:14 and 20:39 — until an unrelated packages/domain merge restarted it. Full table in the task spec's ## Planning finding — 2026-09-11.

mt#4230's docblock stated the closure as src/** plus packages/** and its fix widened only the packages/** half.

Key changes

  • cockpit-tray/src-tauri/src/watcher_backend.rs
    • New daemon_source_tree(repo_root) → src/. cockpit_backend_roots starts from it instead of src/cockpit. A tree root is a superset of the closure by construction — a relative import from an entry under src/ resolves within src/ or (via @minsky/*) within packages/*/src — and cannot drift as new src/ subdirectories appear, the same stance mt#4230 took with package discovery. Rejected alternatives and the accepted residuals (over-restart on ~89 non-closure src/ modules, measured at zero occurrences in the 24h sample; package.json; the shared web/lib/entity-codec.ts) are recorded in the task spec.
    • is_backend_module_file admits .json. Under src/ outside the excluded dirs the only .json files are the two generated artifacts, both imported at boot.
    • cockpit_backend_src (src/cockpit) stays as the presence gate for cockpit_backend_root only.
    • The web exclusion still applies under the wider root because BACKEND_WALK_EXCLUDES matches path components anywhere in the root-relative path — cockpit/web/… is excluded exactly as web/… was.
    • The adoption-time staleness check and the (src @ …) uptime hint read the same root set and filter, so they gain the coverage with no second edit site.
  • cockpit-tray/src-tauri/src/supervisor.rs — wiring comment only.
  • Prose consumers of the watched-set description: src/cockpit/CLAUDE.md, cockpit-tray/README.md, cockpit-tray/CLAUDE.md, docs/architecture/cockpit.md, .minsky/skills/cockpit-tray-dev/content.md (+ its compiled .claude/skills/cockpit-tray-dev/SKILL.md).

Testing

Spec ATs use the task's own numbering.

Execution evidence:

  • AT2 — unit tests over the root set and the relevance filter. Revised: backend_roots_include_the_daemons_real_import_closure (asserts src, asserts src/cockpit is NOT listed beside it), backend_roots_skip_a_package_with_no_src_dir, backend_roots_degrade_when_packages_are_absent (both now expect vec![src]), is_relevant_backend_change_filters_web_test_and_nonts (adds cockpit/server.ts, adapters/shared/commands/calibration.ts, generated/interceptor-catalog.json, generated/completion-manifest.json as relevant; cockpit/web/App.tsx, cockpit/web/lib/entity-codec.ts, __fixtures__/sample.json, cockpit/web/package.json as not). New: backend_roots_cover_a_src_subdirectory_nobody_named_in_code, newest_mtime_across_sees_a_generated_json_only_change (with the src/cockpit-only walk as the root-half negative control inside the test).

    $ cargo test --bin cockpit-tray watcher_backend
    running 16 tests
    test watcher_backend::tests::is_relevant_backend_change_filters_web_test_and_nonts ... ok
    test watcher_backend::tests::backend_roots_degrade_when_packages_are_absent ... ok
    test watcher_backend::tests::backend_roots_skip_a_package_with_no_src_dir ... ok
    test watcher_backend::tests::backend_roots_cover_a_src_subdirectory_nobody_named_in_code ... ok
    test watcher_backend::tests::backend_roots_discover_a_package_nobody_named_in_code ... ok
    test watcher_backend::tests::backend_roots_include_the_daemons_real_import_closure ... ok
    test watcher_backend::tests::newest_mtime_across_sees_a_generated_json_only_change ... ok
    test watcher_backend::tests::newest_mtime_across_sees_a_packages_only_change ... ok
    test watcher_backend::tests::newest_mtime_across_applies_the_relevance_filter_in_every_root ... ok
    test watcher_backend::tests::newest_backend_mtime_ignores_web_test_and_excluded ... ok
    test watcher_backend::tests::newest_mtime_across_ignores_test_adjacent_dirs_under_packages ... ok
    (+5 parse_turn_active_body tests) ... ok
    test result: ok. 16 passed; 0 failed; 0 ignored; 0 measured; 134 filtered out; finished in 0.01s
    
  • AT3 — the crate builds and its full test suite passes; cargo fmt --check clean on both touched files; bun run format:check clean.

    $ cargo test --bin cockpit-tray
    test result: ok. 149 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.02s
    
  • AT1 — post-merge confirmation on the rebuilt tray (this session, after merge + rebuild + reinstall): a main fast-forward confined to src/generated/** or another closure file outside src/cockpit boots the daemon within the debounce window (new PersistenceService initialized successfully line in cockpit-daemon.log; /api/health commit advances). The pre-fix negative control for this AT is the planning measurement above: 0 of 4 such pulls booted the daemon on 2026-09-10.

  • AT4 — build/install-class deliverable (deployed < usable): rebuild + reinstall the tray after merge and confirm the running app carries the change. Done in this session after merge; the merge alone does not deliver it.

Negative control: reverted the two production predicates in place (cockpit_backend_roots back to vec![cockpit_backend_src(repo_root)], EXTS back to [".ts", ".mts", ".cts"]) with the revised tests kept, and ran the same module.

$ cargo test --bin cockpit-tray watcher_backend   # pre-fix predicates, post-fix tests
test watcher_backend::tests::is_relevant_backend_change_filters_web_test_and_nonts ... FAILED
test watcher_backend::tests::backend_roots_degrade_when_packages_are_absent ... FAILED
test watcher_backend::tests::backend_roots_cover_a_src_subdirectory_nobody_named_in_code ... FAILED
test watcher_backend::tests::backend_roots_skip_a_package_with_no_src_dir ... FAILED
test watcher_backend::tests::newest_mtime_across_sees_a_generated_json_only_change ... FAILED
test watcher_backend::tests::backend_roots_include_the_daemons_real_import_closure ... FAILED
thread '...is_relevant_backend_change_filters_web_test_and_nonts' panicked at src/watcher_backend.rs:697:9:
assertion failed: is_relevant_backend_change(Path::new("generated/interceptor-catalog.json"))
thread '...backend_roots_include_the_daemons_real_import_closure' panicked at src/watcher_backend.rs:485:9:
assertion failed: roots.contains(&repo.join("src"))
test result: FAILED. 10 passed; 6 failed; 0 ignored; 0 measured; 134 filtered out

Each half of the fix has its own failing assertion in the control (the .json rejection pins the module class; the missing src root pins the root set). Both predicates restored afterwards; grep -c "NEGATIVE CONTROL" → 0 before the final run.

Deploy verification: src/cockpit/CLAUDE.md is classified deploy surface by isDeploySurfaceFile (it sits under src/**, which deploy-minsky-mcp.yml deploys on push to main); every other changed file is false. It is a documentation file with no runtime effect, but the predicate is the authority, so after merge this session runs deployment_wait-for-latest(service: "minsky-mcp", notBefore: <mergeDate>, expectCommitSha: <merge sha>) and reads buildIdentity, then probes /health for service: "minsky-mcp". The change that matters — the tray binary — is not deployed by any pipeline; its delivery is AT4's rebuild + reinstall, verified by AT1.

Coordinate

mt#3589 (TODO, watcher reliability INSIDE a watched root) — note for its owner: in the 24h sample every watched-root pull restarted the daemon within 7s (6/6) on the currently-installed tray build.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TPNyYTreM4F7PRbvh4xA5b

The backend watcher's src/ root was `src/cockpit`; the daemon is `bun run
src/cli.ts`, whose static closure has 465 files under src/ of which only 143
are under src/cockpit. The module-class filter was also `.ts`-only, so the two
generated .json imports were dropped even where watched. Measured 2026-09-10:
4 of 4 pulls touching closure files outside the watched roots produced no
restart; 6 of 6 watched-root pulls did. The daemon served a two-generations-old
interceptor catalog for ~16h.

- `cockpit_backend_roots` now starts from `daemon_source_tree` (`src/`), the
  entry point's own tree — a superset of the closure by construction, the same
  stance mt#4230 took with `packages/*/src` discovery.
- `is_backend_module_file` admits `.json`.
- `cockpit_backend_src` stays as the presence gate only.
- Tests revised for the new root; two new tests pin the src/ subtree coverage
  and the generated-json case; the web/ exclusion is asserted under the wider
  root.
- The five prose descriptions of the watched set updated; skill recompiled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPNyYTreM4F7PRbvh4xA5b
@minsky-ai minsky-ai Bot added the authorship/co-authored Co-authored by human and AI agent label Sep 11, 2026
@minsky-reviewer

minsky-reviewer Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

Minsky Reviewer Status

Verdict: APPROVED — no blocking findings
Review: View review
Model: openai/gpt-5 | Tokens: 714K prompt, 10K completion | Duration: 127s
Mode: normal

Commands

  • /review — request a fresh review

@minsky-reviewer minsky-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial review (Chinese-wall)
Reviewer: minsky-reviewer[bot] via openai:gpt-5
Tier: 2


Solid, well-scoped change that widens backend watcher coverage to the daemon’s actual closure and admits generated JSON modules; tests and prose were updated coherently. I found no blocking issues. Non-blocking nits: (1) *.test.json/*.spec.json will still trigger restarts — consider excluding JSON test files like TS test suffixes; (2) Rustdoc comments use JSDoc {@link ...} syntax — use Rust intra-doc links instead; (3) with src/ as a root, the freshness walk cost grows — consider small optimizations/caching if it becomes noticeable; (4) the web exclusion applies anywhere in a root-relative path — confirm that broader policy remains intended. With these addressed or tracked, this PR looks ready to merge.

Findings

  • [NON-BLOCKING] cockpit-tray/src-tauri/src/watcher_backend.rs:285 — .json admitted as module but test-file variants (*.test.json/*.spec.json) are not excluded
    is_backend_module_file adds .json to EXTS (cockpit-tray/src-tauri/src/watcher_backend.rs:285-306), but TEST_EXTS excludes only TS/MTS/CTS test/spec suffixes. If a *.test.json or *.spec.json appears under a watched root (not in an excluded dir), it will spuriously trigger a daemon restart. The PR asserts only two .json under src/ today, but this is a future-footgun. Suggestion: add ".test.json" and ".spec.json" to TEST_EXTS (and consider mirroring any other test file-class suffixes you rely on).
  • [NON-BLOCKING] cockpit-tray/src-tauri/src/watcher_backend.rs:137 — Rustdoc link syntax uses {@link ...} which Rust does not render as intra-doc links
    Several doc comments added in this PR use {@link ...} (e.g., watcher_backend.rs:146-175, :201-216, :247-258). That's JSDoc syntax, not Rust's. Rustdoc will render these literally rather than linking. Prefer Rust intra-doc links like [`daemon_source_tree`] and fully-qualified paths for items to keep docs navigable.
  • [NON-BLOCKING] cockpit-tray/src-tauri/src/watcher_backend.rs:319 — Broadened src/ root increases filesystem walk cost for staleness checks; consider bounding or caching
    newest_backend_mtime now walks the entire src/ tree recursively (excluding components in BACKEND_WALK_EXCLUDES) and newest_backend_mtime_across takes a max across roots. With the widened src/ root, adoption-time checks (supervisor.rs uses this during spawn) and runtime debounce evaluations may traverse significantly more files, especially in large repos. Tests assert behavior, but there is no guard against pathological trees (e.g., large vendored directories not caught by excludes). Non-blocking: consider cheap optimizations (e.g., early-return on encountering a file newer than the tracked daemon start, or memoizing per-root newest mtime across bursts) to keep overhead predictable.
  • [NON-BLOCKING] cockpit-tray/src-tauri/src/watcher_backend.rs:262 — backend_path_is_excluded matches component equal to web; ensure no unintended exclusions from common names
    The exclusion applies when any path component equals "web" (backend_path_is_excluded via is_backend_excluded_dir). Under the widened src/ root, paths like cockpit/web/... are correctly excluded, but any future backend subdir named literally web (outside the cockpit frontend) would also be excluded. This may be intended, but it's now a broader policy decision. Non-blocking: consider scoping the web exclusion to cockpit/web if you ever add non-frontend web/ subtrees elsewhere.

Documentation impact

  • updated-in-pr — Multiple user/developer docs were updated to reflect the widened watcher coverage and .json module class: .claude/skills/cockpit-tray-dev/SKILL.md, .minsky/skills/cockpit-tray-dev/content.md, cockpit-tray/README.md (backend watcher roots and relevance wording), cockpit-tray/CLAUDE.md (wiring comment), docs/architecture/cockpit.md (backend auto-restart description), and src/cockpit/CLAUDE.md (operator guidance). These changes align prose with the new behavior; no additional docs appear to need updates beyond what the PR already touches.
    Affected: .claude/skills/cockpit-tray-dev/SKILL.md, .minsky/skills/cockpit-tray-dev/content.md, cockpit-tray/README.md, cockpit-tray/CLAUDE.md, docs/architecture/cockpit.md, src/cockpit/CLAUDE.md

@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

Cockpit Preview

Preview URL: https://cockpit-preview-production.up.railway.app

Deployed from commit 3ef19ea07.
Preview runs in read-only mode — mutations are disabled.

Updated by Cockpit Preview.

…doc links

Two non-blocking findings addressed in-class: `.test.json` / `.spec.json`
join `TEST_EXTS` alongside the `.json` module class (with assertions), and
the docblocks this PR wrote use [`item`] intra-doc links instead of JSDoc
`{@link}`; `cargo doc` resolves all seven with no warning in this file. The
other two findings (walk cost over ~1k files at spawn; the global `web`
component exclusion) are intended and documented in the docblocks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPNyYTreM4F7PRbvh4xA5b

@minsky-reviewer minsky-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial review (Chinese-wall)
Reviewer: minsky-reviewer[bot] via openai:gpt-5
Tier: 2


Reverification complete: the fix widens the backend watcher to the daemon’s entry tree (src/) and admits .json modules, with comprehensive unit tests and supervisor wiring using the same root set. Web remains excluded under the wider root, and the adoption-time staleness check and uptime hint reuse the same logic. Prose consumers were updated across README/CLAUDE/docs to prevent stale guidance. I find no new defects introduced by the follow-up commit; prior non-blocking notes were addressed. One criterion (live reproduction evidence) rests on out-of-repo logs and is unverifiable here, but does not block merge. Verdict: APPROVE.

Spec verification

Criterion Status Evidence
The live behavior is reproduced or falsified, and recorded either way. Reproduced at planning — ## Planning finding — 2026-09-11. Unverifiable Reproduction evidence relies on out-of-repo daemon logs under ~/.local/state/minsky/logs/cockpit-daemon*.log and local reflog correlation (PR body and spec). Those artifacts are not in-repo; reviewer cannot verify from the diff alone per Out-of-repo references policy.
A merged change confined to any file in the daemon's static import closure under src/** — src/generated/interceptor-catalog.json, src/adapters/**, src/domain/**, src/mcp/**, … — causes the tray-supervised daemon to restart within RESTART_DEBOUNCE, exactly as a src/cockpit or packages/*/src change does today. Met cockpit-tray/src-tauri/src/watcher_backend.rs: cockpit_backend_roots() now includes daemon_source_tree(repo_root) → src/ (lines ~152–181) plus discovered packages/*/src; is_relevant_backend_change() filters root-relative paths with web exclusion. Tests backend_roots_cover_a_src_subdirectory_nobody_named_in_code and newest_mtime_across_sees_a_generated_json_only_change assert coverage.
is_relevant_backend_change accepts the .json module class, so the two generated JSON artifacts the daemon imports are relevant changes. Met watcher_backend.rs: is_backend_module_file() admits .json and extends TEST_EXTS with .test.json/.spec.json; tests in is_relevant_backend_change_filters_web_test_and_nonts assert generated/interceptor-catalog.json and generated/completion-manifest.json are relevant while test/fixture JSON are not.
The adoption-time staleness check (adopt_decision → newest_backend_mtime_across) and the (src @ …) uptime hint gain the same coverage, because they read the same root set and relevance filter — no second edit site (mt#4230's single-list invariant holds). Met cockpit-tray/src-tauri/src/supervisor.rs: in do_spawn(), source mtime recorded via cockpit_backend_root(path).and_then(|r| newest_backend_mtime_across(&cockpit_backend_roots(&r))), reusing the same root-set and filter logic as the watcher. Docstrings and README describe single-list invariant.
The chosen root-set mechanism is recorded with its basis, and explicitly addresses mt#4230's hardcoded-set argument rather than silently adding one more literal. Decided at planning: widen the src/ root from the src/cockpit subdirectory to src/ itself — the entry point's own tree. Met watcher_backend.rs docblocks for daemon_source_tree and cockpit_backend_roots explain the entry-point-tree rationale, discovery posture, dynamic-import coverage, and trade-offs; prose consumers updated (README/CLAUDE.md/docs) echo the basis.
Roots stay derived where derivation is possible — a change adding a new src/ subdirectory the daemon imports must not require editing this list again. Met daemon_source_tree() returns repo_root.join("src"); test backend_roots_cover_a_src_subdirectory_nobody_named_in_code asserts a new src/zzz-future/ module is covered without naming.
The existing src/cockpit and packages/*/src behavior is unchanged; web/** edits still never restart the daemon (mt#2299 acceptance test 6 keeps holding through the web component exclusion, which is evaluated on the root-relative path and so still matches cockpit/web/… under the wider root). Met BACKEND_WALK_EXCLUDES includes "web" and backend_path_is_excluded() matches components anywhere; tests in is_relevant_backend_change_filters_web_test_and_nonts assert cockpit/web/* remains excluded under the widened src/ root.
The five prose descriptions of the watched set are updated to the new coverage (enumerated under ## Scope). Met Updated docs: src/cockpit/CLAUDE.md Operator dev loop (now cites mt#5060 and src/**); cockpit-tray/README.md Backend auto-restart section; cockpit-tray/CLAUDE.md two-layer summary; docs/architecture/cockpit.md Daemon lifecycle and tray; .claude/skills/cockpit-tray-dev/SKILL.md regenerated with updated text.
Does NOT cover — Over-restart on src/ modules outside the daemon's closure is accepted. Met Design documented in daemon_source_tree docblock; implementation watches src/ superset by construction. Debounce bounds cost; no change required.
Does NOT cover — package.json at the repo root stays unwatched to avoid restart-before-install loops. Met is_backend_module_file() only admits .ts/.mts/.cts/.json; repo-root package.json is outside watched roots and not a module class; unchanged by this PR.
Does NOT cover — src/cockpit/web/lib/entity-codec.ts is dual-consumed and remains excluded by the web directory rule. Met BACKEND_WALK_EXCLUDES includes "web"; path-component exclusion still filters cockpit/web/** under the widened src/ root.
Does NOT cover — Dynamic imports are not captured by a static-closure enumeration; watching the entry tree is the chosen superset posture. N/A Addressed by design via the src/ tree-root superset; no separate behavior to verify beyond the daemon_source_tree(repo_root) implementation and its tests.

Documentation impact

  • updated-in-pr — Documentation and operator-facing prose were updated to reflect the widened backend watcher coverage and .json module class: src/cockpit/CLAUDE.md (Operator dev loop cites mt#5060), cockpit-tray/README.md (Backend auto-restart now states src/** + packages/*/src and .json), cockpit-tray/CLAUDE.md, docs/architecture/cockpit.md (Daemon lifecycle and tray), and the compiled .claude/skills/cockpit-tray-dev/SKILL.md. These updates prevent stale guidance about src/cockpit-only coverage.
    Affected: src/cockpit/CLAUDE.md, cockpit-tray/README.md, cockpit-tray/CLAUDE.md, docs/architecture/cockpit.md, .claude/skills/cockpit-tray-dev/SKILL.md

@edobry
edobry merged commit 45461a7 into main Sep 11, 2026
24 checks passed
@edobry
edobry deleted the task/mt-5060 branch September 11, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

authorship/co-authored Co-authored by human and AI agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant