fix: harden managed Linux bootstrap

[steipete]

Closes #715.
Closes #816.

Supersedes #742 because GitHub rejected maintainer pushes to the contributor fork despite maintainer edits being enabled. This replacement preserves Andy Ye as co-author and keeps @TurboTheTurtle's changelog credit.

Summary

• Replace managed Linux Code Server root installer scripts with pinned, checksum-verified archives in both CLI and coordinator bootstrap generators.
• Preserve Tailscale's default package contract through its signed APT repository with a checksum-pinned keyring; keep checksum-verified static archives as the explicit pinned mode.
• Keep Go and Worker mode parsing aligned, including case-insensitive pinned selection.
• Remove downloaded archives before copying install roots and disarm temporary-directory traps after successful cleanup.
• Update operator documentation and regression coverage.

Verification

Exact candidate: 21a6f42b27c9cf19bb673a565b946b504b8260cd.

• go vet ./...
• go test -race ./... -count=1 — all changed packages and all other packages passed; the unchanged RunPod rollback timing assertion exceeded its local macOS threshold once by 0.28s, then passed 10/10 isolated (13/13 across retries).
• Worker format, lint, Cloudflare and Node typechecks, 26 files / 789 tests, and dry-run build.
• Documentation command-surface, provider-matrix, 209-file link check, and site build.
• Full-branch autoreview twice, including after the final rebase: no accepted/actionable findings.
• Generated installer SHA-256: a36a324602ca1d5a01b3ca5e020d5fe6dfc05cb16f56d8314e56c70c7195ff6c; neither legacy pipe-to-shell installer URL remains.

Exact live proof

• Disposable Ubuntu 24.04 ARM64 container verified the pinned Code Server ARM64 checksum and executed Code Server 4.126.0.
• Tailscale's pinned keyring checksum verified; signed stable APT repository installed Tailscale 1.98.8.
• A one-use, one-day, ephemeral auth key joined exact hostname crabbox-pr792-21a6f42b-10235, obtained a redacted 100.x address, then logged out.
• Cleanup verified the hostname absent from Tailscale machine inventory, the auth key absent from active keys, the used 1Password item archived, the Docker container absent, and generated installer temp paths absent.

Maintainer compatibility decision

Accepted: the stricter default package mode intentionally supports Debian/Ubuntu codenames and fails closed on unsupported distro, codename, keyring, or downloaded-artifact ownership. The explicit pinned-static mode remains available. No legacy curl-to-shell compatibility fallback is retained.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed July 4, 2026, 2:29 PM ET / 18:29 UTC.

Summary
The PR replaces managed Linux code-server and default Tailscale root installer scripts across the Go CLI and Worker bootstrap generators with checksum-verified code-server archives, a signed Tailscale APT repository path, docs, and regression tests.

Reproducibility: yes. for source reproduction: current main and v0.35.0 still emit the Code and default Tailscale root curl-to-shell installer paths. I did not run a live installer compromise, but source inspection is sufficient to prove the vulnerable bootstrap shape.

Review metrics: 3 noteworthy metrics.

• Changed surface: 11 files, +249/-30. The diff spans both bootstrap generators, tests, docs, and release notes, so parity and upgrade behavior are the review-critical surfaces.
• Bootstrap generators changed: 2 generators. The direct Go CLI path and Worker coordinator path must remain behaviorally aligned for managed Linux leases.
• Exact-head proof comments: 1 live proof refresh. The latest comment materially changes the previous proof assessment by reporting exact-head code-server, Tailscale, tailnet join, and cleanup results.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #715
Summary: This PR is the active candidate fix for the canonical managed Linux bootstrap installer-integrity issue and replaces the earlier closed unmerged implementation PR.

Members:

• canonical: [security] Linux bootstrap executes unpinned installer scripts as root #715 - The PR body uses closing references for the still-open broad managed Linux installer-integrity issue.
• superseded: Pin managed Linux bootstrap installers #742 - The PR body says this replacement supersedes the closed unmerged implementation because maintainer pushes to the contributor fork were unavailable.
• partial_overlap: [security] Linux code-server bootstrap executes an unpinned upstream installer as root #816 - This closed issue tracks the code-server half of the same unpinned root installer problem.
• partial_overlap: [security] Default --tailscale bootstrap executes Tailscale's remote install script as root without pinning #789 - This closed issue tracks the default Tailscale installer half of the same bootstrap-integrity problem.
• partial_overlap: [security] Managed-Linux --code bootstrap executes the remote code-server installer without integrity verification #788 - This closed issue also covers the managed-Linux code-server installer integrity gap.
• 1 more in the report.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

• [P1] Confirm maintainer acceptance of the fail-closed Tailscale package-mode contract before merge.

Risk before merge

• [P1] Default Tailscale package mode now fails closed on unsupported distributions, invalid codenames, keyring checksum changes, and signed-repository setup failures; that is security-positive but can break images the old upstream installer script happened to support.
• [P1] Tailscale-enabled managed Linux leases now depend on the new signed APT repository bootstrap reaching readiness, so repository/keyring or distro-support drift can become a lease availability failure instead of falling back to the old installer script.

Maintainer options:

1. Accept the stricter signed-repository default (recommended)
   Maintainers can land the PR as-is if they want Debian/Ubuntu signed-repository package mode to fail closed instead of preserving the old broad install-script behavior.
2. Add an explicit compatibility escape hatch
   If maintainers need old install-script distro coverage, require a narrow documented opt-in or fallback with tests that keep the secure path as the default.
3. Pause if VM-level proof is mandatory
   If container-plus-tailnet proof is not enough for this bootstrap surface, pause merge until a disposable provider VM can prove the same version, address, and cleanup path.

Next step before merge

• [P1] Human merge review remains for the intentional fail-closed package-mode compatibility and availability tradeoff; I found no narrow automated repair to request.

Security
Cleared: No concrete new security or supply-chain regression was found; the diff removes privileged remote installer-script execution and replaces it with digest or signed-repository verification.

Review details

Best possible solution:

Land the hardening after maintainers accept the stricter fail-closed Tailscale package contract and normal branch protection is green, keeping the Go and Worker bootstrap generators aligned.

Do we have a high-confidence way to reproduce the issue?

Yes for source reproduction: current main and v0.35.0 still emit the Code and default Tailscale root curl-to-shell installer paths. I did not run a live installer compromise, but source inspection is sufficient to prove the vulnerable bootstrap shape.

Is this the best way to solve the issue?

Mostly yes: checksum-verified code-server archives and a signed Tailscale APT repository are the narrow maintainable hardening direction. The remaining choice is whether maintainers accept fail-closed package-mode compatibility behavior as the new default.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 26a72c6db6f4.

Label changes

Label changes:

• add proof: sufficient: Contributor real behavior proof is sufficient. The latest exact-head comment gives enough runtime proof for this PR: generated installer hash, code-server execution, signed-repo Tailscale install, redacted 100.x tailnet join, logout, and cleanup.
• add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🦞 diamond lobster.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The latest exact-head comment gives enough runtime proof for this PR: generated installer hash, code-server execution, signed-repo Tailscale install, redacted 100.x tailnet join, logout, and cleanup.
• remove status: 📣 needs proof: Current PR status label is status: 👀 ready for maintainer look.
• remove rating: 🦪 silver shellfish: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

• P1: This PR addresses a security-sensitive managed Linux bootstrap path and still has maintainer-visible compatibility and availability risk before merge.
• merge-risk: 🚨 compatibility: Replacing the old Tailscale package install script with fail-closed distro/codename/keyring validation can break images or future keyring states that previously installed successfully.
• merge-risk: 🚨 availability: If the new signed repository bootstrap fails at runtime, Tailscale-enabled managed Linux leases can fail readiness.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🦞 diamond lobster.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The latest exact-head comment gives enough runtime proof for this PR: generated installer hash, code-server execution, signed-repo Tailscale install, redacted 100.x tailnet join, logout, and cleanup.
• proof: sufficient: Contributor real behavior proof is sufficient. The latest exact-head comment gives enough runtime proof for this PR: generated installer hash, code-server execution, signed-repo Tailscale install, redacted 100.x tailnet join, logout, and cleanup.

Evidence reviewed

What I checked:

• Repository policy read: Full AGENTS.md was read and applied for Crabbox-specific Go, Worker, docs, provider-neutral, and security/config review guidance. (AGENTS.md:1, 26a72c6db6f4)
• Current main still has code-server installer exposure: Current main still emits the managed Linux code-server curl-to-shell installer when the Code capability is enabled, so the PR is not obsolete. (internal/cli/bootstrap.go:1458, 26a72c6db6f4)
• Current main still has default Tailscale installer exposure: Current main's Worker generator still returns the Tailscale curl-to-shell installer for non-pinned package mode. (worker/src/bootstrap.ts:1501, 26a72c6db6f4)
• Latest release is still affected: The v0.35.0 release commit still contains both root installer-script bootstrap paths, so this hardening has not shipped yet. (internal/cli/bootstrap.go:1458, 192e382e9044)
• PR head adds verified code-server archive install: The PR head downloads the versioned code-server archive, verifies SHA-256, extracts it, restores install-root traversal, links the binary, and cleans the temp directory. (internal/cli/bootstrap.go:1479, 21a6f42b27c9)
• PR head replaces package-mode Tailscale script: The PR head validates distro/codename, checksum-verifies the Tailscale keyring, scopes the signed APT repository with signed-by, and installs the package without piping a script to root. (worker/src/bootstrap.ts:1594, 21a6f42b27c9)

Likely related people:

• steipete: Blame ties the current-main affected bootstrap stanzas to the v0.35.0 release import, repository permission reports admin access, and this PR's commits implement the replacement hardening path. (role: recent area contributor and replacement implementer; confidence: high; commits: 192e382e9044, 4d15c24a7feb, e78dc00f926c; files: internal/cli/bootstrap.go, worker/src/bootstrap.ts, worker/src/tailscale.ts)
• TurboTheTurtle: The PR body says this branch supersedes the earlier hardening PR while preserving Andy Ye as co-author, and the main hardening commit includes that co-author credit. (role: earlier candidate fix contributor; confidence: medium; commits: e78dc00f926c; files: internal/cli/bootstrap.go, worker/src/bootstrap.ts, worker/src/tailscale.ts)
• Vincent Koc: History shows the code lease bridge and Worker code portal were introduced in commits that added the code-server bootstrap surface later hardened here. (role: introduced adjacent code capability; confidence: medium; commits: 30e81c6f17a0, b386b9a7379a; files: internal/cli/bootstrap.go, internal/cli/code.go, worker/src/bootstrap.ts)

What the crustacean ranks mean

• 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
• 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
• 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
• 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
• 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
• 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
• 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (7 earlier review cycles)

• reviewed 2026-07-03T13:57:43.785Z sha 47f7672 :: needs real behavior proof before merge. :: [P3] Fix the Tailscale bootstrap description | [P3] Move release-note text out of the changelog
• reviewed 2026-07-04T08:11:09.911Z sha ed63b03 :: needs real behavior proof before merge. :: [P3] Fix the Tailscale bootstrap description | [P3] Move release-note text out of the changelog
• reviewed 2026-07-04T08:15:19.595Z sha ed63b03 :: needs real behavior proof before merge. :: [P3] Fix the Tailscale bootstrap description | [P3] Move release-note text out of the changelog
• reviewed 2026-07-04T09:34:47.958Z sha 36cb72d :: needs real behavior proof before merge. :: [P3] Move release-note text out of the changelog
• reviewed 2026-07-04T09:40:05.115Z sha 36cb72d :: needs real behavior proof before merge. :: [P3] Move release-note text out of the changelog
• reviewed 2026-07-04T11:32:37.038Z sha 36cb72d :: needs real behavior proof before merge. :: [P3] Move release-note text out of the changelog
• reviewed 2026-07-04T13:18:35.019Z sha 73aac03 :: needs real behavior proof before merge. :: [P3] Move release-note text out of the changelog

[steipete]

Exact-head refresh: 21a6f42b27c9cf19bb673a565b946b504b8260cd is rebased through e1732002 and applies cleanly over the newer unrelated Pages-workflow commit on main.

Live proof on this exact head:

• Generated installer SHA-256: a36a324602ca1d5a01b3ca5e020d5fe6dfc05cb16f56d8314e56c70c7195ff6c; neither legacy pipe-to-shell installer URL is present.

• • Disposable Ubuntu 24.04 ARM64 container verified the pinned code-server ARM64 checksum and executed code-server 4.126.0.
• • Tailscale's pinned keyring checksum verified; signed stable APT repository installed Tailscale 1.98.8.
• • A one-use, one-day, ephemeral auth key joined the exact hostname crabbox-pr792-21a6f42b-10235, obtained a redacted 100.x address, then logged out.
• • Post-cleanup: the hostname is absent from the Tailscale admin machine inventory, the auth key is absent from active keys, the used 1Password item is archived, the Docker container is absent, and generated install temp paths are absent.
    Local verification:

• go vet ./...

• • go test -race ./... -count=1: all changed packages and all other packages passed; the unchanged RunPod rollback timing assertion exceeded its local macOS threshold once by 0.28s, then passed 10/10 isolated here (13/13 across retries). Hosted exact-head CI remains authoritative for the integrated gate.
• • Worker format, lint, Cloudflare and Node typechecks, 26 files / 789 tests, and dry-run build.
• • Documentation command-surface, provider-matrix, 209-file link check, and site build.
• • Autoreview twice, including after the final rebase: no accepted/actionable findings.
    Maintainer acceptance: the stricter default package mode intentionally supports Debian/Ubuntu codenames and fails closed on unsupported distro, codename, keyring, or ownership of the downloaded artifact. The explicit pinned-static mode remains available. The changelog entry is intentionally retained because this is a maintainer-authored replacement PR, with contributor credit preserved for @TurboTheTurtle.

[steipete]

Landed on main as 6d96c86268d56f49bf98412b84ada61c726b7830 from exact PR head 21a6f42b27c9cf19bb673a565b946b504b8260cd.

Closeout proof:

• All 11 hosted checks passed on the exact PR head: Go, Apple VZ, Worker, Scripts, Docs, Release Check, four CodeQL analyses, and aggregate CodeQL.
• • The exact generated bootstrap was live-tested in disposable Ubuntu 24.04 on ARM64: code-server 4.126.0 checksum verified and executed; Tailscale keyring checksum verified and signed-repository package 1.98.8 installed.
• • A one-use ephemeral Tailscale key completed the owned 100.x join/logout path; the exact hostname is absent from admin inventory, the active key is absent, its used 1Password item is archived, and the disposable container/install paths are absent.
• • Local Worker/typecheck/build, docs/site, focused cleanup regression, Go vet, and two autoreviews passed. One unchanged RunPod timing assertion exceeded its local macOS threshold once during the full race run and then passed 10/10 isolated (13/13 total retries); hosted Go passed.
• • [security] Linux bootstrap executes unpinned installer scripts as root #715 is closed by this landing; the narrower [security] Linux code-server bootstrap executes an unpinned upstream installer as root #816 was already closed in favor of the canonical issue.
    Local checkout is synced cleanly to origin/main at the landed commit.