Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/deps-verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,10 +115,14 @@ jobs:
steps:
- uses: actions/checkout@v7

# Isolation is hard-coded, not a caller input: a verification job that
# can see yesterday's tree is not verifying (DODI-00020).
- id: setup
uses: dodi-smart/.github/actions/setup-stack@main
with:
stack: ${{ needs.gate.outputs.stack }}
isolate: true
cache: false
install: ${{ inputs.install }}
build: ${{ inputs.build }}
typecheck: ${{ inputs.typecheck }}
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/pr-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ jobs:
uses: dodi-smart/.github/actions/setup-stack@main
with:
stack: ${{ inputs.stack }}
isolate: false
cache: auto
install: ${{ inputs.install }}
lint: ${{ inputs.lint }}
typecheck: ${{ inputs.typecheck }}
Expand Down Expand Up @@ -119,6 +121,8 @@ jobs:
uses: dodi-smart/.github/actions/setup-stack@main
with:
stack: ${{ inputs.stack }}
isolate: false
cache: auto
install: ${{ inputs.install }}
test: ${{ inputs.test }}
java-version: ${{ inputs.java-version }}
Expand Down Expand Up @@ -155,6 +159,8 @@ jobs:
uses: dodi-smart/.github/actions/setup-stack@main
with:
stack: ${{ inputs.stack }}
isolate: false
cache: auto
install: ${{ inputs.install }}
build: ${{ inputs.build }}
java-version: ${{ inputs.java-version }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ jobs:
stack: ${{ inputs.stack }}
node-version: ${{ inputs.node-version }}
bun-version: ${{ inputs.bun-version }}
cache: false # hosted-only; skip package cache. Never bun-on-GitHub.
install: '' # the release path installs what it needs, below

# semantic-release itself always runs on Node, whatever the repo's stack
Expand Down
3 changes: 3 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ be verified rather than asserted.
| `.github/workflows/pick-runner.yml` | **DODI-00010** — runners are selected by generic capability; public repos and fork PRs always use hosted | Never select by a hardware nickname or a machine name — those are re-registration-unstable and tie every caller to today's fleet. Public repos and fork PRs have no opt-out: a fork PR would run attacker-authored code on our hardware against a cache that persists into the next job. |
| `.github/workflows/pick-runner.yml` | **DODI-00012** — the picker validates its own selector | Falling back to hosted on error is correct, but it makes a selector matching nothing look exactly like a busy fleet. The validation is why that bug cannot hide again. It **warns rather than fails**: a bad selector still runs correctly on hosted, and failing CI over a cost regression would be worse than the bug. |
| `actions/setup-stack/` | **DODI-00015** — stack is an input; there is no template per stack | Never add a per-stack caller template, and never put a product-specific task, module or scheme name in this repo. The previous Gradle template carried one product's task name into every repo told to copy it. |
| `actions/setup-stack/` | **DODI-00020** — package caches persist per runner; GitHub cache is hosted-only and never uses restore-keys | Always-isolating to `RUNNER_TEMP` made named volumes unused and still let `setup-gradle` talk to GitHub's cache, which is how `restore-keys` reimported poison. `isolate: true` and the self-hosted **Linux** ARM64 pool stay on `RUNNER_TEMP` — not plain ARM64, or the self-hosted Mac runs cold for a Pi's reason. Self-hosted X64 uses `~/.bun` / `~/.npm` / `~/.cache/pnpm` / `~/.gradle` / `~/.cargo`, the dirs the runner image mounts volumes for and LRU-caps. A home dir is only worth using if a volume backs it, so `PUB_CACHE` is job-scoped in every mode. Isolation must name every package manager the image persists — pnpm and yarn read their own env vars (`npm_config_store_dir`, `YARN_CACHE_FOLDER`), so `npm_config_cache` alone left an isolated job writing to the shared store. Hosted GitHub cache is one mechanism per stack (`setup-gradle` / `rust-cache` / `flutter-action`), never bun, never `setup-java cache: gradle`, never `restore-keys` on a package store. |
| `.github/workflows/pr-checks.yml` | **DODI-00020** — PR checks use local home on X64 self-hosted; `cache: auto` | All three `setup-stack` calls pass `isolate: false` and `cache: auto`. Auto becomes true only on `github-hosted`. |
| `.github/workflows/deps-verify.yml` | **DODI-00020** — verification isolates and never uses GitHub package cache | Hard-coded `isolate: true` and `cache: false`. Not a caller-facing input: a verification job that can see yesterday's tree is not verifying. |
| all workflows | **DODI-00017** — callers pin a released tag | `v1` moves only after a change runs green on a real repo. Changing or removing an input is breaking: add an alias and warn (as `deps-verify` does for `setup:`), or cut `v2`. |
| `README.md` | **DODI-00018** — this repo is public and self-contained | Never link to the private standards repo. Restate the reasoning instead: a link into a private repo is a 404 to everyone who follows it, which is worse than no link because it reads as an offer. |

Expand Down
27 changes: 22 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ exactly like a busy fleet (DODI-00012).
|---|---|
| `actions/agent-gate` | Decides whether an agent may run. Evaluates `agent:no-touch` first, always |
| `actions/run-agent` | Invokes the agent with the org's tool allowlist and reporting defaults |
| `actions/setup-stack` | Installs a toolchain, isolates its caches, supplies conventional commands |
| `actions/setup-stack` | Installs a toolchain, resolves cache isolation, supplies conventional commands |

## Two things you can rely on

Expand Down Expand Up @@ -123,10 +123,27 @@ There used to be a caller template per stack. It failed the way templates fail:
the Gradle one carried one product's Gradle task name, in a file every other
Gradle repo was told to copy (DODI-00015).

Package caches are pinned to job-scoped directories. Self-hosted runners persist
between jobs, so default caches are shared, and a half-written entry poisons
every later run — which `restore-keys` then faithfully restores, so re-running
does not clear it.
Package caches are **not** always job-scoped. `setup-stack` resolves three
modes from `isolate`, `cache`, `runner.environment`, `runner.os` and
`runner.arch` (DODI-00020):

- `isolate: true` (deps-verify) and the self-hosted **Linux** ARM64 pool pin
caches to `RUNNER_TEMP` and disable GitHub cache — verification must not see
yesterday's tree, and a Pi with 8 GB must not grow bun/Gradle volumes. The
check is Linux ARM64, not ARM64: the self-hosted Mac is ARM64 too, has no
container volumes to grow, and gains nothing from running every job cold.
- Self-hosted X64 uses default home dirs (`~/.bun`, `~/.npm`, `~/.cache/pnpm`,
`~/.gradle`, `~/.cargo`) so per-runner named volumes are actually read.
GitHub cache stays off.
- `~/.pub-cache` is job-scoped in **every** mode. A home dir is only worth
using if a volume backs it, and the runner image mounts none for pub — so on
a persistent runner it would grow in the container layer, outside the
post-job LRU cap that bounds every other store.
- GitHub-hosted uses those same home dirs; `cache: auto` becomes true for
`setup-gradle` / `rust-cache` / `flutter-action` only. Bun is never
uploaded. One mechanism per stack — never *also* `setup-java cache: gradle`,
never `restore-keys` on a package store. That pairing is how a partial
tarball came back on every retry.

## Runners

Expand Down
138 changes: 116 additions & 22 deletions actions/setup-stack/action.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Install a toolchain, and isolate its package cache.
# Install a toolchain, and decide where its package cache lives.
#
# WHY THIS IS ONE ACTION AND NOT ONE TEMPLATE PER STACK
#
Expand All @@ -9,15 +9,40 @@
# Gradle repo was meant to copy, where it produces a build failure whose cause is
# a different product. Stack is a VALUE, so it belongs in an input.
#
# WHY THE CACHE DIRS ARE JOB-SCOPED
# WHY THE CACHE IS NOT ALWAYS JOB-SCOPED
#
# Self-hosted runners persist between jobs, so the default per-user caches are
# shared. A half-written entry poisons every later run — `Fail extracting
# tarball for "next"` — and `restore-keys` faithfully restores the poison, so
# re-running does NOT clear it. That cost real days. Isolation costs a cold
# install and buys determinism, which is the entire point of a verification job.
# This action used to point every toolchain at $RUNNER_TEMP. That made named
# volumes on the X64 self-hosted pool unused, so every job went cold, and it
# still let setup-gradle talk to GitHub's cache — which is how a partial bun
# tarball plus restore-keys reimported poison. Three modes, resolved here:
#
# isolate=true -> RUNNER_TEMP dirs, cache-disabled (deps-verify)
# self-hosted Linux ARM64 -> isolate anyway (Pi 8 GB must not grow volumes)
# self-hosted + X64 -> default home dirs; GitHub cache-disabled
# (the named volume IS the cache)
# github-hosted -> default home dirs; cache=auto becomes true
# (setup-gradle / rust-cache / flutter-action only)
#
# A HOME DIR IS ONLY WORTH USING IF A VOLUME BACKS IT
#
# The runner image mounts one named volume per runner for ~/.bun, ~/.npm,
# ~/.cache/pnpm, ~/.gradle and ~/.cargo, and its post-job hook LRU-caps them.
# PUB_CACHE has no such volume, so on a persistent runner ~/.pub-cache would
# grow in the container's writable layer where nothing caps it. It stays
# pinned to RUNNER_TEMP in every mode until the image mounts and caps it.
#
# The forced-isolate arch check is Linux ARM64, not ARM64. The Pi is Linux
# ARM64; the self-hosted macOS runner is also ARM64 but is not a Pi, has no
# container volumes to grow, and has no reason to run every job cold.
#
# Hosted GitHub cache is ONE mechanism per stack. Never also setup-java
# cache: gradle, never a manual actions/cache of ~/.gradle, never restore-keys
# on a package store. No package STORE is ever uploaded to GitHub — bun's is
# 420 MB to restore against an 8s cold install, and restore-keys on a store is
# how poison reimports. (setup-bun still caches the bun EXECUTABLE, which is a
# small exact-keyed binary, not a store; that one is fine.)
name: Setup stack
description: Install the toolchain for a stack and isolate its package cache.
description: Install the toolchain for a stack and resolve its package-cache mode.

inputs:
stack:
Expand Down Expand Up @@ -47,9 +72,19 @@ inputs:
xcode-version:
description: 'For stack=xcode. Empty takes the runner default.'
default: ''
isolate:
description: >-
Pin package caches to RUNNER_TEMP and disable GitHub cache. Forced on
for the self-hosted Linux ARM64 pool. Default false: X64 self-hosted
uses home dirs (named volumes); hosted uses home dirs plus toolchain
cache actions. PUB_CACHE is job-scoped regardless — no volume backs it.
default: 'false'
cache:
description: 'Restore and save the package cache'
default: 'true'
description: >-
auto | true | false. auto becomes true only on github-hosted, and is
ignored when isolated or on self-hosted (those never use GitHub cache).
Applies to setup-gradle / rust-cache / flutter-action only — never bun.
default: 'auto'

# Command overrides. The sentinel "@stack" means "use this stack's
# conventional command"; an explicit empty string means "this repo has no such
Expand Down Expand Up @@ -83,49 +118,108 @@ outputs:
runs:
using: composite
steps:
# Point every toolchain at a directory this job owns. See the header.
- id: isolate
- id: resolve
shell: bash
env:
STACK: ${{ inputs.stack }}
ISOLATE: ${{ inputs.isolate }}
CACHE: ${{ inputs.cache }}
RUNNER_ENVIRONMENT: ${{ runner.environment }}
RUNNER_ARCH: ${{ runner.arch }}
RUNNER_OS: ${{ runner.os }}
run: |
set -euo pipefail
case "$STACK" in
bun|node|gradle|android|flutter|rust|xcode|none) : ;;
*) echo "::error::unknown stack '$STACK' (expected bun|node|gradle|android|flutter|rust|xcode|none)"; exit 1 ;;
esac
case "$CACHE" in
auto|true|false) : ;;
*) echo "::error::cache must be auto, true, or false (got '$CACHE')"; exit 1 ;;
esac

# isolate=true, or the self-hosted Linux ARM64 pool — the Pis, whose
# 8 GB must not grow bun/Gradle volumes. Deliberately NOT plain ARM64:
# the self-hosted macOS runner is ARM64 too, and forcing it cold would
# be a cost with no matching risk.
do_isolate=false
if [ "$ISOLATE" = "true" ]; then
do_isolate=true
elif [ "${RUNNER_ENVIRONMENT:-}" = "self-hosted" ] \
&& [ "${RUNNER_OS:-}" = "Linux" ] && [ "${RUNNER_ARCH:-}" = "ARM64" ]; then
do_isolate=true
fi

# GitHub Actions cache: never when isolated, never on self-hosted.
# auto becomes true only on github-hosted. Unknown environment falls
# through to hosted so a beelink job landing on ubuntu-latest still
# caches.
cache_enabled=false
if [ "$do_isolate" != "true" ] && [ "${RUNNER_ENVIRONMENT:-}" != "self-hosted" ]; then
if [ "$CACHE" = "true" ] || [ "$CACHE" = "auto" ]; then
cache_enabled=true
fi
fi

# No volume backs ~/.pub-cache and the image's LRU cap does not reach
# it, so it is job-scoped in every mode. See the header.
echo "PUB_CACHE=${RUNNER_TEMP}/pub-cache" >> "$GITHUB_ENV"

if [ "$do_isolate" = "true" ]; then
{
echo "BUN_INSTALL_CACHE_DIR=${RUNNER_TEMP}/bun-cache"
echo "npm_config_cache=${RUNNER_TEMP}/npm-cache"
# pnpm reads npm_config_* as its own config: npm_config_store_dir
# is store-dir, npm_config_cache_dir is cache-dir. npm's `cache`
# setting above is NOT pnpm's, so both are needed — without them
# an isolated job still wrote to the shared ~/.cache/pnpm store.
echo "npm_config_store_dir=${RUNNER_TEMP}/pnpm-store"
echo "npm_config_cache_dir=${RUNNER_TEMP}/pnpm-cache"
# Berry derives YARN_CACHE_FOLDER/YARN_GLOBAL_FOLDER from
# cacheFolder/globalFolder; classic reads YARN_CACHE_FOLDER.
echo "YARN_CACHE_FOLDER=${RUNNER_TEMP}/yarn-cache"
echo "YARN_GLOBAL_FOLDER=${RUNNER_TEMP}/yarn-global"
echo "GRADLE_USER_HOME=${RUNNER_TEMP}/gradle-home"
echo "CARGO_HOME=${RUNNER_TEMP}/cargo-home"
} >> "$GITHUB_ENV"
fi

{
echo "BUN_INSTALL_CACHE_DIR=${RUNNER_TEMP}/bun-cache"
echo "npm_config_cache=${RUNNER_TEMP}/npm-cache"
echo "GRADLE_USER_HOME=${RUNNER_TEMP}/gradle-home"
echo "PUB_CACHE=${RUNNER_TEMP}/pub-cache"
echo "CARGO_HOME=${RUNNER_TEMP}/cargo-home"
} >> "$GITHUB_ENV"
echo "stack: $STACK, caches scoped to $RUNNER_TEMP"
echo "isolated=$do_isolate"
echo "cache_enabled=$cache_enabled"
} >> "$GITHUB_OUTPUT"
echo "stack: $STACK, isolate=$do_isolate, cache=$cache_enabled (input cache=$CACHE, env=${RUNNER_ENVIRONMENT:-unknown}/${RUNNER_OS:-unknown}/${RUNNER_ARCH:-unknown})"

- if: inputs.stack == 'bun'
uses: oven-sh/setup-bun@v2
with:
bun-version: ${{ inputs.bun-version }}
# The bun INSTALL CACHE is never uploaded to GitHub, on any runner:
# the tarball store is large and restore-keys on it is how poison
# reimports. setup-bun's own cache of the bun executable stays on —
# that is a small binary under an exact version key, not a store.

- if: inputs.stack == 'node'
uses: actions/setup-node@v7
with:
node-version: ${{ inputs.node-version }}
# No cache: npm. Hosted cache is one mechanism per stack, and this
# action never uploads a package store under restore-keys.

- if: (inputs.stack == 'gradle' || inputs.stack == 'android') || (inputs.stack == 'flutter' && inputs.java-version != '')
uses: actions/setup-java@v5
with:
java-version: ${{ inputs.java-version }}
distribution: temurin
# No cache: gradle. The hosted Gradle cache is setup-gradle's job.

- if: inputs.stack == 'gradle' || inputs.stack == 'android'
uses: gradle/actions/setup-gradle@v6
with:
# Publishing a build scan from a verification job leaks the dependency
# graph of a private repo to a third party. Off unless a repo opts in.
build-scan-publish: false
cache-read-only: ${{ inputs.cache != 'true' }}
cache-disabled: ${{ steps.resolve.outputs.cache_enabled != 'true' }}

- if: inputs.stack == 'android'
uses: android-actions/setup-android@v4
Expand All @@ -135,7 +229,7 @@ runs:
with:
flutter-version: ${{ inputs.flutter-version }}
channel: ${{ inputs.flutter-channel }}
cache: ${{ inputs.cache }}
cache: ${{ steps.resolve.outputs.cache_enabled == 'true' }}

# Pinned to @master with an explicit `toolchain`, which is how this action
# is meant to be parameterised — the `@stable` form the README leads with
Expand All @@ -146,7 +240,7 @@ runs:
toolchain: ${{ inputs.rust-toolchain }}
components: ${{ inputs.rust-components }}

- if: inputs.stack == 'rust' && inputs.cache == 'true'
- if: inputs.stack == 'rust' && steps.resolve.outputs.cache_enabled == 'true'
uses: Swatinem/rust-cache@v2

- if: inputs.stack == 'xcode' && inputs.xcode-version != ''
Expand Down