Skip to content

Commit 83961ac

Browse files
anandgupta42claude
andcommitted
fix: Windows grep outage and test-run telemetry pollution
Two production defects found in Azure telemetry over 2026-07-22 → 2026-08-05. **Windows `grep` broken for 99 machines** `core_failure` showed 328 events across 99 distinct Windows machines (of 617 total) carrying `? is not recognized as an internal or external command, operable program or batch file.` and its German, French, Spanish and Portuguese translations. Present on released 0.9.2, 0.9.3 and 0.9.4. Root cause: ripgrep's Windows release is a zip, and `RipgrepBinary` extracted it via `powershell.exe -Command Expand-Archive`, falling back to the literal string `"powershell.exe"` when neither `powershell.exe` nor `pwsh.exe` resolved. `cross-spawn`'s `parseNonShell()` sets `needsShell = true` when `resolveCommand()` returns undefined and re-spawns through `cmd.exe /d /s /c`, so cmd.exe produced that message. `throw new Error(result.stderr.trim())` made it the error verbatim, and since `RipgrepBinary.filepath` is `Effect.cached`, one failed extraction broke grep for the whole session. Upstream carries the same fragility: anomalyco/opencode#24291 is open, reporting `Expand-Archive` unusable when spawned from the Bun-compiled binary, affecting `grep`, `glob` and `skill`. Their #23457 fix only corrected how paths were passed to PowerShell (the `$args` → inlined-and-escaped form we already carry); it did not remove the dependency on PowerShell being resolvable. Extract the zip in-process with `@zip.js/zip.js`, converging on the approach the `packages/opencode/src/file/ripgrep.ts` shim already uses in production. `unzipExecutable` is exported so archive handling is tested directly, and decodes with `checkSignature: true` — zip.js defaults it off, and a CRC-corrupt download would otherwise be written to the cache and trusted by every later session. Install the binary atomically (stage to `rg.exe.tmp`, then rename). `filepath` trusts the cached binary on existence alone, so an interrupted write previously left a truncated `rg.exe` that every later session reused — the same permanent breakage `checkSignature` guards against, which CRC cannot catch because it is verified before the write. The tar path installs the same way. Attribute resolution failures. Child stderr was reported verbatim, so a shell-level failure was indistinguishable from a tool bug; and because a resolve failure is memoized, it is re-reported on every later grep in the session. The tar branch now names ripgrep, and any typed filesystem or HTTP failure is wrapped as `ripgrep binary resolve failed: …`. Note the blast radius is wider than the `grep` tool: `@opencode-ai/core/ripgrep` also backs the HTTP-API file handlers and `cli/cmd/debug/ripgrep.ts`. **Test runs shipped telemetry to the production resource** 1,020 of 3,135 machine ids in the same window emitted `provider_id="test"` / `cli_version="local"` — test processes that regenerate their machine id every run, inflating install and active-machine counts by roughly a third. `doInit()` gated only on `ALTIMATE_TELEMETRY_DISABLED`. Refuse the baked-in connection string when `NODE_ENV=test`, `BUN_TEST`, `VITEST` or `JEST_WORKER_ID` is set. Keyed on test runners, deliberately not on CI: `altimate-code-actions` wraps this CLI, so every run of that shipped product sets `CI`/`GITHUB_ACTIONS`, and gating on those would blind a real product surface. `bun test` sets `NODE_ENV=test`, which covers CI and developer machines alike. An explicit `APPLICATIONINSIGHTS_CONNECTION_STRING` is always honoured, so suites with their own sink are unaffected; `ALTIMATE_TELEMETRY_FORCE=true` overrides the default-sink refusal, and the existing opt-outs still win over both. Test plan: 8 ripgrep tests — including a layer-level test that drives `filepath` through the Windows zip path with a spawner that fails if invoked, asserting the staged-then-renamed install, and a CRC case that fails without `checkSignature` — plus 15 telemetry-gate tests covering CI-alone-still-reports and one that relies on the real runner's `NODE_ENV`. `turbo typecheck` green, marker guard green, `packages/core` and `packages/opencode` suites show no failures beyond the pre-existing set on `main`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 03b9459 commit 83961ac

10 files changed

Lines changed: 788 additions & 32 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -379,6 +379,39 @@ jobs:
379379
$config.Output.Verbosity = "Detailed"
380380
Invoke-Pester -Configuration $config
381381
382+
# altimate_change start — Windows ripgrep E2E (issue #1072)
383+
# ---------------------------------------------------------------------------
384+
# Real Windows check for ripgrep binary resolution. Downloads and extracts the
385+
# actual archive with PowerShell stripped from PATH, then executes the binary.
386+
# This is the condition that broke grep for 99 Windows machines; no amount of
387+
# unit testing on Linux/macOS covers it.
388+
# ---------------------------------------------------------------------------
389+
windows-ripgrep-e2e:
390+
name: Windows ripgrep E2E
391+
needs: changes
392+
if: needs.changes.outputs.typescript == 'true' || github.event_name == 'push'
393+
runs-on: windows-latest
394+
timeout-minutes: 15
395+
steps:
396+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
397+
398+
- uses: oven-sh/setup-bun@ecf28ddc73e819eb6fa29df6b34ef8921c743461 # v2
399+
with:
400+
bun-version: "1.3.14"
401+
402+
# --ignore-scripts: `tree-sitter-powershell` has no Windows prebuild and its fallback
403+
# compile needs Visual Studio Build Tools, which this runner does not have (see
404+
# anomalyco/opencode#25563). This check only needs the pure-JS dependency graph
405+
# (effect, @zip.js/zip.js, which, xdg-basedir), so skipping lifecycle scripts is enough.
406+
- name: Install dependencies
407+
run: bun install --ignore-scripts
408+
409+
# Run from packages/core so `effect` and the other deps resolve — they are not root deps.
410+
- name: Resolve ripgrep with PowerShell unavailable
411+
working-directory: packages/core
412+
run: bun run script/windows-ripgrep-e2e.ts
413+
# altimate_change end
414+
382415
# ---------------------------------------------------------------------------
383416
# dbt-tools E2E — slow (~3 min), only on push to main.
384417
# Tests dbt CLI fallbacks against real dbt versions (1.8, 1.10, 1.11) and

‎bun.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/docs/reference/telemetry.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,24 @@ export ALTIMATE_TELEMETRY_DISABLED=true
126126

127127
When telemetry is disabled, no events are sent and no network requests are made to the telemetry endpoint.
128128

129+
### Test runs are excluded
130+
131+
Test runners never reach the default telemetry endpoint. Telemetry is suppressed when `NODE_ENV=test`,
132+
`BUN_TEST`, `VITEST`, or `JEST_WORKER_ID` is present. This exists because test processes regenerate
133+
their machine ID on every run, so without the exclusion they dominate install and active-machine counts.
134+
135+
Running in CI is **not** excluded — that is ordinary product usage (for example
136+
[altimate-code-actions](https://github.com/AltimateAI/altimate-code-actions) wraps this CLI), so
137+
`CI` and `GITHUB_ACTIONS` on their own do not suppress anything.
138+
139+
Two escape hatches exist for reporting from a test run deliberately:
140+
141+
- Set `APPLICATIONINSIGHTS_CONNECTION_STRING` to your own endpoint — an explicitly-configured sink
142+
is always honoured, which is how the project's own telemetry tests work.
143+
- Set `ALTIMATE_TELEMETRY_FORCE=true` to use the default endpoint anyway.
144+
145+
`ALTIMATE_TELEMETRY_DISABLED` and the config opt-out take precedence over both.
146+
129147
## Privacy
130148

131149
We take your privacy seriously. Altimate Code telemetry **never** collects:

‎packages/core/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@
9797
"@opentelemetry/sdk-trace-base": "2.6.1",
9898
"@parcel/watcher": "2.5.1",
9999
"@silvia-odwyer/photon-node": "0.3.4",
100+
"@zip.js/zip.js": "2.7.62",
100101
"@openrouter/ai-sdk-provider": "2.9.0",
101102
"ai-gateway-provider": "3.1.2",
102103
"bun-pty": "0.4.8",
Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
/**
2+
* Windows end-to-end check for ripgrep binary resolution.
3+
*
4+
* Reproduces as much as a GitHub runner allows of the condition behind the outage in
5+
* https://github.com/AltimateAI/altimate-code/issues/1072: a cold cache on a Windows machine
6+
* where PowerShell cannot be resolved from PATH. The old implementation extracted ripgrep's zip
7+
* by shelling out to `powershell.exe -Command Expand-Archive`, so cross-spawn fell back to
8+
* `cmd.exe /d /s /c` and the extraction died with "is not recognized as an internal or external
9+
* command" — which `Effect.cached` then replayed for the rest of the session.
10+
*
11+
* This performs a real download and a real extraction, then executes the resulting binary. It is
12+
* deliberately not a unit test: the point is to exercise the actual filesystem, the actual archive
13+
* and the actual process launch on a real Windows host.
14+
*
15+
* Scope note: stripping PATH does NOT make PowerShell unspawnable on Windows (see the control
16+
* probe at the end), so this does not reproduce the affected machines. The guarantee that
17+
* extraction spawns nothing at all is established by test/ripgrep-windows.test.ts.
18+
*
19+
* Run: bun run script/windows-ripgrep-e2e.ts (from packages/core — `effect` resolves there)
20+
*/
21+
import { execFileSync } from "node:child_process"
22+
import { mkdtempSync, existsSync, readdirSync, statSync, rmSync } from "node:fs"
23+
import { tmpdir } from "node:os"
24+
import path from "node:path"
25+
26+
function fail(message: string): never {
27+
console.error(`FAIL: ${message}`)
28+
process.exit(1)
29+
}
30+
31+
function ok(message: string) {
32+
console.log(`ok ${message}`)
33+
}
34+
35+
if (process.platform !== "win32") fail(`this check only means anything on Windows (got ${process.platform})`)
36+
37+
// Cold cache: point the XDG cache root at a throwaway directory *before* importing anything that
38+
// reads it, since Global computes its paths at module load.
39+
const cacheRoot = mkdtempSync(path.join(tmpdir(), "rg-e2e-"))
40+
process.env.XDG_CACHE_HOME = cacheRoot
41+
process.env.LOCALAPPDATA = cacheRoot
42+
43+
// Remove every PATH entry that could provide PowerShell. System32 itself is kept so cmd.exe and
44+
// the rest of Windows still work — this simulates the locked-down/sanitized-PATH machines in the
45+
// telemetry, not a broken OS.
46+
const originalPath = process.env.PATH ?? process.env.Path ?? ""
47+
const stripped = originalPath
48+
.split(path.delimiter)
49+
.filter((entry) => entry && !/powershell/i.test(entry))
50+
.join(path.delimiter)
51+
process.env.PATH = stripped
52+
process.env.Path = stripped
53+
54+
const { which } = await import("../src/util/which")
55+
56+
// Guard against a vacuous run: if PowerShell is still resolvable, this proves nothing.
57+
const ps = which("powershell.exe")
58+
const pwsh = which("pwsh.exe")
59+
if (ps || pwsh) fail(`PowerShell is still resolvable (${ps ?? pwsh}) — the scenario was not reproduced`)
60+
ok("PowerShell is not resolvable on PATH (failure condition reproduced)")
61+
62+
// Also assert the binary really is absent, so we exercise download + extract rather than a cache hit.
63+
const { Global } = await import("../src/global")
64+
const target = path.join(Global.Path.bin, "rg.exe")
65+
if (existsSync(target)) fail(`expected a cold cache but ${target} already exists`)
66+
ok(`cold cache at ${Global.Path.bin}`)
67+
68+
const { Effect } = await import("effect")
69+
const { RipgrepBinary } = await import("../src/ripgrep/binary")
70+
71+
/** Resolve through the real layer: real HTTP, real filesystem, real process launch. */
72+
async function resolveBinary(): Promise<string> {
73+
const program = Effect.gen(function* () {
74+
const binary = yield* RipgrepBinary.Service
75+
return yield* binary.filepath
76+
}).pipe(Effect.provide(RipgrepBinary.defaultLayer))
77+
return (await Effect.runPromise(program as never)) as string
78+
}
79+
80+
let resolved: string
81+
try {
82+
resolved = await resolveBinary()
83+
} catch (err: unknown) {
84+
fail(`binary.filepath failed: ${err instanceof Error ? err.message : String(err)}`)
85+
}
86+
87+
ok(`resolved ${resolved}`)
88+
89+
if (!existsSync(resolved)) fail(`resolved path does not exist: ${resolved}`)
90+
const size = statSync(resolved).size
91+
if (size < 100_000) fail(`resolved binary is implausibly small (${size} bytes) — likely a partial write`)
92+
ok(`binary present, ${size} bytes`)
93+
94+
// No staging files should survive a successful install.
95+
const leftovers = readdirSync(Global.Path.bin).filter((f) => f.endsWith(".tmp"))
96+
if (leftovers.length > 0) fail(`staging files left behind: ${leftovers.join(", ")}`)
97+
ok("no staging files left behind")
98+
99+
// The real proof: the extracted binary actually executes.
100+
const version = execFileSync(resolved, ["--version"], { encoding: "utf8" })
101+
if (!/ripgrep\s+\d/.test(version)) fail(`unexpected --version output: ${version.trim()}`)
102+
ok(`executes: ${version.split("\n")[0]!.trim()}`)
103+
104+
// And it can actually search.
105+
const hit = execFileSync(resolved, ["--no-config", "NEEDLE_MARKER", "--", import.meta.filename], {
106+
encoding: "utf8",
107+
})
108+
if (!hit.includes("NEEDLE_MARKER")) fail("ripgrep did not return the expected match")
109+
ok("search returns matches") // NEEDLE_MARKER
110+
111+
// A second resolve must hit the cache and stay valid.
112+
const again = await resolveBinary()
113+
if (again !== resolved) fail(`second resolve returned a different path: ${again}`)
114+
ok("second resolve hits the cache")
115+
116+
// ---------------------------------------------------------------------------
117+
// Control probe — informational, deliberately not a hard failure.
118+
//
119+
// It would be neater to also show that the OLD implementation fails here, making this a true
120+
// counterfactual. It does not, and that is worth recording: emptying PATH is not enough to make
121+
// PowerShell unspawnable on Windows. cross-spawn falls back to `cmd.exe /d /s /c`, and Windows
122+
// process creation searches beyond PATH (the caller's directory, the system directories, and the
123+
// App Paths registry key), so `powershell.exe` still starts on a stock GitHub runner even though
124+
// `which()` cannot see it.
125+
//
126+
// So this job does NOT reproduce the affected machines. What it does prove is the part that
127+
// matters: on real Windows the new path downloads, extracts in-process, installs and produces a
128+
// working rg.exe. That PowerShell's availability is irrelevant to it is established separately and
129+
// structurally by `test/ripgrep-windows.test.ts`, which drives the same code with a spawner that
130+
// fails the test if anything is launched at all.
131+
// ---------------------------------------------------------------------------
132+
const launch = (await import("cross-spawn")).default
133+
const control = launch.sync("powershell.exe", ["-NoProfile", "-NonInteractive", "-Command", "exit 0"], {
134+
encoding: "utf8",
135+
})
136+
137+
if (control.status === 0) {
138+
console.log(
139+
"note this runner still spawns powershell.exe despite the stripped PATH (Windows resolves\n" +
140+
" executables beyond PATH), so the affected environment is not reproduced here — the\n" +
141+
" no-spawn guarantee comes from the unit layer test, not from this job",
142+
)
143+
} else {
144+
const output = `${control.stderr ?? ""}${control.stdout ?? ""}${control.error?.message ?? ""}`.trim()
145+
ok(`bonus: the old PowerShell path also fails here (${output.split("\n")[0]?.slice(0, 100) || `status=${control.status}`})`)
146+
}
147+
148+
rmSync(cacheRoot, { recursive: true, force: true })
149+
console.log("\nPASS — on real Windows, ripgrep downloads, extracts in-process, installs atomically")
150+
console.log(" and runs. See the note above for what this job does and does not establish.")

‎packages/core/src/ripgrep.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,7 +140,11 @@ export const layer = Layer.effect(
140140
return yield* new InvalidPatternError({ pattern: input.pattern, message: stderr.trim() })
141141
}
142142
if (code !== 0 && code !== 1 && code !== 2) {
143-
return yield* failure(stderr.trim() || `ripgrep failed with code ${code}`)
143+
// altimate_change start — upstream_fix: keep child stderr attributable to ripgrep.
144+
// Reporting stderr verbatim made shell-level failures (e.g. a Windows "not recognized"
145+
// message) look like they came from the tool itself, with no hint of the real source.
146+
return yield* failure(`ripgrep failed with code ${code}: ${stderr.trim() || "no output"}`)
147+
// altimate_change end
144148
}
145149
return { items: code === 1 ? [] : rows, truncated: false, partial: code === 2 }
146150
}),

0 commit comments

Comments
 (0)