Skip to content

Commit bc6e68d

Browse files
authored
⚡ Remeasure test weights at ed5369f and recalibrate the runtime shards (#721)
* ⚡ Remeasure test weights at ed5369f and install the measured shard floors The committed weights were measured at 170a021, fifty commits back; 43 Deno, 32 Node and 31 Bun files had no weight and ran at the corpus maximum, so the partition predicted 587/634/909ms-thousands per shard and every runtime ran over the 300s ceiling on main (run 33574538193: Deno 399s, Node 388s, Bun 491s). test-weights.json is byte-for-byte the product of measure-test-weights run 33576103883 at ed5369f on ubuntu-latest (deno 2.9.5, node v22.23.2, bun 1.3.14). Every applicable file is measured. Serial sums: Deno 2596s (370 files), Node 1118s (286), Bun 715s (285); floors 9/4/3, installed here as the calibration's starting counts. * ⚡ Calibrate the runtime shards to 10/5/4 after the floors missed Run 33581198496 at 1ea893b (9/4/3): Deno worst Test step 357s, window 373s; Node worst 452s, window 459s; Bun worst 282s but window 338s. Each runtime increments by one, per the rule in specs/testing-spec.md. * ⚡ Calibrate the runtime shards to 11/6/4 after Deno and Node missed at 10/5 Run 33581898672 at 1981e9b (10/5/4): Deno worst Test step 323s, window 332s; Node worst 334s, window 345s; Bun passed at 225s. Deno and Node increment by one; Bun holds. * ⚡ Calibrate the Node shards to 7 after a 301s step at 6 Run 33582364710 at 5bc7e0a (11/6/4): Node worst Test step 301s. Every Deno step passed (worst 298s); its 409s window is 124s of runner queueing on a 33-job run, which the shard count cannot shorten, so Deno holds at 11 pending a decision on the window clause. Bun passed at 218s. * ⚡ Calibrate the Deno shards to 12 after a 323s step at 11 Run 33582959074 at 8746706 (11/7/4): Deno shard 4/11 took 323s against a 236s prediction; Node passed at 251s and Bun at 223s. Deno increments by one; Node and Bun hold. * ⚡ Calibrate the Deno shards to 13 after a 296s step at 12 Run 33583487112 attempt 3 at 77d4cbc (12/7/4): Deno shard 7/12 took 296s and the runtime's window was 301s — the shard itself is at the ceiling. Node's steps were under (worst 255s); its 306s window was 74s of runner queueing while #718's CI ran alongside, so Node holds at 7. Bun passed at 237s. * ⚡ Remeasure test weights at a7f60c0, on Bun 1.4.0, with #718's new file main moved during calibration: #718 added packages/cli/tests/stdout-delivery.test.ts, and the workflows moved Bun from 1.3.14 to 1.4.0. test-weights.json is byte-for-byte the product of measure-test-weights run 33585991716 at a7f60c0 on ubuntu-latest (deno 2.9.5, node v22.23.2, bun 1.4.0); every applicable file is measured. Against the ed5369f measurement the per-file ratios are uniform — median 1.32 under Deno and 1.31 under Node across every file — which is a slower runner VM, not the corpus; Bun's median ratio is 1.04, so 1.4.0 performs as 1.3.14 did. syntax-cli.test.ts alone grew for real (×1.7–2.1), by #718's three pipe rows. The partition depends on relative weights, so the split at 13/7/4 is unchanged in shape; the floors move to 11/5/3. --------- Co-authored-by: Taras Mankovski <74687+taras@users.noreply.github.com>
1 parent a7f60c0 commit bc6e68d

2 files changed

Lines changed: 951 additions & 842 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -82,11 +82,11 @@ jobs:
8282
# an artifact-and-dependency boundary between jobs that #279 spent its length
8383
# removing.
8484
test-deno:
85-
name: test-deno (${{ matrix.shard }}/9)
85+
name: test-deno (${{ matrix.shard }}/13)
8686
strategy:
8787
fail-fast: false
8888
matrix:
89-
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9]
89+
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
9090
runs-on: ubuntu-latest
9191
steps:
9292
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
@@ -121,7 +121,7 @@ jobs:
121121
run: deno task build:web
122122

123123
- name: Test
124-
run: deno run --allow-all --frozen scripts/runtime-tests.ts deno ${{ matrix.shard }}/9
124+
run: deno run --allow-all --frozen scripts/runtime-tests.ts deno ${{ matrix.shard }}/13
125125

126126
# The same command the release publishes with, minus --dry-run. Catches slow
127127
# types and unresolvable specifiers before they reach a tag.
@@ -573,11 +573,11 @@ jobs:
573573
working-directory: site
574574

575575
test-node:
576-
name: test-node (${{ matrix.shard }}/5)
576+
name: test-node (${{ matrix.shard }}/7)
577577
strategy:
578578
fail-fast: false
579579
matrix:
580-
shard: [1, 2, 3, 4, 5]
580+
shard: [1, 2, 3, 4, 5, 6, 7]
581581
runs-on: ubuntu-latest
582582
steps:
583583
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
@@ -615,14 +615,14 @@ jobs:
615615
run: pnpm exec tsc --project tsconfig.node.json --noEmit
616616

617617
- name: Test
618-
run: pnpm test:node ${{ matrix.shard }}/5
618+
run: pnpm test:node ${{ matrix.shard }}/7
619619

620620
test-bun:
621-
name: test-bun (${{ matrix.shard }}/2)
621+
name: test-bun (${{ matrix.shard }}/4)
622622
strategy:
623623
fail-fast: false
624624
matrix:
625-
shard: [1, 2]
625+
shard: [1, 2, 3, 4]
626626
runs-on: ubuntu-latest
627627
steps:
628628
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
@@ -634,7 +634,7 @@ jobs:
634634
- run: bun install
635635

636636
- name: Test
637-
run: bun run test:bun ${{ matrix.shard }}/2
637+
run: bun run test:bun ${{ matrix.shard }}/4
638638

639639
# test:bun never loads packages/cli/src/bun.ts, so it cannot tell whether
640640
# the Bun entrypoint's API.Env providers work. This document drives a

0 commit comments

Comments
 (0)