Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
5a82634
engine: profile harness fidelity -- mimalloc backing and an LTM mode
bpowers Aug 10, 2026
7664729
engine: fuse SetCond;If[;AssignCurr] into a conditional-select opcode
bpowers Aug 10, 2026
2d34ba4
engine: give builtins a real arity instead of padding Apply to three …
bpowers Aug 10, 2026
4c68ef3
engine: memoize the direct LTM fragment compile
bpowers Aug 10, 2026
c556957
engine: fuse leaf stores and give module inputs a fusible leaf form
bpowers Aug 10, 2026
fb82209
engine: skip the no-op canonicalize in is_dimension_name
bpowers Aug 10, 2026
9a9934e
engine: share LTM equation ASTs between the shaped memo and emission
bpowers Aug 10, 2026
6b27dec
diagram: share one font database across PNG renders
bpowers Aug 10, 2026
f47a73c
engine: resolve a lookup's constant element offset at compile time
bpowers Aug 10, 2026
6e3334b
engine: derive variable_dimensions without a second parse
bpowers Aug 10, 2026
819d23d
build: optimize the resvg rasterization stack in dev builds
bpowers Aug 10, 2026
9bb392b
build: optimize the salsa/indexmap query substrate in dev builds
bpowers Aug 10, 2026
e74d4d6
engine: pin which slots carry information across a simulation step
bpowers Aug 10, 2026
bdf605c
test: run the metasd macro expansion tier one model per test
bpowers Aug 10, 2026
a32766d
test: stop re-simulating the protobuf round-trip in the corpus harness
bpowers Aug 10, 2026
89261c3
doc: separate the measurement channels and their noise floors
bpowers Aug 10, 2026
eef3b31
engine: make compile_implicit_var_fragment a salsa query
bpowers Aug 10, 2026
ada4339
doc: correct the stated reason for the lazy-If NO-GO (#711)
bpowers Aug 10, 2026
44ff669
build: cache prettier results in the format check
bpowers Aug 10, 2026
457fb25
engine: memoize the cycle gate's symbolic fragment probe
bpowers Aug 10, 2026
0fa2621
engine: omit LTM arms provably equal to PREVIOUS(target)
bpowers Aug 10, 2026
ea342b2
engine: fold constant operands into LoadPrev and Apply
bpowers Aug 10, 2026
c07afbf
engine: FxHash the topological-sort probe maps
bpowers Aug 10, 2026
eec0c81
engine: fuse the PREVIOUS delta into a single opcode
bpowers Aug 10, 2026
2e96af6
engine: answer the engine's own separators without the case tables
bpowers Aug 10, 2026
b7ca9f8
doc: record compile round 3 and the parallel fan-out constraints
bpowers Aug 10, 2026
26b65be
engine: add a value-level LTM gate at two scales
bpowers Aug 10, 2026
3235124
build: skip wasm-opt in the pre-commit hook's TypeScript build
bpowers Aug 10, 2026
99bf33d
engine: pin that fusion never raises a program's peak stack depth
bpowers Aug 10, 2026
116b855
ci: run wasm-opt and test the optimized bundle on engine changes
bpowers Aug 10, 2026
d3074a3
engine: name what enforces emit_apply's arity discipline
bpowers Aug 10, 2026
69d56f9
build: resolve the cargo target dir instead of assuming it
bpowers Aug 10, 2026
b43ec32
doc: record what a warm single-equation edit costs (C6)
bpowers Aug 10, 2026
105def7
engine: correct four comments claiming compilation is rayon-parallel
bpowers Aug 10, 2026
e913c3e
engine: name the two corpus lists apart
bpowers Aug 10, 2026
7d9c60c
engine: record why the LTM implicit-var parse is duplicated
bpowers Aug 10, 2026
da10cfb
engine: make the graphical-function block scan exhaustive over opcodes
bpowers Aug 10, 2026
2ac7ac5
doc: size a fast path by the work it replaces, and record why one did…
bpowers Aug 10, 2026
b890eaf
Merge branch 'perf/tests' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
27fb1fb
Merge branch 'perf/compile' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
5012443
Merge branch 'perf/vm-dispatch' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
5a0dd0e
Merge branch 'perf/ltm-compile' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
3d48866
doc: record the superinstruction family and the LTM arm omission
bpowers Aug 10, 2026
929051e
build: add an interleaved A/B harness for the profiling example
bpowers Aug 10, 2026
a8fb763
engine: generate LookupDirect in the merge proptests, and read GF run…
bpowers Aug 10, 2026
f27f465
build: make the WASM staging cache record whether wasm-opt ran
bpowers Aug 10, 2026
37f441f
engine: require lag alignment before omitting an LTM arm
bpowers Aug 10, 2026
1f7d1ba
engine: gate variable_dimensions on the A2A equation having tokens
bpowers Aug 10, 2026
4b47f5e
engine: make the LTM fragment index-to-variable coupling checkable
bpowers Aug 10, 2026
2809ae3
Merge branch 'fix-wasm' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
bc85187
Merge branch 'fix-dims' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
93c0abe
Merge branch 'fix-ltm' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
47a5872
Merge branch 'fix-proptest' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
721c3e4
engine: pin the double-lag residual and drop a workaround the fix ret…
bpowers Aug 10, 2026
4d7564d
Merge branch 'fix-ltm' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
5595a0a
build: drop the jq dependency from the cargo target-dir lookup
bpowers Aug 10, 2026
6ee73fd
build: correct what the REQUIRE_WASM_OPT guard claims to cover
bpowers Aug 10, 2026
2ad5906
Merge branch 'fix-wasm' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
14a04ca
engine: split the two corpus sweeps into per-model tests, and gate th…
bpowers Aug 10, 2026
8a5b9ad
Merge branch 'fix-proptest' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
f8a00ff
engine: quantize the C-LEARN LTM digest relatively, as documented
bpowers Aug 10, 2026
d97ecb6
Merge branch 'fix-ltm' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
2afe626
engine: make the harness allocator claim true, and pin the pysimlin a…
bpowers Aug 10, 2026
f49ed5c
engine: disclose and pin the LTM omission's non-finite value change
bpowers Aug 10, 2026
89d2079
Merge branch 'fix-ltm' into engine-perf-pass-2026-08
bpowers Aug 10, 2026
b8b3af8
build: install a pinned binaryen instead of the distro's, and ignore …
bpowers Aug 10, 2026
2f9194b
engine: describe Apply's real arity at the three sites that still cla…
bpowers Aug 11, 2026
ef01c77
Merge branch 'fix-proptest' into engine-perf-pass-2026-08
bpowers Aug 11, 2026
d9b575a
build: invalidate the WASM mode stamp before restaging, not just after
bpowers Aug 11, 2026
ef16665
ci: trigger the wasm-opt lane on changes to its binaryen installer
bpowers Aug 11, 2026
db0301b
engine: cover every LTM slot in the C-LEARN digest, and bind it to id…
bpowers Aug 11, 2026
8a9cd94
Merge branch 'fix-wasm' into engine-perf-pass-2026-08
bpowers Aug 11, 2026
9d62631
Merge branch 'fix-ltm' into engine-perf-pass-2026-08
bpowers Aug 11, 2026
b689e03
engine: make the digest's permutation test constrain identity, not order
bpowers Aug 11, 2026
3029f13
Merge branch 'fix-ltm' into engine-perf-pass-2026-08
bpowers Aug 11, 2026
4ab02a7
build: drop the undeclared PyYAML import from the workflow-paths lint
bpowers Aug 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/ts-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,12 @@ jobs:
rustup show
rustup target add wasm32-unknown-unknown

# A pinned upstream release, NOT `apt-get install binaryen`: the distro
# version is older than the flags build.sh passes and fails with a bare
# "Unknown option '--enable-bulk-memory-opt'". Shared with ts-release.yml,
# which optimizes the bundle it publishes and would fail identically.
- name: Install wasm-opt
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq binaryen
run: ./scripts/install-binaryen.sh

- uses: pnpm/action-setup@v4

Expand Down
156 changes: 156 additions & 0 deletions .github/workflows/wasm-opt.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
---
# The ONLY automated job that runs `wasm-opt` and then executes the resulting
# blob under test.
#
# `src/engine/build.sh` runs `wasm-opt -O3` IN PLACE over `core/*.wasm`, which
# is the file the TypeScript tests load -- so "did wasm-opt run" and "was the
# optimized bundle executed under test" are the same question. Everywhere else
# deliberately answers no: both of `ci.yaml`'s build steps set
# DISABLE_WASM_OPT=1 (the pass is multi-minute and a PR gate does not need
# release-quality WASM), and `scripts/pre-commit` does the same. Only
# `ts-release.yml` installs binaryen at all, and by then the artifact is being
# published. This job is what keeps that gap from reaching a release. See
# GH #1019.
#
# It is path-filtered to the sources that can change the emitted WASM, which is
# why it lives in its own workflow rather than as a job in `ci.yaml`: GitHub
# applies `paths` per workflow, not per job, and filtering inside `ci.yaml`
# would mean adding a third-party paths-filter action.
#
# Do NOT make this a required status check in branch protection as-is. A
# path-filtered workflow reports nothing at all on a PR that touches none of
# these paths, and a required check that never reports blocks the PR forever.
# If it needs to be required, the standard workaround is an always-triggered
# companion job that succeeds trivially when the filter does not match.
#
# IF THIS JOB FAILS AND `ci.yaml` PASSED, the difference is wasm-opt. The same
# TypeScript tests run in `ci.yaml` against an UNOPTIMIZED blob; if they pass
# there and fail here, the engine's WASM did not survive binaryen's -O3 pass --
# a miscompilation, an unsupported feature, or a binaryen version
# incompatibility -- not a defect in the TypeScript under test. Reproduce
# locally with `bash src/engine/build.sh && pnpm -C src/engine test` (note the
# absence of DISABLE_WASM_OPT), and compare against
# `DISABLE_WASM_OPT=1 bash src/engine/build.sh && pnpm -C src/engine test`.
name: WASM optimized-bundle check

"on":
push:
branches:
- main
paths:
- 'src/simlin-engine/**'
- 'src/libsimlin/**'
- 'src/engine/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.cargo/config.toml'
# The compiler selects what WASM gets emitted, and with it whether
# binaryen can still read it -- and the ordinary frontend lane runs with
# DISABLE_WASM_OPT=1, so a toolchain bump would otherwise reach a release
# without either blob having been optimized once.
- 'rust-toolchain.toml'
# This lane and ts-release.yml are the installer's only consumers, and
# ts-release runs on tags and manual dispatch -- so a version bump, a
# changed asset name or a dead URL would otherwise first surface DURING
# an npm release.
- 'scripts/install-binaryen.sh'
- '.github/workflows/wasm-opt.yml'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include the Binaryen installer in path filters

When a PR changes scripts/install-binaryen.sh—for example, bumping the pinned version or altering an asset name—neither path list triggers this workflow, even though this job consumes that script and the only other consumer, ts-release.yml, runs only for tags or manual dispatch. A broken download or incompatible default can therefore merge without executing the installer and first surface during an npm release; add the script to both the push and pull-request path lists.

Useful? React with 👍 / 👎.

pull_request:
branches:
- main
paths:
- 'src/simlin-engine/**'
- 'src/libsimlin/**'
- 'src/engine/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.cargo/config.toml'
# The compiler selects what WASM gets emitted, and with it whether
# binaryen can still read it -- and the ordinary frontend lane runs with
# DISABLE_WASM_OPT=1, so a toolchain bump would otherwise reach a release
# without either blob having been optimized once.
- 'rust-toolchain.toml'
# This lane and ts-release.yml are the installer's only consumers, and
# ts-release runs on tags and manual dispatch -- so a version bump, a
# changed asset name or a dead URL would otherwise first surface DURING
# an npm release.
- 'scripts/install-binaryen.sh'
- '.github/workflows/wasm-opt.yml'
Comment on lines +65 to +78

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Trigger the optimized-WASM check on toolchain changes

When a PR changes rust-toolchain.toml, this path-filtered workflow does not run even though the selected Rust compiler directly changes the emitted WASM and can change binaryen compatibility. The ordinary frontend CI build deliberately sets DISABLE_WASM_OPT=1, so such a compiler upgrade can merge without executing either optimized blob; the first optimized validation may then occur only during a release. Add rust-toolchain.toml to both path lists so the advertised pre-release optimized-bundle check covers compiler changes.

Useful? React with 👍 / 👎.


env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0

jobs:
wasm-opt:
name: Build with wasm-opt and run the engine tests against it
runs-on: ubuntu-latest
# wasm-opt -O3 is ~170s on our two blobs (90s + 76s), on top of the wasm
# cargo build. Generous cap so a cold cargo cache does not trip it.
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Rust toolchain
run: rustup show

# A pinned upstream release, NOT `apt-get install binaryen`: the distro
# version is older than the flags build.sh passes and fails with a bare
# "Unknown option '--enable-bulk-memory-opt'". Shared with ts-release.yml,
# which optimizes the bundle it publishes and would fail identically.
- name: Install wasm-opt
run: ./scripts/install-binaryen.sh

- name: Install pnpm
uses: pnpm/action-setup@v4

- name: Install node
uses: actions/setup-node@v4
with:
node-version: '22'
cache: 'pnpm'

- name: Cache cargo registry and target
uses: actions/cache@v4
with:
path: |
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: cargo-wasmopt-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
cargo-wasmopt-

- name: Install pnpm dependencies
run: pnpm install

# No DISABLE_WASM_OPT here -- that omission is the entire point of this
# workflow, so do not "fix" it to match ci.yaml.
- name: Build with wasm-opt enabled
run: pnpm build

# Fail loudly if the pass silently did not run: build.sh skips wasm-opt
# when binaryen is absent, printing "Skipping wasm-opt" and exiting 0, so
# without this check a broken install would turn this job into an
# expensive duplicate of ci.yaml's frontend job.
- name: Assert the blobs are actually optimized
run: |
set -euo pipefail
for f in src/engine/core/libsimlin.wasm src/engine/core/libsimlin-browser.wasm; do
if [ ! -f "$f.raw" ]; then
echo "ERROR: $f.raw missing -- src/engine/build.sh did not stage this blob" >&2
exit 1
fi
if cmp -s "$f" "$f.raw"; then
echo "ERROR: $f is byte-identical to the pre-wasm-opt output, so" >&2
echo " wasm-opt did not run. Is binaryen installed, and is" >&2
echo " DISABLE_WASM_OPT unset? This job exists to run it." >&2
exit 1
fi
printf '%s: optimized (%s -> %s bytes)\n' "$f" "$(wc -c < "$f.raw")" "$(wc -c < "$f")"
done

- name: Run the TypeScript tests against the optimized bundle
run: pnpm test
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

56 changes: 56 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,26 @@ strip = false
# --extract-json payload (with float_roundtrip) on every `cargo test`.
# - wasm-interpreter/checked: every wasmgen parity test executes an emitted
# blob under this pure-Rust interpreter.
# - the resvg rasterization stack: the `diagram::render_png` tests rasterize
# a diagram through resvg, and unoptimized SVG filter code dominates them
# (~80% of the slowest one is `resvg::filter::morphology::apply`, per
# `perf`, with the un-inlined `u8::max` inside it alone at ~35%). Pinning
# the stack takes those 16 tests from 2.52s to 0.33s and drops the engine's
# lib-test binary below its previous one-test parallel floor.
# - salsa/hashbrown/indexmap: the incremental compilation substrate. Every
# model any test compiles goes through salsa's query engine and its
# indexmap-backed dependency edges, so this is the one pin that is spread
# across the whole suite rather than concentrated in a few tests: it cuts
# ~17% of BOTH engine test binaries' CPU. Note this `hashbrown` is the
# standalone crate behind indexmap and salsa, not the copy vendored into
# std, so `std::collections::HashMap` is unaffected.
#
# The asymmetry that makes these worth it: a dependency is not recompiled by an
# edit to a workspace crate, so a pin costs one rebuild of that dependency (and
# whatever sits above it) and then nothing at all on the edit-compile-test loop.
# Measured on the render stack: no change to a cold `cargo test -p simlin-engine
# --no-run` (61.3s vs 58.7s, inside run-to-run noise). Pinning a WORKSPACE crate
# is a different trade and is deliberately not done here.
[profile.dev.package.serde_json]
opt-level = 3

Expand All @@ -43,3 +63,39 @@ opt-level = 3

[profile.dev.package.checked]
opt-level = 3

[profile.dev.package.resvg]
opt-level = 3

[profile.dev.package.usvg]
opt-level = 3

[profile.dev.package.tiny-skia]
opt-level = 3

[profile.dev.package.tiny-skia-path]
opt-level = 3

[profile.dev.package.png]
opt-level = 3

[profile.dev.package.rustybuzz]
opt-level = 3

[profile.dev.package.ttf-parser]
opt-level = 3

[profile.dev.package.fontdb]
opt-level = 3

[profile.dev.package.svgtypes]
opt-level = 3

[profile.dev.package.salsa]
opt-level = 3

[profile.dev.package.hashbrown]
opt-level = 3

[profile.dev.package.indexmap]
opt-level = 3
14 changes: 14 additions & 0 deletions deny.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,20 @@ ignore = [
# entry, purely a maintenance-status advisory -- no code-execution or
# memory-safety risk.
"RUSTSEC-2026-0206",
# bitmaps, im and sized-chunks are unmaintained -- one author's cluster,
# all three declared at once, all reached through the same single edge:
# simlin-serve -> loro -> loro-internal -> im -> {bitmaps, sized-chunks}.
# Each advisory states "No safe upgrade is available"; im's own
# announcement points at the imbl fork, which is loro's migration to make,
# not ours. Purely maintenance-status advisories -- no code-execution or
# memory-safety claim in any of the three.
#
# Revisit when loro drops im (watch loro-internal's dependency on it): all
# three go together, because bitmaps and sized-chunks are only here as im's
# dependencies.
"RUSTSEC-2026-0247",
"RUSTSEC-2026-0248",
"RUSTSEC-2026-0251",
]

[licenses]
Expand Down
Loading
Loading