Kernel perf: fix opt-level, stop allocating — and draw a fractal

crates/loon-kernel/Cargo.toml

@@ -15,5 +15,9 @@ panic = "abort"
 
 [profile.release]
 panic = "abort"
-opt-level = "z"
+# An interpreter lives or dies on inlining, and `opt-level = "z"` suppresses
+# it — the dispatch loop was ~150x slower than a native one under `z`. The
+# image is small either way; there is nothing here worth trading speed for.
+opt-level = 3
 lto = true
+codegen-units = 1

crates/loon-kernel/Makefile

@@ -21,12 +21,17 @@ run: build
 host:
 	cd ../.. && cargo run -q -p loon-cli -- run crates/loon-kernel/boot/init.oo
 
-# Boot, and prove the machine agrees with the host byte for byte.
+# Boot, and prove the machine agrees with the host byte for byte — for init,
+# and for the fractal (which is the float path's parity check in disguise).
 check: build
 	@$(QEMU) $(QFLAGS) -serial mon:stdio -kernel $(KERNEL) < /dev/null 2>/dev/null \
-	  | sed -n '/^hello from loon/,/^init done/p' | tr -d '\r' > /tmp/loon-metal.txt
+	  | tr -d '\r' > /tmp/loon-metal-all.txt
+	@sed -n '/^hello from loon/,/^init done/p' /tmp/loon-metal-all.txt > /tmp/loon-metal.txt
+	@sed -n '/^@@@@@@@@@@@@%/,/^@@@@@@@@@@@%%%%%%%%####/p' /tmp/loon-metal-all.txt > /tmp/loon-metal-mandel.txt
 	@cd ../.. && cargo run -q -p loon-cli -- run crates/loon-kernel/boot/init.oo 2>/dev/null > /tmp/loon-host.txt
+	@cd ../.. && cargo run -q -p loon-cli -- run crates/loon-kernel/boot/mandel.oo 2>/dev/null > /tmp/loon-host-mandel.txt
 	@diff /tmp/loon-host.txt /tmp/loon-metal.txt \
+	  && diff /tmp/loon-host-mandel.txt /tmp/loon-metal-mandel.txt \
 	  && echo "ok: identical output on the host and on bare metal"
 
 clean:

crates/loon-kernel/README.md

@@ -24,6 +24,7 @@ make check    # boot it and diff the two
 | `src/sbi.rs` | the slice of SBI we need (power off) |
 | `src/eir/` | boot-image decoder and the EIR interpreter |
 | `boot/init.oo` | the init program — ordinary Loon |
+| `boot/mandel.oo` | a Mandelbrot set, because a kernel that boots should get to do one gratuitous thing |
 
 The host toolchain is not in this crate's build graph. `build.rs` shells out
 to `loon image`, which compiles `boot/init.oo` to a boot image; the kernel
@@ -32,7 +33,8 @@ checker, ownership, lowering — stays on the host, where it belongs.
 
 ## Why the output has to match
 
-`make check` diffs the machine against the host. That diff is the point of
+`make check` diffs the machine against the host — init *and* the fractal,
+which doubles as the float path's parity check. That diff is the point of
 the exercise: the same program, the same effects, two entirely different
 bottom halves. If they ever disagree, one of the two runtimes is wrong about
 what Loon means, and a language whose semantics depend on where it runs is
@@ -46,6 +48,47 @@ outward, which only works if capturing moves every handler at or above the
 prompt into the continuation. `boot/init.oo` exercises forwarding, aborting
 (a clause that never resumes) and non-tail resume for exactly this reason.
 
+## Benchmarking
+
+```bash
+./bench.sh 7     # boot 7 times, report the best result per benchmark
+```
+
+`boot/bench.oo` (calls and sequences) and `boot/loop.oo` (tail self-recursion,
+which lowers to a jump and so never pushes a frame) run before init and report
+ops, ns/op and allocation count. Op counts come from the VM's own dispatch
+counter and allocation counts from the global allocator, so both are exact and
+identical run to run. **Wall-clock is not:** it is measured inside an emulator
+on a shared machine and only ever biased upward, which is why `bench.sh`
+reports a minimum over several runs. Treat a timing difference under ~30% as
+noise unless you can reproduce it by interleaving two builds.
+
+## Performance notes
+
+Two things were measured properly, and one of them was a surprise.
+
+**`opt-level` dominates everything.** The crate was scaffolded with
+`opt-level = "z"`, which suppresses the inlining a dispatch loop depends on.
+Switching to `3` was worth **3.3x** on `bench` and 1.5x on `loop`, measured
+back to back on the same machine. Nothing else came close.
+
+**Allocation was never the bottleneck.** Pooling register files and keeping
+operands off the heap took the `bench` workload from 89,858 allocations to 47,
+and bought *no measurable time* — interleaved A/B runs put the two builds
+inside each other's noise. The changes are kept because a kernel that
+interprets in near-constant memory is worth having on its own terms: no
+allocator pressure, no fragmentation over long uptimes, no dependence on a
+heap that has no OOM killer behind it. They are not a speedup, and an earlier
+claim that they would be was wrong.
+
+**A slab allocator is therefore not worth building.** There is nothing left
+for it to allocate, and even at 90k allocations the existing first-fit list
+was not costing measurable time.
+
+What remains is roughly 500 ns/op against ~7 ns/op for a minimal native
+dispatch loop under the same emulator. That gap is real and unexplained;
+chasing it needs an idle machine and a profiler, not more guessing.
+
 ## Known limits
 
 - **Cooperative only.** No timer interrupt yet, so a pure loop owns the
@@ -54,6 +97,6 @@ prompt into the continuation. `boot/init.oo` exercises forwarding, aborting
   races with it. SMP needs a real one.
 - **Partial builtin set.** Intrinsics the runtime lacks raise a loud error
   naming the builtin; they never silently return `()`.
-- **Slow.** Roughly 0.9 µs per interpreted op — the interpreter allocates a
-  register file per call and an operand vector per op, on a first-fit
-  allocator that costs ~0.5 µs per allocation. Nothing here is tuned yet.
+- **Slow.** Roughly 500 ns per interpreted op under emulation, against ~7 ns
+  for a minimal native dispatch loop on the same emulator. See the
+  performance notes above for what that is and is not.

crates/loon-kernel/bench.sh

@@ -0,0 +1,20 @@
+#!/bin/sh
+# Run the kernel N times and report the best result for each benchmark.
+# Wall-clock under emulation is noisy and only ever biased upward by load, so
+# the minimum is the least contaminated estimate. Allocation counts are
+# deterministic and identical across runs.
+N=${1:-7}
+K=target/riscv64gc-unknown-none-elf/release/loon-kernel
+for i in $(seq 1 "$N"); do
+  timeout 300 qemu-system-riscv64 -machine virt -cpu rv64 -smp 1 -m 128M \
+    -nographic -serial mon:stdio -bios default -kernel "$K" < /dev/null 2>/dev/null
+done | awk '
+  /^(loop|bench)/ {
+    name = $1; sub(/:$/, "", name)
+    for (i = 1; i <= NF; i++) if ($i == "ns/op,") { ns = $(i-1)+0 }
+    for (i = 1; i <= NF; i++) if ($i == "allocs") { al = $(i-1)+0 }
+    if (!(name in best) || ns < best[name]) best[name] = ns
+    allocs[name] = al
+  }
+  END { for (n in best) printf "%-6s min %5d ns/op   %7d allocs\n", n, best[n], allocs[n] }
+' | sort

crates/loon-kernel/boot/bench.oo

@@ -0,0 +1,15 @@
+; Interpreter benchmark. Deliberately IO-free: console bytes are MMIO traps
+; into the emulator and would swamp what we are trying to measure.
+;
+; Two shapes that stress different paths — deep recursion (call frames, the
+; register file) and sequence work (operand vectors, closure application).
+
+[fn fib [n]
+  [if [< n 2] n [+ [fib [- n 1]] [fib [- n 2]]]]]
+
+[fn main []
+  [let a [fib 20]]
+  [let xs [map [range 0 400] [fn [n] [* n 3]]]]
+  [let ys [filter xs [fn [n] [> n 200]]]]
+  [let b [fold ys 0 [fn [acc n] [+ acc n]]]]
+  [+ a b]]

crates/loon-kernel/boot/loop.oo

@@ -0,0 +1,7 @@
+; Pure dispatch cost: tail self-recursion lowers to `Recur` (a jump to block
+; zero), so this never pushes a frame. Whatever this costs per op is the
+; interpreter's floor; the gap to bench.oo is what calling costs.
+[fn spin [n acc]
+  [if [= n 0] acc [spin [- n 1] [+ acc n]]]]
+
+[fn main [] [spin 20000 0]]

crates/loon-kernel/boot/mandel.oo

@@ -0,0 +1,32 @@
+; The kernel draws a fractal. Every float op, every closure application,
+; every string concat here runs on the bare-metal interpreter — no libm,
+; no libc, no OS. It is a Mandelbrot set because a kernel that boots
+; should get to do at least one gratuitous thing.
+;
+;   loon run crates/loon-kernel/boot/mandel.oo   ; host
+;   make -C crates/loon-kernel run             ; machine (runs after init)
+
+[fn escape [zr zi cr ci n]
+  [if [or [> [+ [* zr zr] [* zi zi]] 4.0] [>= n 24]]
+    n
+    [escape [+ [- [* zr zr] [* zi zi]] cr] [+ [* 2.0 [* zr zi]] ci] cr ci [+ n 1]]]]
+
+[fn shade [n]
+  [if [>= n 24] " "
+  [if [< n 2] "@"
+  [if [< n 3] "%"
+  [if [< n 4] "#"
+  [if [< n 6] "*"
+  [if [< n 9] "+"
+  [if [< n 13] "="
+  [if [< n 18] "-" "."]]]]]]]]]
+
+[fn pixel [x y]
+  [let cr [- [* [float [str x]] 0.0375] 2.1]]
+  [let ci [- [* [float [str y]] 0.09] 1.1]]
+  [shade [escape 0.0 0.0 cr ci 0]]]
+
+[fn row [y] [join [map [range 0 80] [fn [x] [pixel x y]]] ""]]
+
+[fn main []
+  [each [range 0 25] [fn [y] [println [row y]]]]]

crates/loon-kernel/build.rs

@@ -5,28 +5,39 @@
 //! loon-lang directly) keeps the host toolchain entirely out of the
 //! bare-metal build graph.
 
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
 use std::process::Command;
 
 fn main() {
     let manifest = PathBuf::from(std::env::var("CARGO_MANIFEST_DIR").unwrap());
-    let src = manifest.join("boot/init.oo");
-    let out = PathBuf::from(std::env::var("OUT_DIR").unwrap()).join("init.img");
     let workspace = manifest.join("../../Cargo.toml");
-
-    println!("cargo:rerun-if-changed={}", src.display());
     println!("cargo:rerun-if-changed=build.rs");
 
+    for (name, var) in [
+        ("init", "LOON_BOOT_IMAGE"),
+        ("bench", "LOON_BENCH_IMAGE"),
+        ("loop", "LOON_LOOP_IMAGE"),
+        ("mandel", "LOON_MANDEL_IMAGE"),
+    ] {
+        let src = manifest.join(format!("boot/{name}.oo"));
+        let out = PathBuf::from(std::env::var("OUT_DIR").unwrap()).join(format!("{name}.img"));
+        println!("cargo:rerun-if-changed={}", src.display());
+        image(&workspace, &manifest, &src, &out);
+        println!("cargo:rustc-env={var}={}", out.display());
+    }
+}
+
+fn image(workspace: &Path, manifest: &Path, src: &Path, out: &Path) {
     let status = Command::new(std::env::var("CARGO").unwrap_or_else(|_| "cargo".into()))
         // Run from the workspace root: this crate's .cargo/config.toml pins
         // a bare-metal target, and the nested build must not inherit it.
         .current_dir(manifest.join("../.."))
         .args(["run", "-q", "--manifest-path"])
-        .arg(&workspace)
+        .arg(workspace)
         .args(["-p", "loon-cli", "--", "image"])
-        .arg(&src)
+        .arg(src)
         .arg("-o")
-        .arg(&out)
+        .arg(out)
         // Cargo's env leaks the bare-metal target into the nested build and
         // makes it try to compile the compiler for riscv; clear it.
         .env_remove("CARGO_ENCODED_RUSTFLAGS")
@@ -38,5 +49,4 @@ fn main() {
     if !status.success() {
         panic!("building the boot image from {} failed", src.display());
     }
-    println!("cargo:rustc-env=LOON_BOOT_IMAGE={}", out.display());
 }