Add Mac metal MLX inference#50

Open

Ma-Dan wants to merge 16 commits into

Robbyant:mainfrom

Commits on Apr 26, 2026

Add Mac metal MLX inference

Dan
authored and
Dan
committed

Commits on Apr 27, 2026

Merge branch 'main' into mac_mlx

Dan
authored and
Dan
committed

Commits on Apr 29, 2026

Fix float16 scipy error; cap k_special growth

Dan
authored and
Dan
committed
Speed up _bilinear: precompute flat gather indices, cache as MLX arrays

Dan
authored and
Dan
committed
Merge branch 'main' into mac_mlx

Dan
authored and
Dan
committed
Add mx.compile steady-state fast path to CausalAttentionMLX

Dan
authored and
Dan
committed

Commits on Apr 30, 2026

Reduce per-frame overhead: eliminate k_cat temp, cache RoPE, drop eval sync points

Dan
authored and
Dan
committed
Remove redundant mx.eval(patch_tokens) sync; add steady-state bench tools

Dan
authored and
Dan
committed
Remove mx.eval(agg_list) mid-forward sync; add bench_e2e.py

Dan
authored and
Dan
committed
Update demo.py help text: document scale-frames as speed lever, fix kv-sliding-window token count

Dan
authored and
Dan
committed
Merge branch 'main' into mac_mlx

Dan
authored and
Dan
committed
Fix tokens_per_frame to use actual image dimensions (was assuming square)

Dan
authored and
Dan
committed
Correct scale-frames help text; add CLI args to bench_e2e.py; fix tokens_per_frame for non-square images

Dan
authored and
Dan
committed
Update help text: document sw+sf interaction, add combined speedup examples

Dan
authored and
Dan
committed

Commits on May 1, 2026

Add mlx optional dependency extra and guard import with clear error

Dan
authored and
Dan
committed
Add mlx_infer README

Dan
authored and
Dan
committed