fix: use --backend=symlink on Linux (hardlink backend EPERMs in docker CI) - #96
fix: use --backend=symlink on Linux (hardlink backend EPERMs in docker CI)#96ebauger wants to merge 2 commits into
Conversation
bun's default hardlink backend fails with EPERM on overlay2 filesystems (Docker, CI), so bun2nix builds could not run on Linux. The darwin path already forced --backend=symlink; use it everywhere. Symlink works on any filesystem.
15ac261 to
e732ee0
Compare
baileylu121
left a comment
There was a problem hiding this comment.
Thanks for the PR, I appreciate the effort, but I just had a couple questions on the proposed changeset.
| { | ||
| fetchBunDeps.extractPackage = pkgs.writeShellApplication { | ||
| name = "extract-bun-package"; | ||
| excludeShellChecks = [ "SC2148" ]; |
There was a problem hiding this comment.
I'm unsure how this would fail, since a shebang is hard coded into the generated script, could you tell me more about how you encountered this?
There was a problem hiding this comment.
Thanks a lot for pushing back on this one — you were right to be skeptical, and I owe you an apology: I should have shipped a reproducible example with this PR instead of asking you to trust a CI log I didn't share.
I went back and re-verified properly. I rebuilt the exact derivation that had failed for us (writeShellApplication { name = "bun2nix"; text = ""; }, same nixpkgs pin, aarch64-linux) in a fresh nixos/nix:2.24.9 container — the very image our CI was running at the time — and… it builds fine. Shebang present, shellcheck green, exactly as you said.
What we had captured back then was real (shellcheck reporting SC2148 against an empty line 1 — the generated file had lost its whole text), but it turns out to have been an artifact of that CI environment: the same stale image was also instant-failing __structuredAttrs FODs with empty logs, and every failure in that class vanished once we bumped to a current nix image. Not a bun2nix bug at all.
I'll drop both shellcheck commits and rebase this PR down to just the install-backend change. Sorry for the noise, and thanks for making me re-check.
| [ | ||
| "--linker=isolated" | ||
| ]; | ||
| bunDefaultInstallFlags = [ |
There was a problem hiding this comment.
Hard links are usually better for performance and it was a significant enough difference in the past that it made sense as a default considering the majority of machines with nix are run in places that wouldn't encounter any issue (i.e. outside of docker), and this is trivial enough to configure downstream in the case you encounter issues.
There was a problem hiding this comment.
That's totally fair, and again — my bad for opening this PR without a reproduction you could run. Let me fix that now, because I was able to reproduce it on GitLab, in a form you can clone:
https://gitlab.com/ebauger/bun-hardlink-repro
It uses upstream bun2nix (not my fork) with a neutral 30-dependency package.json, and runs four jobs on GitLab.com SaaS runners (docker executor), amd64 and arm64 (pipeline):
-
hook-hardlink-*— a plainmkDerivationwith upstream defaults (on Linux the hook passes no--backend, so bun picks hardlink) — fails:repro-hardlink> bun install flags: --linker=isolated --ignore-scripts repro-hardlink> EPERM: Operation not permitted: failed to link package: @babel/core@7.24.7 (link) repro-hardlink> EPERM: Operation not permitted: failed to link package: axios@1.7.2 (link) ... repro-hardlink> Failed to install 604 packages -
hook-symlink-*— the identical derivation withbunInstallFlags = "--linker=isolated --backend=symlink"— green.
One thing I found genuinely interesting while narrowing it down (it had me doubting myself for a while): plain bun install --backend=hardlink with a bun-populated cache passes on those same runners, even at the same package count. The EPERM only appears when installing from the fetchBunDeps-extracted cache — so it's specifically bun2nix's own hook path that trips in docker-executor CI, not generic "bun in docker" flakiness. Which also means a Linux consumer on any docker-based CI hits a wall of several hundred opaque EPERMs by default, with nothing in the output pointing at the backend (our real ~600-package workspace failed identically).
Totally agreed it's configurable downstream — small note though: bunInstallFlags replaces the default array, so the consumer also has to know to re-supply --linker=isolated, after first tracing the EPERMs back to the backend choice.
I'm happy to go whichever way you prefer:
- keep this patch (symlink everywhere — darwin already does it for an analogous backend-reliability reason), or
- keep the hardlink default for perf, and I rework this PR into a troubleshooting docs entry with the exact EPERM signature and the flags to set.
Either works for us — I mostly want the next docker-CI user to not lose a day on this. Thanks for maintaining bun2nix, it's been a pleasure to build on.
e732ee0 to
a73d8ee
Compare
What
One small fix so
bun2nix.mkDerivationbuilds on Linux CI: pass--backend=symlinkto theoffline
bun installon Linux too (nix/mk-derivation/hook.nix).Today the flag is only set on macOS. On Linux bun falls back to its default hardlink
backend, which fails with
EPERM: ... failed to link packagefor every package when thebuild runs on a docker-executor CI runner (e.g. GitLab.com SaaS runners). Symlink works on
any filesystem and is what the macOS path already used, so this merges the two branches
into one.
Reproduction
https://gitlab.com/ebauger/bun-hardlink-repro — upstream bun2nix (this repo, not my fork),
a neutral 30-dependency package set, four jobs on GitLab.com SaaS runners (amd64 + arm64):
the default-backend jobs fail with ~600
EPERM: Operation not permitted: failed to link package: … (link), the identical derivations with--backend=symlinkare green.Interestingly, plain
bun install --backend=hardlinkwith a bun-populated cache passes onthe same runners — the EPERM is specific to installing from the
fetchBunDeps-extractedcache, i.e. it's precisely bun2nix's hook path that trips it.
Notes
nix fmt.bun installbackend).(see review discussion) those turned out to be an artifact of a stale CI nix image on our
side, not a bun2nix bug — dropped.