Skip to content

fix: use --backend=symlink on Linux (hardlink backend EPERMs in docker CI) - #96

Open
ebauger wants to merge 2 commits into
nix-community:masterfrom
zbranchio:fix/linux-builder-support
Open

fix: use --backend=symlink on Linux (hardlink backend EPERMs in docker CI)#96
ebauger wants to merge 2 commits into
nix-community:masterfrom
zbranchio:fix/linux-builder-support

Conversation

@ebauger

@ebauger ebauger commented Jul 8, 2026

Copy link
Copy Markdown

What

One small fix so bun2nix.mkDerivation builds on Linux CI: pass --backend=symlink to the
offline bun install on Linux too (nix/mk-derivation/hook.nix).

Today the flag is only set on macOS. On Linux bun falls back to its default hardlink
backend, which fails with EPERM: ... failed to link package for every package when the
build runs on a docker-executor CI runner (e.g. GitLab.com SaaS runners). Symlink works on
any filesystem and is what the macOS path already used, so this merges the two branches
into one.

Reproduction

https://gitlab.com/ebauger/bun-hardlink-repro — upstream bun2nix (this repo, not my fork),
a neutral 30-dependency package set, four jobs on GitLab.com SaaS runners (amd64 + arm64):
the default-backend jobs fail with ~600 EPERM: Operation not permitted: failed to link package: … (link), the identical derivations with --backend=symlink are green.
Interestingly, plain bun install --backend=hardlink with a bun-populated cache passes on
the same runners — the EPERM is specific to installing from the fetchBunDeps-extracted
cache, i.e. it's precisely bun2nix's hook path that trips it.

Notes

bun's default hardlink backend fails with EPERM on overlay2 filesystems (Docker, CI), so bun2nix builds could not run on Linux. The darwin path already forced --backend=symlink; use it everywhere. Symlink works on any filesystem.
@ebauger
ebauger force-pushed the fix/linux-builder-support branch from 15ac261 to e732ee0 Compare July 8, 2026 21:21
@ebauger
ebauger marked this pull request as ready for review July 8, 2026 21:27

@baileylu121 baileylu121 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, I appreciate the effort, but I just had a couple questions on the proposed changeset.

Comment thread nix/fetch-bun-deps/extract-package.nix Outdated
{
fetchBunDeps.extractPackage = pkgs.writeShellApplication {
name = "extract-bun-package";
excludeShellChecks = [ "SC2148" ];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unsure how this would fail, since a shebang is hard coded into the generated script, could you tell me more about how you encountered this?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for pushing back on this one — you were right to be skeptical, and I owe you an apology: I should have shipped a reproducible example with this PR instead of asking you to trust a CI log I didn't share.

I went back and re-verified properly. I rebuilt the exact derivation that had failed for us (writeShellApplication { name = "bun2nix"; text = ""; }, same nixpkgs pin, aarch64-linux) in a fresh nixos/nix:2.24.9 container — the very image our CI was running at the time — and… it builds fine. Shebang present, shellcheck green, exactly as you said.

What we had captured back then was real (shellcheck reporting SC2148 against an empty line 1 — the generated file had lost its whole text), but it turns out to have been an artifact of that CI environment: the same stale image was also instant-failing __structuredAttrs FODs with empty logs, and every failure in that class vanished once we bumped to a current nix image. Not a bun2nix bug at all.

I'll drop both shellcheck commits and rebase this PR down to just the install-backend change. Sorry for the noise, and thanks for making me re-check.

[
"--linker=isolated"
];
bunDefaultInstallFlags = [

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard links are usually better for performance and it was a significant enough difference in the past that it made sense as a default considering the majority of machines with nix are run in places that wouldn't encounter any issue (i.e. outside of docker), and this is trivial enough to configure downstream in the case you encounter issues.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's totally fair, and again — my bad for opening this PR without a reproduction you could run. Let me fix that now, because I was able to reproduce it on GitLab, in a form you can clone:

https://gitlab.com/ebauger/bun-hardlink-repro

It uses upstream bun2nix (not my fork) with a neutral 30-dependency package.json, and runs four jobs on GitLab.com SaaS runners (docker executor), amd64 and arm64 (pipeline):

  • hook-hardlink-* — a plain mkDerivation with upstream defaults (on Linux the hook passes no --backend, so bun picks hardlink) — fails:

    repro-hardlink> bun install flags: --linker=isolated --ignore-scripts
    repro-hardlink> EPERM: Operation not permitted: failed to link package: @babel/core@7.24.7 (link)
    repro-hardlink> EPERM: Operation not permitted: failed to link package: axios@1.7.2 (link)
    ...
    repro-hardlink> Failed to install 604 packages
    
  • hook-symlink-* — the identical derivation with bunInstallFlags = "--linker=isolated --backend=symlink"green.

One thing I found genuinely interesting while narrowing it down (it had me doubting myself for a while): plain bun install --backend=hardlink with a bun-populated cache passes on those same runners, even at the same package count. The EPERM only appears when installing from the fetchBunDeps-extracted cache — so it's specifically bun2nix's own hook path that trips in docker-executor CI, not generic "bun in docker" flakiness. Which also means a Linux consumer on any docker-based CI hits a wall of several hundred opaque EPERMs by default, with nothing in the output pointing at the backend (our real ~600-package workspace failed identically).

Totally agreed it's configurable downstream — small note though: bunInstallFlags replaces the default array, so the consumer also has to know to re-supply --linker=isolated, after first tracing the EPERMs back to the backend choice.

I'm happy to go whichever way you prefer:

  1. keep this patch (symlink everywhere — darwin already does it for an analogous backend-reliability reason), or
  2. keep the hardlink default for perf, and I rework this PR into a troubleshooting docs entry with the exact EPERM signature and the flags to set.

Either works for us — I mostly want the next docker-CI user to not lose a day on this. Thanks for maintaining bun2nix, it's been a pleasure to build on.

@ebauger
ebauger force-pushed the fix/linux-builder-support branch from e732ee0 to a73d8ee Compare July 22, 2026 00:10
@ebauger ebauger changed the title fix: build on Linux (symlink backend + shellcheck SC2148) fix: use --backend=symlink on Linux (hardlink backend EPERMs in docker CI) Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants