Skip to content

fix(engine): audio scratch files Windows refuses to delete no longer fail a render - #6474

Merged
miguel-heygen merged 9 commits into
mainfrom
fix/audio-scratch-eperm
Oct 27, 2026
Merged

miguel-heygen merged 9 commits into
mainfrom
fix/audio-scratch-eperm

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

What was wrong

On Windows, render with audio failed with Audio mix failed; output would be video-only: EPERM: operation not permitted (0.8.198 and 0.8.199 reports; people got a render only after pointing TEMP/TMP at a folder they could write to). The mix writes its pieces into scratch folders under the system temp folder and tidies them up in finally blocks, with a plain rmSync or renameSync. Where the temp folder refuses deletes (a locked-down or protected TEMP), that tidy-up throws, and the throw replaces a mix that had already worked.

I reproduced it in CI on a Windows runner with the released 0.8.204: TEMP set to a folder that denies delete on its children (icacls /deny (OI)(CI)(DE,DC)), one project with an audio track. It fails with EPERM: operation not permitted, unlink. With the first half of this change the failure moves to EPERM: operation not permitted, rename, which is the reported message; with all of it, the same render finishes with a video and an audio stream.

Fix

  • removeScratch (engine, new, 11 lines) removes a scratch file or folder and leaves it behind if the system refuses. The mix's scratch folders, the PCM source the true-peak guard reads, the AAC encode folder, the audio effect host folder and the pad/trim scratch all use it, so no tidy-up can fail a finished mix.
  • renameWithRetry/renameWithRetryAsync take { copyFallback: true }: after the existing retries are spent on Windows, copy the file to the target and try to remove the source, instead of failing. It is off by default, so the atomic publishers (the output file, the baked volume WAV, the caches, the plan files) keep rename-or-throw. Only the three audio scratch renames opt in: the summed group track and the kept encoder input in audioMixer, and the true-peak corrected file in audioPadTrim. If the copy fails, the rename error is thrown unchanged.
  • The audio stage logs [audio] scratch left behind with a count when a removal was refused. The count is per run (collectScratchLeftovers, an async-local collector around the stage), so concurrent renders do not see each other's leftovers and nothing accumulates outside a run.

Design calls:

  • Leftover scratch is the cheaper failure: the render's own temp-folder cleanup takes it later, whereas an audio-less video is a wrong result.
  • The copy is not atomic, which is why it is opt-in and limited to scratch files that sit inside the render's work folder: a failed copy can only cost that run's own audio, which then fails as before.
  • Not changed: removals that report a real problem (the output file itself, the project folder).

Why scratch is not moved under the render's work folder

Every audio scratch folder except one (the audio effect host page, now covered by removeScratch) is already created inside the render's own work folder, next to the mixed audio. That work folder is itself created under the system temp folder on Windows and macOS (resolveRenderWorkDirPrefix), so a temp folder that denies delete denies the work folder's cleanup too: moving the scratch changes nothing there. Where a removal is refused, removeScratch now records the path and the audio stage logs [audio] scratch left behind with the count once per render, so a leak is visible instead of silent. Whether the render's work folder should live somewhere else on Windows is a separate decision and not part of this PR.

Test

What I did NOT exercise

A real user machine and the exact reported environment (the reporters' TEMP restriction could differ from an ACL that denies delete). The CI check shows the delete-denied case only; other causes of the same rename error (a held file that never clears) are covered by the copy fallback in principle, not by a run.

Why under 100 lines

About 40 lines of source and 50 of tests; the rest is the two import lines of each touched file. It does not fold into other open PRs: it shares no file with them.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 2 blocker · 1 important · 0 nits · head f465e9cb (pinned 47204d20 has been superseded)

  • blocker (CI): the required Preflight (lint + format) fails oxfmt --check on packages/engine/src/utils/renameWithRetry.test.ts, which means regression, Render on windows-latest and Tests on windows-latest can't pass. 47204d20 failed the same format:check. Run oxfmt on that file.
Scope

I read the full diff and checked all 20+ renameWithRetry callers in core/engine/producer, the audio-path rmSync sites, and the 47204d20 → f465e9cb delta. The removeScratch swaps are fine: each one is a best-effort cleanup in a finally or after the result.

— Genesis

Comment thread packages/engine/src/utils/renameWithRetry.ts Outdated
Comment thread packages/engine/src/utils/removeScratch.ts Outdated

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES — 1 resolved (format), 2 open, 0 new · f465e9cb → ce894665. The shared-helper copy fallback (blocker, renameWithRetry.ts:46) and the process-wide left list (removeScratch.ts:3) haven't changed.

— Genesis

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 155 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (2)

  • seqplainkeys-none-center-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqplainkeys-none-px-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES — 0 resolved, 2 open, 0 new · ce894665 → 63917f64. The nextStep split only refactors the loop. The copy fallback is still the default in the shared renameWithRetry/renameWithRetryAsync, so it still applies to artifactTransaction.commit and the other atomic publishers (blocker, renameWithRetry.ts). The process-wide left list in removeScratch.ts:3 is unchanged. If you'd rather keep the fallback global, please reply on that thread with the reasoning.

— Genesis

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 2 resolved, 0 open, 0 new · 63917f64 → 7b5892be. The copy fallback is now opt-in (copyFallback) and used only at the three audio scratch renames. A failed copy in enforceAacTruePeak returns success: false, as before. Leftovers are scoped to each run with AsyncLocalStorage. Every run on this head is green. (b7d8f1bd failed format; this head only drops a blank line.)

— Genesis

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit dcec31b Oct 27, 2026
76 checks passed
@miguel-heygen
miguel-heygen deleted the fix/audio-scratch-eperm branch October 27, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant