Skip to content

Commit 30359f8

Browse files
fix(audio): master bus follow-ups: a scene renders with its own chain; long mixes render in windows (#5474)
* fix(core): a scene rendered on its own takes its master chain from its own root The standalone wrapper for a scene file reused the project index's root attributes, so the index's data-fx-chain became the master bus. The wrapper now carries the scene root's own chain, or none when the scene declares none, as the audio-effects reference describes. — Jerrai * fix(engine): run long audio chains window by window instead of reading the file whole applyAudioFxChain read the whole input WAV into memory, which Node refuses past 2 GiB (a stereo float mix beyond about 93 minutes), and the browser page then held the track twice. Inputs above 512 MiB of samples are now read, rendered and written in 5 minute windows, each with 10 s of preceding and 1 s of following audio rendered and discarded so stateful effects arrive warm. The output header is patched at the end. Inputs below the threshold keep the existing whole-file path. Parameter automation is clocked from the file start and is not windowed. An input past 2 GiB that carries automation now fails with an AudioFxRenderError naming that, instead of Node's raw file-size error. — Jerrai * chore(engine): trim the browser-lease comment to its live invariant — Jerrai * fix(core): a scene mounted as the index's own root takes its own master chain When the index's top-level composition is itself the scene's mount, the standalone extraction returned before the scene's chain was applied, so the index's chain still won. — Jerrai * fix(engine): windowed audio renders follow the file's clock and warm up for the whole chain - Pass each window's start time into the page and on to buildFxChain, and seed the pitch-shift worklet's tap phase from it, so chorus, phaser and pitch shift continue across a window seam instead of restarting from time zero. - Size the pre-roll from the chain (delay repeats, reverb room, dynamics release, to -60 dB), never below the configured floor and never above a cap. - Read a WAV data size of 0 or 0xFFFFFFFF as running to the end of the file, so audio past 4 GiB is no longer dropped. - Close the input descriptor when the output cannot be opened, and name the real cause when an input over 2 GiB cannot be streamed. - Cover reverb, a slow-release compressor, chorus and pitch shift in the windowed-vs-whole test, with an output-length check. — Jerrai * fix(engine): start each audio window on a render quantum and test the seams against a hard chain Windows that begin off a 128-frame block see worklet blocks shifted against a whole render, which left spikes in the tail after the last input sample. Align the window start, and make the windowed-vs-whole test cover modulation, delay, reverb and a slow-release compressor on windows whose start is not zero, failing above 0.001 where it previously tolerated 0.02. — Jerrai * fix(engine): keep a limiter's lookahead, seed bitcrush holds, close-check without /proc - Pad the render source with the limiter's lookahead of silence so the last samples of a clip leave the lookahead line instead of being dropped. - Seed the bitcrush hold counter from the window's start so holds that do not divide the render quantum stay in step across seams, and cover bitcrush in the windowed-vs-whole test. - Check that the input descriptor is closed through the fs mock instead of /proc/self/fd, which only exists on Linux. — Jerrai * test(engine): pick a bitcrush hold the window starts do not divide The earlier hold of 3 divided both window starts, so an unseeded counter passed. — Jerrai * fix(engine): warn at the pre-roll cap, share tail formulas, read a zero data size alike - Warn when a chain's memory exceeds the pre-roll cap, so cut repeats at window seams are not silent. - Export the delay and reverb tail formulas from audio-fx-tail and use them for the window warm-up too. - Both WAV readers now treat a declared data size of 0 or 0xFFFFFFFF as the rest of the file. — Jerrai
1 parent 2922c40 commit 30359f8

11 files changed

Lines changed: 1085 additions & 95 deletions

‎packages/core/src/audio/audioFxGraph.ts‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -253,8 +253,10 @@ function onePoleBuilder(kind: "highpass" | "lowpass"): Builder {
253253
}
254254

255255
function workletBuilder(processor: string, resettable = false): Builder {
256-
return (ctx, p) => {
257-
const node = new AudioWorkletNode(ctx, processor, { processorOptions: { ...p } });
256+
return (ctx, p, elapsed) => {
257+
const node = new AudioWorkletNode(ctx, processor, {
258+
processorOptions: { ...p, __hfElapsed: elapsed },
259+
});
258260
return {
259261
input: node,
260262
output: node,

‎packages/core/src/audio/audioFxTail.test.ts‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
import { describe, expect, it } from "vitest";
2-
import { chainLatencySamples, chainTailSeconds, MAX_FX_TAIL_SECONDS } from "./audioFxTail.js";
2+
import {
3+
chainLatencySamples,
4+
chainTailSeconds,
5+
delayTailSeconds,
6+
MAX_FX_TAIL_SECONDS,
7+
reverbRoomSeconds,
8+
} from "./audioFxTail.js";
39
import { truePeakLatencySamples } from "./audioFxTruePeak.js";
410
import { synthesizeReverbImpulse } from "./audioFxGraph.js";
511
import type { HfAudioFxChain } from "../audioFx.js";
@@ -130,3 +136,15 @@ describe("chainLatencySamples", () => {
130136
);
131137
});
132138
});
139+
140+
describe("tail helpers shared with the engine's window warm-up", () => {
141+
it("counts a delay's repeats to -60 dB, uncapped", () => {
142+
expect(delayTailSeconds(5000, 0.95)).toBeCloseTo(135 * 5, 5);
143+
expect(delayTailSeconds(400, 0.5)).toBeCloseTo(10 * 0.4, 5);
144+
});
145+
146+
it("is the reverb impulse's length", () => {
147+
expect(reverbRoomSeconds(0.5)).toBeCloseTo(1.9, 5);
148+
expect(reverbRoomSeconds(4)).toBeCloseTo(3.2, 5);
149+
});
150+
});

‎packages/core/src/audio/audioFxTail.ts‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ function knobMax(node: HfAudioFxNode, key: string, automation?: HfAutomation): n
5656
* Repeats until a feedback loop falls under the floor, times the gap between
5757
* them. `feedback` is capped below 1 by the registry, so this terminates.
5858
*/
59-
function delayTail(time: number, feedback: number): number {
59+
export function delayTailSeconds(time: number, feedback: number): number {
6060
const gap = Math.min(5, time / 1000);
6161
if (gap <= 0) return 0;
6262
const fb = Math.max(0, Math.min(0.999, feedback));
@@ -66,15 +66,19 @@ function delayTail(time: number, feedback: number): number {
6666

6767
// Exactly the generated impulse's length — see synthesizeReverbImpulse, which
6868
// is the same expression. A convolution is as long as its impulse.
69+
export function reverbRoomSeconds(size: number): number {
70+
return 0.6 + Math.max(0, Math.min(1, size)) * 2.6;
71+
}
72+
6973
function reverbTail(node: HfAudioFxNode, automation?: HfAutomation): number {
7074
return knobMax(node, "wet", automation) > 0
71-
? 0.6 + Math.max(0, Math.min(1, knobMax(node, "size", automation))) * 2.6
75+
? reverbRoomSeconds(knobMax(node, "size", automation))
7276
: 0;
7377
}
7478

7579
function delayNodeTail(node: HfAudioFxNode, automation?: HfAutomation): number {
7680
return knobMax(node, "mix", automation) > 0
77-
? delayTail(knobMax(node, "time", automation), knobMax(node, "feedback", automation))
81+
? delayTailSeconds(knobMax(node, "time", automation), knobMax(node, "feedback", automation))
7882
: 0;
7983
}
8084

‎packages/core/src/audio/audioFxWorklets.ts‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,7 @@ class HfBitcrush extends AudioWorkletProcessor {
192192
this.p = o.processorOptions || {};
193193
this.holds = [];
194194
this.held = [];
195+
this.start = Math.round((this.p.__hfElapsed ?? 0) * sampleRate);
195196
this.port.onmessage = (e) => {
196197
if (e.data && e.data.__hfDispose) { this.dead = true; return; }
197198
this.p = { ...this.p, ...e.data };
@@ -208,7 +209,8 @@ class HfBitcrush extends AudioWorkletProcessor {
208209
for (let ch = 0; ch < i.length; ch++) {
209210
const inp = i[ch], out = o[ch];
210211
if (this.held[ch] === undefined) this.held[ch] = 0;
211-
if (this.holds[ch] === undefined) this.holds[ch] = 0;
212+
// The hold counter runs from the file's first sample, so a node built part-way in starts where it would be.
213+
if (this.holds[ch] === undefined) this.holds[ch] = this.start % step;
212214
for (let n = 0; n < inp.length; n++) {
213215
// Per channel: advancing one shared counter only on the last channel
214216
// left every earlier channel either unheld or frozen for a whole
@@ -255,7 +257,11 @@ class HfPitchshift extends AudioWorkletProcessor {
255257
this.grain = Math.round(sampleRate * 0.1);
256258
this.buf = [];
257259
this.write = 0;
258-
this.phase = 0;
260+
// The taps sweep at a constant rate from the start of the file, so a node built
261+
// part-way in joins the sweep where it would already be.
262+
const ratio0 = Math.pow(2, Math.max(-12, Math.min(12, this.p.semitones ?? 0)) / 12);
263+
const turns = (this.p.__hfElapsed ?? 0) * sampleRate * ((1 - ratio0) / this.grain);
264+
this.phase = turns - Math.floor(turns);
259265
// Samples written so far, capped at one grain. The taps read up to a grain
260266
// behind the write head, so until this fills they would read the ring's
261267
// zeros, so the head of every clip came out attenuated or silent.

‎packages/core/src/compiler/standaloneEntry.test.ts‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import { parseHTML } from "linkedom";
12
import { describe, expect, it } from "vitest";
23
import { extractStandaloneEntryFromIndex } from "./standaloneEntry";
34

@@ -112,4 +113,77 @@ describe("extractStandaloneEntryFromIndex", () => {
112113
expect(extracted).toContain('data-duration="2"');
113114
expect(extracted).not.toContain('data-duration="12"');
114115
});
116+
describe("master bus chain", () => {
117+
const hostIndexHtml = `<!DOCTYPE html>
118+
<html>
119+
<body>
120+
<div id="scene1" data-composition-id="scene1" data-composition-src="compositions/scene1.html" data-width="640" data-height="360" data-fx-chain="{&quot;version&quot;:1,&quot;nodes&quot;:[{&quot;type&quot;:&quot;truepeak&quot;,&quot;id&quot;:&quot;index&quot;}]}"></div>
121+
</body>
122+
</html>`;
123+
const indexHtml = `<!DOCTYPE html>
124+
<html>
125+
<body>
126+
<div data-composition-id="master" data-width="640" data-height="360" data-fx-chain="{&quot;version&quot;:1,&quot;nodes&quot;:[{&quot;type&quot;:&quot;truepeak&quot;,&quot;id&quot;:&quot;index&quot;}]}">
127+
<div id="scene1" data-composition-id="scene1" data-composition-src="compositions/scene1.html" data-start="0"></div>
128+
</div>
129+
</body>
130+
</html>`;
131+
const chainOf = (html: string): string | null => {
132+
const { document } = parseHTML(html);
133+
const roots = Array.from(document.querySelectorAll("[data-composition-id]")) as Element[];
134+
const top = roots.find((el) => !el.parentElement?.closest("[data-composition-id]"));
135+
return top?.getAttribute("data-fx-chain") ?? null;
136+
};
137+
138+
it("carries the scene's own chain, not the index's", () => {
139+
const chain = '{"version":1,"nodes":[{"type":"truepeak","id":"scene"}]}';
140+
const sceneHtml = `<template id="t"><div data-composition-id="scene1" data-width="640" data-height="360" data-fx-chain='${chain}'></div></template>`;
141+
142+
const extracted = extractStandaloneEntryFromIndex(
143+
indexHtml,
144+
"compositions/scene1.html",
145+
sceneHtml,
146+
);
147+
148+
expect(chainOf(extracted ?? "")).toBe(chain);
149+
});
150+
151+
it("drops the index's chain when the scene declares none", () => {
152+
const sceneHtml = `<template id="t"><div data-composition-id="scene1" data-width="640" data-height="360"></div></template>`;
153+
154+
const extracted = extractStandaloneEntryFromIndex(
155+
indexHtml,
156+
"compositions/scene1.html",
157+
sceneHtml,
158+
);
159+
160+
expect(chainOf(extracted ?? "")).toBeNull();
161+
});
162+
163+
it("carries the scene's chain when the index's top-level root is the mount itself", () => {
164+
const chain = '{"version":1,"nodes":[{"type":"truepeak","id":"scene"}]}';
165+
const sceneHtml = `<template id="t"><div data-composition-id="scene1" data-width="640" data-height="360" data-fx-chain='${chain}'></div></template>`;
166+
167+
const extracted = extractStandaloneEntryFromIndex(
168+
hostIndexHtml,
169+
"compositions/scene1.html",
170+
sceneHtml,
171+
);
172+
173+
expect(chainOf(extracted ?? "")).toBe(chain);
174+
});
175+
176+
it("drops the index's chain from a top-level mount when the scene declares none", () => {
177+
const sceneHtml = `<template id="t"><div data-composition-id="scene1" data-width="640" data-height="360"></div></template>`;
178+
179+
const extracted = extractStandaloneEntryFromIndex(
180+
hostIndexHtml,
181+
"compositions/scene1.html",
182+
sceneHtml,
183+
);
184+
185+
expect(extracted).not.toBeNull();
186+
expect(chainOf(extracted ?? "")).toBeNull();
187+
});
188+
});
115189
});

‎packages/core/src/compiler/standaloneEntry.ts‎

Lines changed: 23 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import { parseHTML } from "linkedom";
2+
import { HF_AUDIO_FX_ATTR } from "../audioFx";
23

34
export function isTemplateEntry(html: string): boolean {
45
return html.trimStart().startsWith("<template");
@@ -9,24 +10,36 @@ function normalizeCompositionSrcPath(srcPath: string): string {
910
}
1011

1112
// linkedom has no inert <template> content, so the template's innerHTML is re-parsed.
12-
function readSceneRootDuration(entryHtml: string | undefined): string | null {
13+
function readSceneRoot(entryHtml: string | undefined): Element | null {
1314
if (!entryHtml) return null;
1415
const { document } = parseHTML(entryHtml);
1516
const template = document.querySelector("template");
1617
const scope = template ? parseHTML(template.innerHTML).document : document;
17-
const root = scope.querySelector("[data-composition-id]") as Element | null;
18-
return root?.getAttribute("data-duration") ?? null;
18+
return scope.querySelector("[data-composition-id]") as Element | null;
19+
}
20+
21+
// The scene's own chain replaces the index's, which belongs to the whole project.
22+
function setSceneFxChain(el: Element, sceneFxChain: string | null): void {
23+
if (sceneFxChain) {
24+
el.setAttribute(HF_AUDIO_FX_ATTR, sceneFxChain);
25+
} else {
26+
el.removeAttribute(HF_AUDIO_FX_ATTR);
27+
}
1928
}
2029

2130
function createStandaloneEntryRenderClone(
2231
root: Element,
2332
host: Element,
2433
sceneDuration: string | null,
34+
sceneFxChain: string | null,
2535
): Element {
2636
const hostClone = host.cloneNode(true) as Element;
2737
hostClone.setAttribute("data-start", "0");
2838

29-
if (root === host) return hostClone;
39+
if (root === host) {
40+
setSceneFxChain(hostClone, sceneFxChain);
41+
return hostClone;
42+
}
3043

3144
const rootClone = root.cloneNode(false) as Element;
3245
// The wrapper takes the scene's length, not the whole project's; with none declared
@@ -36,6 +49,7 @@ function createStandaloneEntryRenderClone(
3649
} else {
3750
rootClone.removeAttribute("data-duration");
3851
}
52+
setSceneFxChain(rootClone, sceneFxChain);
3953
rootClone.appendChild(hostClone);
4054
return rootClone;
4155
}
@@ -78,9 +92,12 @@ export function extractStandaloneEntryFromIndex(
7892
if (!root) return null;
7993

8094
// The scene file owns its duration; the mount's window is the fallback.
81-
const sceneDuration = readSceneRootDuration(entryHtml) ?? host.getAttribute("data-duration");
95+
const sceneRoot = readSceneRoot(entryHtml);
96+
const sceneDuration =
97+
sceneRoot?.getAttribute("data-duration") ?? host.getAttribute("data-duration");
98+
const sceneFxChain = sceneRoot?.getAttribute(HF_AUDIO_FX_ATTR) || null;
8299

83-
const renderClone = createStandaloneEntryRenderClone(root, host, sceneDuration);
100+
const renderClone = createStandaloneEntryRenderClone(root, host, sceneDuration, sceneFxChain);
84101
replaceBodyWithRenderClone(body, renderClone);
85102

86103
return document.toString();

‎packages/core/stubs/audio-fx-runtime-entry.ts‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
* ends share a single implementation rather than two that have to be kept in
88
* agreement.
99
*
10-
* Exposes `window.__HF_AUDIO_FX.render(pcm, sampleRate, chain, automation)`,
10+
* Exposes `window.__HF_AUDIO_FX.render(pcm, sampleRate, chain, automation, elapsed)`,
1111
* which returns the processed samples.
1212
*/
1313

@@ -29,6 +29,7 @@ declare global {
2929
sampleRate: number,
3030
chainJson: string,
3131
automationJson?: string,
32+
elapsed?: number,
3233
): Promise<Float32Array[]>;
3334
};
3435
}
@@ -45,6 +46,9 @@ declare global {
4546
* comes from the settings (`chainTailSeconds`), capped, and the returned planes
4647
* are correspondingly longer than what came in — the mixer decides how much of
4748
* that it lets through past the clip's end.
49+
*
50+
* `elapsed` is where in the file `planes` begins, for a window cut from a longer
51+
* input: modulated effects follow the file's clock, not the window's.
4852
*/
4953

5054
/** The clip's audio as an AudioBuffer, a plane per channel. */
@@ -77,6 +81,7 @@ async function render(
7781
sampleRate: number,
7882
chainJson: string,
7983
automationJson?: string,
84+
elapsed = 0,
8085
): Promise<Float32Array[]> {
8186
const chain: HfAudioFxChain = parseAudioFxChain(chainJson);
8287
const channels = Math.max(1, planes.length);
@@ -98,9 +103,11 @@ async function render(
98103
if (chainNeedsWorklets(chain)) await ensureAudioFxWorklets(ctx);
99104

100105
const source = ctx.createBufferSource();
101-
source.buffer = toBuffer(ctx, planes, channels, frames, sampleRate);
106+
// Padded with silence by the limiter's lookahead: once a source ends its worklets stop being
107+
// fed, so without it the last `latency` samples of the clip never leave the lookahead line.
108+
source.buffer = toBuffer(ctx, planes, channels, frames + latency, sampleRate);
102109

103-
const fx = buildFxChain(ctx, chain);
110+
const fx = buildFxChain(ctx, chain, elapsed);
104111

105112
// The input WAV is the clip's own audio from its first sample, so clip-local
106113
// time is offline time — the envelope needs no offset here. Same scheduler as

0 commit comments

Comments
 (0)