Skip to content

Commit d7285e4

Browse files
fix(engine): preserve composition root background in alpha exports (#3976)
* fix(engine): preserve composition root background in alpha exports initTransparentBackground() forced html, body, AND any [data-composition-id] element (the composition root, or a nested sub-composition root) to transparent for every alpha-carrying capture session. That's correct for the HDR two-pass layered DOM pass, where the HDR video itself is the backdrop — but for a plain alpha export (MOV ProRes4444, webm+alpha) of a normal composition, an author's own background painted on the root is real, intentional content, and it was being silently dropped. Add CaptureOptions.clearCompositionRootBackground, threaded through session.options into every general-session call site (default false — preserve the root's authored background). The two HDR-layered-DOM- session call sites set it to true, keeping their existing (correct) behavior. initTransparentBackground's injected stylesheet is now idempotently rewritten rather than insert-once/skip, so a caller's choice is never silently ignored by an earlier call on the same page. Add a producer test fixture + transparency-test.ts check proving a composition root's authored background survives into alpha output, plus unit tests for both flag values and the rewrite-on-repeat-call behavior directly. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * test(producer): register the new transparency fixture with the shard planner transparency-regression-root-bg is exercised by transparency-test.ts directly (asserts a real alpha channel), not by the sharded regression harness, same as its sibling transparency-regression. The shard planner hard-fails on any fixture directory it doesn't recognize as either scheduled or explicitly excluded, so register it in the excluded list with the same reason as its sibling. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * fix(engine): restore explicit transparent-background call for HDR layered sessions The previous commit removed the two HDR-layered-DOM-session call sites' own explicit initTransparentBackground() call, relying on initializeSession()'s internal call (gated behind session.options.format === "png") plus a new clearCompositionRootBackground session option. That was wrong: a plain HDR render's own DOM session captures "jpeg", not "png" — the final HDR MP4/HEVC output doesn't itself carry an alpha channel, only the layered-compositing step does. So initializeSession()'s internal call never fired at all for this common case, and the HDR video layer never got a transparent DOM backdrop to composite into. Reproduced directly against the hdr-regression fixture (part of the regression-shards CI suite): a real local render showed a severe PSNR collapse (~8-56dB) in the "Window B: z-order sandwich" segment, where a DOM element behind the HDR video, the video itself, and a translucent DOM overlay in front all need correct z-ordering — with the composition root's own background no longer cleared, it painted solid over the video slot instead of letting the video composite through. Confirmed via bisection: reverting to the prior commit's files reproduces 0 failed frames (passing); the removed-call version reproduces the exact failure; restoring the explicit calls fixes it again (100/100 checkpoints, 0 failed frames). Restore the explicit, unconditional call in both HDR call sites, right after initializeSession(). clearCompositionRootBackground still does its job for the general/plain-alpha-export path (frameCapture.ts), which is unaffected by this fix. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com> * refactor(engine): let the HDR layered pass own composition-root clearing Drop the CaptureOptions.clearCompositionRootBackground field: every HDR layered DOM session already calls initTransparentBackground right after init, so that call is the one owner. Engine paths use the default, which keeps root backgrounds. * chore(producer): drop comment additions the comment ratchet rejects --------- Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
1 parent 50864d5 commit d7285e4

9 files changed

Lines changed: 182 additions & 50 deletions

File tree

‎packages/engine/src/services/frameCapture.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1486,7 +1486,7 @@ async function constructCaptureSession(
14861486

14871487
// Transparent-background setup is intentionally NOT done here. Chrome resets
14881488
// the default-background-color override on navigation, and the
1489-
// `[data-composition-id]{background:transparent}` stylesheet that
1489+
// `html,body{background:transparent}` stylesheet that
14901490
// `initTransparentBackground` injects must land in a real `document.head`.
14911491
// See `initializeSession()` below — it calls `initTransparentBackground` for
14921492
// PNG captures after `page.goto(...)` and the `window.__hf` readiness poll.

‎packages/engine/src/services/screenshotService.test.ts‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import {
1313
injectVideoFramesBatch,
1414
syncVideoFrameVisibility,
1515
shouldDefaultCaptureBeyondViewport,
16+
initTransparentBackground,
1617
DOM_LAYER_MASK_STYLE_ID,
1718
} from "./screenshotService.js";
1819

@@ -175,6 +176,70 @@ describe("pageContentExceedsCaptureHeight", () => {
175176
});
176177
});
177178

179+
describe("initTransparentBackground", () => {
180+
// evaluate() runs against a linkedom document in globalThis; the cached CDP stub skips createCDPSession().
181+
function makeFakePageWithDom() {
182+
const { document } = parseHTML("<html><body></body></html>");
183+
const send = vi.fn().mockResolvedValue({});
184+
const fakeSession = { send } as unknown as import("puppeteer-core").CDPSession;
185+
const fakePage = {
186+
evaluate: async (fn: (...args: never[]) => void, ...args: never[]) => fn(...args),
187+
} as unknown as Page;
188+
cdpSessionCache.set(fakePage, fakeSession);
189+
const globals = globalThis as unknown as { document?: Document };
190+
const previousDocument = globals.document;
191+
globals.document = document;
192+
return {
193+
page: fakePage,
194+
document,
195+
send,
196+
restore: () => {
197+
globals.document = previousDocument;
198+
},
199+
};
200+
}
201+
202+
it("clears only html/body when clearCompositionRoot is false — an authored composition-root background must survive", async () => {
203+
const { page, document, send, restore } = makeFakePageWithDom();
204+
try {
205+
await initTransparentBackground(page, { clearCompositionRoot: false });
206+
const style = document.getElementById("__hf_transparent_bg__");
207+
expect(style?.textContent).toContain("html,body{");
208+
expect(style?.textContent).not.toContain("[data-composition-id]");
209+
expect(send).toHaveBeenCalledWith("Emulation.setDefaultBackgroundColorOverride", {
210+
color: { r: 0, g: 0, b: 0, a: 0 },
211+
});
212+
} finally {
213+
restore();
214+
}
215+
});
216+
217+
it("also clears the composition root when clearCompositionRoot is true (HDR layered DOM pass)", async () => {
218+
const { page, document, restore } = makeFakePageWithDom();
219+
try {
220+
await initTransparentBackground(page, { clearCompositionRoot: true });
221+
const style = document.getElementById("__hf_transparent_bg__");
222+
expect(style?.textContent).toContain("[data-composition-id]");
223+
} finally {
224+
restore();
225+
}
226+
});
227+
228+
it("a later call replaces the earlier call's rule instead of no-op'ing on the existing style element", async () => {
229+
const { page, document, restore } = makeFakePageWithDom();
230+
try {
231+
await initTransparentBackground(page, { clearCompositionRoot: false });
232+
await initTransparentBackground(page, { clearCompositionRoot: true });
233+
expect(document.querySelectorAll('style[id="__hf_transparent_bg__"]').length).toBe(1);
234+
expect(document.getElementById("__hf_transparent_bg__")?.textContent).toContain(
235+
"[data-composition-id]",
236+
);
237+
} finally {
238+
restore();
239+
}
240+
});
241+
});
242+
178243
describe("injectVideoFramesBatch replacement layout", () => {
179244
it("does not copy opposing inset constraints onto the injected frame image", async () => {
180245
const { window, document } = parseHTML(

‎packages/engine/src/services/screenshotService.ts‎

Lines changed: 21 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -279,43 +279,34 @@ export async function captureScreenshotWithAlpha(
279279
}
280280

281281
/**
282-
* Set the page background to transparent once for a dedicated HDR DOM session.
283-
*
284-
* Call this once after session initialization. Then use captureAlphaPng() per
285-
* frame instead of captureScreenshotWithAlpha() to skip the per-frame CDP
286-
* background override round-trips.
287-
*
288-
* Only use on sessions that are exclusively dedicated to transparent capture
289-
* (e.g., the HDR two-pass DOM layer session) — the background will stay
290-
* transparent for the lifetime of the session.
291-
*
292-
* NOTE on the injected stylesheet: `Emulation.setDefaultBackgroundColorOverride`
293-
* only replaces the *default* page background. Compositions almost always set
294-
* `body { background: ... }` and `#root { background: ... }`, which paint over
295-
* the override and ruin alpha capture for layered HDR compositing — the
296-
* composition root's full-frame background paints across the entire viewport
297-
* and wipes out HDR content captured beneath it.
298-
*
299-
* We force `html`, `body`, and any element marked as a composition root
300-
* (`[data-composition-id]`) to transparent. In HDR layered compositing the HDR
301-
* video itself is the backdrop, so DOM layers must only contribute their
302-
* foreground UI pixels — never a page-spanning solid backdrop.
282+
* Make the page transparent for alpha capture, then use captureAlphaPng() per frame. html/body always clear;
283+
* [data-composition-id] roots (nested too) keep their background unless clearCompositionRoot (HDR layered pass).
284+
* A repeat call rewrites the rule, so the last call wins.
303285
*/
304286
const TRANSPARENT_BG_STYLE_ID = "__hf_transparent_bg__";
305287

306-
export async function initTransparentBackground(page: Page): Promise<void> {
288+
export async function initTransparentBackground(
289+
page: Page,
290+
{ clearCompositionRoot = false }: { clearCompositionRoot?: boolean } = {},
291+
): Promise<void> {
307292
const client = await getCdpSession(page);
308293
await client.send("Emulation.setDefaultBackgroundColorOverride", {
309294
color: { r: 0, g: 0, b: 0, a: 0 },
310295
});
311-
await page.evaluate((styleId: string) => {
312-
if (document.getElementById(styleId)) return;
313-
const style = document.createElement("style");
314-
style.id = styleId;
315-
style.textContent =
316-
"html,body,[data-composition-id]{background:transparent !important;background-color:transparent !important;background-image:none !important;}";
317-
document.head.appendChild(style);
318-
}, TRANSPARENT_BG_STYLE_ID);
296+
await page.evaluate(
297+
(styleId: string, clearCompositionRoot: boolean) => {
298+
const selector = clearCompositionRoot ? "html,body,[data-composition-id]" : "html,body";
299+
let style = document.getElementById(styleId);
300+
if (!style) {
301+
style = document.createElement("style");
302+
style.id = styleId;
303+
document.head.appendChild(style);
304+
}
305+
style.textContent = `${selector}{background:transparent !important;background-color:transparent !important;background-image:none !important;}`;
306+
},
307+
TRANSPARENT_BG_STYLE_ID,
308+
clearCompositionRoot,
309+
);
319310
}
320311

321312
/**

‎packages/producer/src/services/render/stages/captureHdrHybridLoop.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ export async function runHybridLayeredFrameLoop(input: HybridLoopInput): Promise
128128
cfg,
129129
);
130130
await initializeSession(s);
131-
await initTransparentBackground(s.page);
131+
await initTransparentBackground(s.page, { clearCompositionRoot: true });
132132
workerSessions.push(s);
133133
}
134134
const sessions: CaptureSession[] = [domSession, ...workerSessions];

‎packages/producer/src/services/render/stages/captureHdrStage.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,7 +222,7 @@ export async function runCaptureHdrStage(
222222
await initializeSession(domSession);
223223
assertNotAborted();
224224
lastBrowserConsole = domSession.browserConsoleBuffer;
225-
await initTransparentBackground(domSession.page);
225+
await initTransparentBackground(domSession.page, { clearCompositionRoot: true });
226226

227227
// ── Scene detection for shader transitions ──────────────────────────
228228
const transitionMeta: HdrTransitionMeta[] = await domSession.page.evaluate(() => {

‎packages/producer/src/transparency-test.ts‎

Lines changed: 53 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ import { createRenderJob, executeRenderJob } from "./services/renderOrchestrator
3131
const moduleDir = dirname(fileURLToPath(import.meta.url));
3232
const FIXTURE_DIR = resolve(moduleDir, "../tests/transparency-regression");
3333
const FIXTURE_SRC = join(FIXTURE_DIR, "src");
34+
const ROOT_BG_FIXTURE_DIR = resolve(moduleDir, "../tests/transparency-regression-root-bg");
35+
const ROOT_BG_FIXTURE_SRC = join(ROOT_BG_FIXTURE_DIR, "src");
3436
const SHADER_FIXTURE_DIR = resolve(moduleDir, "../tests/page-side-shader-compositor-render-compat");
3537
const SHADER_FIXTURE_SRC = join(SHADER_FIXTURE_DIR, "src");
3638
const SHADER_GOLDEN = join(SHADER_FIXTURE_DIR, "output", "output.mp4");
@@ -43,6 +45,10 @@ const TRANSPARENT_X = 10; // expected fully transparent
4345
const TRANSPARENT_Y = 10;
4446
const OPAQUE_X = 100; // inside the 50–150 red card
4547
const OPAQUE_Y = 100;
48+
// Root-bg fixture: same canvas and red card, but the composition root paints rgb(0,0,255) full-frame,
49+
// so (10,10) must decode opaque blue in alpha output.
50+
const ROOT_BG_X = 10;
51+
const ROOT_BG_Y = 10;
4652

4753
function pixelOffset(x: number, y: number, width: number): number {
4854
return (y * width + x) * 4;
@@ -52,7 +58,7 @@ function assertAlphaPixel(
5258
png: { data: Uint8Array; width: number; height: number },
5359
x: number,
5460
y: number,
55-
expectAlpha: "transparent" | "opaque-red",
61+
expectAlpha: "transparent" | "opaque-red" | "opaque-blue",
5662
label: string,
5763
): void {
5864
assert.equal(png.width, WIDTH, `${label}: width mismatch`);
@@ -68,23 +74,25 @@ function assertAlphaPixel(
6874
0,
6975
`${label}: pixel (${x},${y}) expected fully transparent (alpha=0), got rgba(${r},${g},${b},${a})`,
7076
);
71-
} else {
72-
assert.equal(
73-
a,
74-
255,
75-
`${label}: pixel (${x},${y}) expected fully opaque (alpha=255), got rgba(${r},${g},${b},${a})`,
76-
);
77-
assert.ok(
78-
typeof r === "number" && r >= 240,
79-
`${label}: pixel (${x},${y}) expected red >= 240, got rgba(${r},${g},${b},${a})`,
80-
);
81-
assert.ok(
82-
typeof g === "number" && g <= 30,
83-
`${label}: pixel (${x},${y}) expected green <= 30, got rgba(${r},${g},${b},${a})`,
84-
);
77+
return;
78+
}
79+
assert.equal(
80+
a,
81+
255,
82+
`${label}: pixel (${x},${y}) expected fully opaque (alpha=255), got rgba(${r},${g},${b},${a})`,
83+
);
84+
const expectRed = expectAlpha === "opaque-red";
85+
const dominantName = expectRed ? "red" : "blue";
86+
const dominantValue = expectRed ? r : b;
87+
const otherChannels = expectRed ? { green: g, blue: b } : { red: r, green: g };
88+
assert.ok(
89+
typeof dominantValue === "number" && dominantValue >= 240,
90+
`${label}: pixel (${x},${y}) expected ${dominantName} >= 240, got rgba(${r},${g},${b},${a})`,
91+
);
92+
for (const [name, value] of Object.entries(otherChannels)) {
8593
assert.ok(
86-
typeof b === "number" && b <= 30,
87-
`${label}: pixel (${x},${y}) expected blue <= 30, got rgba(${r},${g},${b},${a})`,
94+
typeof value === "number" && value <= 30,
95+
`${label}: pixel (${x},${y}) expected ${name} <= 30, got rgba(${r},${g},${b},${a})`,
8896
);
8997
}
9098
}
@@ -187,6 +195,30 @@ async function runWebmCheck(workRoot: string): Promise<void> {
187195
console.log("[webm] PASS — transparent + opaque-red pixels verified");
188196
}
189197

198+
async function runRootBackgroundCheck(workRoot: string): Promise<void> {
199+
console.log("\n[root-bg] rendering transparency-regression-root-bg …");
200+
const outDir = join(workRoot, "root-bg");
201+
mkdirSync(outDir, { recursive: true });
202+
const outPath = join(outDir, "out.webm");
203+
204+
const job = createRenderJob({
205+
fps: FPS,
206+
quality: "draft",
207+
format: "webm",
208+
});
209+
210+
await executeRenderJob(job, ROOT_BG_FIXTURE_SRC, outPath);
211+
assert.equal(job.status, "complete", `root-bg render did not complete: status=${job.status}`);
212+
assert.ok(existsSync(outPath), `root-bg output not written to ${outPath}`);
213+
214+
const framePng = join(outDir, "frame-0.png");
215+
await extractFirstFrameFromWebm(outPath, framePng);
216+
const decoded = decodePng(readFileSync(framePng));
217+
assertAlphaPixel(decoded, ROOT_BG_X, ROOT_BG_Y, "opaque-blue", "root-bg");
218+
assertAlphaPixel(decoded, OPAQUE_X, OPAQUE_Y, "opaque-red", "root-bg");
219+
console.log("[root-bg] PASS — composition root background preserved in alpha output");
220+
}
221+
190222
async function runGifCheck(workRoot: string): Promise<void> {
191223
console.log("\n[gif] rendering transparency-regression …");
192224
const outDir = join(workRoot, "gif");
@@ -306,6 +338,9 @@ async function main(): Promise<void> {
306338
if (!existsSync(FIXTURE_SRC)) {
307339
throw new Error(`Fixture missing: ${FIXTURE_SRC}`);
308340
}
341+
if (!existsSync(ROOT_BG_FIXTURE_SRC)) {
342+
throw new Error(`Fixture missing: ${ROOT_BG_FIXTURE_SRC}`);
343+
}
309344
if (!existsSync(SHADER_FIXTURE_SRC) || !existsSync(SHADER_GOLDEN)) {
310345
throw new Error(`Shader fixture or golden missing: ${SHADER_FIXTURE_DIR}`);
311346
}
@@ -315,6 +350,7 @@ async function main(): Promise<void> {
315350

316351
try {
317352
await runWebmCheck(workRoot);
353+
await runRootBackgroundCheck(workRoot);
318354
await runGifCheck(workRoot);
319355
await runGifShaderTransitionCheck(workRoot);
320356
await runPngSequenceCheck(workRoot);

‎packages/producer/tests/shard-schedule.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@
9393
"three-boundary-deferred": "Was absent from the hand-written shard matrix, so it has never run in CI and its cost and pass state are unknown. Triage and either schedule it or record a real reason here.",
9494
"timed-descendant-visibility": "Was absent from the hand-written shard matrix, so it has never run in CI and its cost and pass state are unknown. Triage and either schedule it or record a real reason here.",
9595
"transparency-regression": "Not a harness fixture by design \u2014 its meta.json says it is exercised by `tsx src/transparency-test.ts`, which asserts a real alpha channel rather than comparing against a golden MP4.",
96+
"transparency-regression-root-bg": "Not a harness fixture by design, same reason as its sibling `transparency-regression` \u2014 exercised by `tsx src/transparency-test.ts`, which asserts a real alpha channel rather than comparing against a golden MP4.",
9697
"video-hfid-no-id": "Was absent from the hand-written shard matrix, so it has never run in CI and its cost and pass state are unknown. Triage and either schedule it or record a real reason here.",
9798
"webgl-video-texture-render-compat": "Was absent from the hand-written shard matrix, so it has never run in CI and its cost and pass state are unknown. Triage and either schedule it or record a real reason here.",
9899
"wysiwyg-subcomp-css": "Rejected by the harness at load time (meta.json: maxAudioLagWindows must be >= 1), so it has never run despite looking scheduled-able. Fix the metadata, then schedule it."
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"name": "Transparency Regression — Composition Root Background",
3+
"description": "Asserts that an author-painted background on the composition root itself (not html/body) survives into alpha-carrying output instead of being force-cleared. Exercised by `tsx src/transparency-test.ts`, NOT by the standard regression harness (which compares against a golden MP4).",
4+
"tags": ["transparency", "alpha", "smoke"],
5+
"renderConfig": {
6+
"fps": 30
7+
}
8+
}
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
<!doctype html>
2+
<html>
3+
<head>
4+
<style>
5+
/* Only the composition root paints a background; body stays unpainted. */
6+
body {
7+
margin: 0;
8+
}
9+
</style>
10+
</head>
11+
<body>
12+
<div
13+
id="root"
14+
data-composition-id="transparency-regression-root-bg"
15+
data-width="200"
16+
data-height="200"
17+
data-start="0"
18+
data-duration="1"
19+
data-no-timeline
20+
style="position: absolute; inset: 0; background: rgb(0, 0, 255)"
21+
>
22+
<div
23+
class="card"
24+
style="position: absolute; left: 50px; top: 50px; width: 100px; height: 100px; background: rgb(255, 0, 0)"
25+
></div>
26+
</div>
27+
<script>
28+
window.__timelines = window.__timelines || {};
29+
</script>
30+
</body>
31+
</html>

0 commit comments

Comments
 (0)