Repository navigation
perf(producer): load injected video frames by URL during export - #4940
miguel-heygen wants to merge 4 commits into
Conversation
Every captured frame of a composition with video sent each visible clip's frame to the page as a multi-MB base64 data URI inside the CDP evaluate call. The page then parsed it, fetched the data URL and garbage-collected it on the main thread, frame after frame. The URL-served path was turned off in #1630 because the file server read files synchronously and had no Range support, so image loads starved behind full video downloads. The file server now streams and answers Range requests, and extracted frames are already linked into the compiled dir, so the injector gets the compiled-dir resolver back. One helper owns the render injector.
…ported render/shared.ts imports only engine types, and stage tests replace the engine module with a partial mock. A runtime import of createVideoFrameInjector there broke encodeStage's tests at load time, so the helper moves into renderOrchestrator.ts, which already imports the engine at runtime.
injectVideoFramesBatch swallowed img.decode() rejections, hid the native video and reported the frame as painted, so a frame that failed to load became a blank or stale frame in the render. With frames served by URL a missing file is a 404 in the page rather than a readFile error in Node, so the batch now rejects with the video id and frame source.
Edit accuracy: accurate 1216 (base branch 1216), smooth 1078 of thoseThe gate passes. Quarantined, measured but not gated (1)
Unstable (1)
|
|
Closing this after measuring it: loading injected video frames by URL does not make exports measurably faster, so it does not earn its extra moving part. What I measured (public launch films, main vs this branch, renders interleaved on the same 32-core Linux machine, order flipped each round, 5 rounds per arm, load recorded per run):
What stays true and may help whoever picks this up: the file server now streams with Range support, so #1630's reason for inline frames is gone; and a frame that fails to load is swallowed by Separate finding from these runs: two renders of the same film on main are not always byte-identical, because scene scripts that measure text run before web fonts load. That is being fixed on its own. |
What
Exports of compositions with video load each injected video frame by URL from the render's file server,
instead of sending it to the page as a base64 data URI on every captured frame. Output is byte-identical.
Why
During capture, every frame with a visible video clip shipped that clip's frame (a multi-MB JPEG, base64
encoded) inside the CDP
evaluatecall. The capture page then parsed the string, fetched the data URL andgarbage-collected it on its main thread, frame after frame.
A Chrome trace of the capture browser (single worker, hyperframes-launch from the public launch-films repo,
1080p, BeginFrame capture) over the same 38 s window:
Related work
Refs #1630, which turned the URL-served path off because the file server then read files synchronously
and had no Range support, so frame loads starved behind full video downloads and renders timed out. The
file server has since moved to streamed reads with Range support (
fileServer.ts), and the extract stagelinks extracted frames into the compiled dir, so the resolver #1630 left in place applies again.
How
createRenderVideoFrameInjector(render/shared.ts) is the one place the render builds its video frameinjector, with the compiled-dir frame resolver. The orchestrator calls it; the stale fix(producer): inline base64 frames in injector to unblock video-heavy renders #1630 comment goes.
their own injector for now: their frames live outside the served dir, so routing them needs a plan-layout
change and comes as a follow-up PR.
Test plan
/__hyperframes_video_frames/..., never a data URI.Fails with the resolver removed, passes with it.
Range: bytes=100-199with 206 and the exact bytes, and serves a framethrough a symlinked frame dir (how the extract stage links frames).
TABLE_PLACEHOLDER