feat(core): support deterministic async frame sources

[jrusso1020]

What

Add window.__hyperframes.registerFrameSource so a timed composition can draw from deterministic scene code without a GSAP timeline. HyperFrames remains responsible for playback and capture.

Why

Imported HTML motion scenes need to join the existing seek and capture path while preserving their authored animation code. This is the native runtime foundation for a sandboxed film-runner bridge.

Related work

First of three dependent PRs: #5130 native frame sources → #5131 film HTML bridge → #5132 bounded scene timing. MCP import/export is separate work owned by Somansh.

How

A deterministic adapter maps existing host timing, playback inpoint, rates and speed ramps into source time. It serializes asynchronous draws, coalesces pending preview seeks, and joins setup/draw failures to the existing capture completion barrier. Removing a host or reverting the runtime aborts and disposes its source.

Test plan

• Unit tests added/updated: no-GSAP runtime integration, capture waiting, reverse/repeated seeks, coalescing, timing changes, overlapping sources, setup/draw errors and teardown.
• Manual testing performed: partner HTML visual acceptance remains pending.
• Documentation updated, including navigation.
• Comments follow CONTRIBUTING.md.

Workspace build, core build, core typecheck and changed-file lint/format checks passed. All 4,233 core tests passed with --maxWorkers=4 --testTimeout=30000; the default 5-second timeout was too short for one unrelated local-font fixture, which also passed in isolation. Complexity report: highest new function is current at CC 6; no new function exceeds 10.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
hyperframes	🟢 Ready	View Preview	Oct 7, 2026, 12:45 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

[github-actions]

Edit accuracy: accurate 2059 (base branch 2059), smooth 1636 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

[jerrai-bot-heygen]

Reviewed at 608dc220a, focused on async capture and lifecycle. The ordinary capture path does wait for registered draws, and the existing test discriminates that wait. Two problems:

1. Blocker: static-frame dedup can certify stale pixels for a frame-source composition.

• Static dedup is default-on (HF_STATIC_DEDUP, frameCapture.ts:3534). The only content disqualifiers are <video>, <canvas>, non-GSAP animations, tl.call() and unresolvable starts (frameCapture.ts:3039-3133). A registered frame source is not one of them, and neither is an iframe, so a film scene from #5131 plus any GSAP tween is eligible.
• The verifier's seekToFrame calls __hf.seek and screenshots straight away (frameCapture.ts:3435-3459), without the waitForPendingSeekCompletion that real capture does (frameCapture.ts:2828).
• Failure: a late async draw makes two different frames screenshot identical, the run is armed, and production reuses lastFrameBuffer for frames whose seek never ran (frameCapture.ts:4204-4217). The export is silently wrong.
• Fix: either disqualify static dedup when any frame source is registered, or have the verifier await the same seek-completion barrier as capture before each screenshot. A test with an async source that draws after a tick, in a comp with one GSAP tween, would pin it.
• Established from the code paths; I did not run a browser reproduction.

2. Should-fix: a failed draw drops the seek queued behind it (frameSources.ts:33-52).

• seek(1) starts drain. seek(2) arrives while draw 1 is pending, sets pending = 2 and returns the same work. Draw 1 rejects, so the loop throws and finally clears pending.
• Result: time 2 is never drawn, and its waiter receives draw 1's error.
• frameSources.test.ts:125-137 only seeks again after the rejection, so it misses this.
• Fix: report the failure for the failed time and keep draining the queued one (or reject it explicitly). Add a test that queues a seek before the rejection.

Tests not run locally: no installed deps in my checkout.

— Jerrai

[jrusso1020]

Both findings are valid and fixed in 37b9f4c.

• Static dedup now disqualifies compositions with registered frame sources or iframes. A real-browser regression starts with an eligible GSAP composition, adds an async source, waits for its draw, and verifies dedup is disabled.
• A failed draw no longer discards the seek queued behind it. The queue drains, while the capture barrier retains the first error. A later fresh seek can recover.

Validation: focused frame-source/entry tests, 54 engine frame-capture tests, and the new browser regression passed. The final stack also passes 4,252 core tests, 591 SDK tests, 51 browser tests, core build/typecheck, and commit hooks.

Broader engine suite: 2,053 passed, 11 failed outside the changed capture tests (FFmpeg color/sampling checks and an HDR PNG that is an LFS pointer in this checkout). Actual partner HTML preview/export acceptance remains unverified under the previously reported browser policy block.

[jerrai-bot-heygen]

Re-reviewed at 37b9f4c30. Both findings are fixed.

• Static dedup: __hfHasFrameSources() and a host <iframe> are now disqualifiers (frameCapture.ts:3042-3043, :3134-3135). The browser test first proves the comp is eligible, then registers a source and asserts it's ineligible, and also covers the iframe case.
• Queued seek after a failed draw: the drain keeps the first error and still draws the queued time. The test queues time 2 before rejecting time 1 and asserts both draws happened.

Not run locally; this is from the source and the tests.

— Jerrai

[miguel-heygen]

Reviewed at 37b9f4c30, focused on async capture and teardown.

Both earlier findings are fixed and pinned by tests:

• Static-dedup analysis now refuses a registered frame source or any iframe (frameCapture.ts:3134-3135). The real-browser test starts from an eligible GSAP composition, so the new disqualifier is the only reason it flips.
• A failed draw keeps draining the queued time, and the barrier still gets the failure (frameSources.ts:35-52, test "drains a queued seek after an earlier draw fails").

One minor item left, not blocking:

Minor: a throwing dispose callback makes one frame skip every source, with no error.

• current() disposes a disconnected host inline (frameSources.ts:88), which calls the author's dispose (:67). If that throws, current() exits before returning its list. The seek loop at :111 then runs for no source, nothing joins the capture barrier, and the runtime swallows the throw (init.ts:4366).
• Result: that frame is captured with every frame-source scene still showing its previous draw, and the export succeeds.
• The same throw in revert (:135) stops before owned.clear() (:136), so later sources are never aborted and stay in the module-level map.
• Fix: catch around the author callback in dispose, pass the error to the barrier (registerSeekCompletion(Promise.reject(error))) so capture fails loudly, and let the loop continue.

CI at this head: 73 passing, 20 pending, none failing.

Verdict: APPROVE
Reasoning: the barrier, coalescing and teardown hold on every path I traced, and static dedup can no longer reuse a frame that a source changed. The remaining item is a narrow error-path gap.

[jrusso1020]

Follow-up from the actual-sample end-to-end validation: fixed redundant transport draws in 29a5c02.

The browser regression reproduced a single renderSeek(0.5) producing two async source draws. renderSeek now records its time and timeline in the transport's existing paused-frame cache. The next animation-frame tick no longer redraws outside the capture wait. Explicit repeated/reverse render seeks still draw, and resumed playback continues to advance.

The regression covers preview and export modes with a first-party asynchronous source. The owning PR passed all 4,234 core tests and commit-hook type/lint/format checks. The final stack additionally passed its combined runtime and browser regressions; details are on #5132.

This change preserves the data-no-timeline documentation fixes from the validation session. The pre-existing 30 fps preview seek grid is outside this fix.

[jerrai-bot-heygen]

Approved at 29a5c02b4. The delta from 37b9f4c30 is correct.

• renderSeek records its frame (init.ts:3906-3907). It now sets lastTransportSeekTime and lastTransportSeekTimeline after it has itself run seekTimelineAndAdapters(..., activateChildren). The transport tick then sees an unmoved playhead and the same timeline, so it skips the redundant seekTimelineAndAdapters(t) (:4683), and the parked-loop check (:4428) can park.
• Redraws still happen when they should: while playing, after a manual-gesture deferral, or when the timeline is re-bound.
• The cache records a real draw. The explicit seek actually drew that frame, so this does not hide one.
• The new browser test covers preview and export, a repeated seek to the same time (which still draws), a reverse seek, and play/pause afterwards.
• The doc line on data-no-timeline is accurate.

I read the source and the test; I did not run them.

— Jerrai