Skip to content

Commit 45b8e17

Browse files
vanceingallsclaude
andauthored
feat(studio): sync origin, out-of-sync repair, linked selection toggle and grouped clip menu (7/8) (#4820)
* feat(studio): order the clip menu into time, sound, picture and delete groups Split and Freeze frame; Normalize, Voice, link items and Duck; Look and Crop; then Delete beside Delete this clip only. Each item file renders its pieces separately, empty groups collapse, and the viewport clamp uses the measured menu height instead of a row-count estimate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(studio-server): mint freeze-frame right-half links with the shared lk-N helper Freeze frame now calls core relinkSplitHalves, so split halves and freeze halves share one link-id convention instead of L -> L-2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(studio): apply speed edits to every data-link member in one undo step The inspector speed slider, rate lane edits and ramp presets on a linked clip now commit once, writing the selection and, via expandToLinkedMembers, each partner from its own current value. Unlinked clips keep their usual path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(studio): drop the audio parent tick in favour of the data-link chain badge A linked clip now shows one link indicator: the chain badge driven by data-link. The waveform no longer draws its 1px parent tick. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(studio): follow late clip-menu rows and target link partners like S2 The menu height follows a ResizeObserver so rows that read the live preview after mount still clamp inside the viewport. Speed fan-out builds partner targets with buildPatchTarget, matching the link edit plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(studio): show the sound strip and audio menu items without studio's shell The audible-video waveform strip drops AudioWaveform's 16px label band, which left a 0px canvas on short lanes. Normalize loudness and Duck under voice fall back to the timeline session's project, the live preview iframe store and a new onNotice edit callback when no StudioShellProvider is mounted. usePreviewIframeStore is exported so a host can publish its preview. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(studio): show duck under voice only when a voice overlaps The item now appears only when an overlapping voice exists or the bed is already ducked. Its check mark moves to the right edge so unchecked rows align with siblings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(studio): name the partner in the clip menu link and unlink items Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(studio): clip menu stops offering unlink once the clips are unlinked The timeline skipped store updates when only data-link changed, so the menu kept reading the stale link. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(core): sync origin, offset math and move/slip into sync for linked pairs data-sync-origin survives Unlink; split gives right halves a fresh origin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(studio-server): split re-mints the sync origin of unlinked pairs too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(studio): detach, link and keep-sound cutout write a sync origin that unlink keeps TimelineElement.syncOrigin is read on both parse paths; merge back drops it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(studio): expect sync-origin on keep-sound cutout write Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(studio): linked selection toggle in the timeline toolbar, on by default Off: a click selects one clip; drag, trim, razor, speed and delete stay on it (delete unlinks the survivor). Alt-click still selects one half when on. Right-click holds the out-of-sync indicator preference. Both persist in UI prefs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(studio): link groups share a label colour; out-of-sync badges with move and slip Red badge on both halves of a sync-origin pair that drifted (frames, s:ff past a second); hidden when rates differ or the preference is off. The whole badge opens Move / Slip into Sync; each edits that clip alone in one undo step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(studio): take link label colours from theme tokens Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(sdk): syncOffset, moveIntoSync and slipIntoSync for sync-origin pairs Core gains findSyncPartner, the DOM pairing shared by the SDK and the CLI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): timeline rows report syncOffsetFrames and print out-of-sync Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(cli): the timeline row fixture carries syncOffsetFrames Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(skills): data-sync-origin in hyperframes-core Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core,sdk): keep a one-sided cut's sync origin and slip the in-point playback reads relinkSplitHalves reminted the right half's sync origin even when only one member of an unlinked source pair was cut, so the uncut partner kept the old origin and the new half lost its badge. The origin is now reminted only when both the video and the audio were cut. Slip into Sync wrote data-media-start whenever it existed, but playback reads data-playback-start first, so a clip carrying both never moved. The link timing reader now uses playback's precedence and the slip writes the attribute that wins; lint compares the same field. A sync partner is also looked up only in the clip's own composition (core) or source file (Studio). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(studio): flip the out-of-sync menu above the pointer near the bottom edge Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor: trim comments the sync-origin layer added Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(core): split the scoped candidate lookup out of findSyncPartner Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core,studio): find sync partners inside the link scope findSyncPartner and Studio's syncPartnerOf now scope by the same rule as link groups: the nearest data-composition-id or data-composition-file ancestor, plus the row's compositionScope in Studio. An origin reused in a file host or an inline composition no longer pairs across it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent b5e3ffa commit 45b8e17

75 files changed

Lines changed: 2046 additions & 253 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎packages/cli/src/timeline/describeProject.test.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,20 @@ describe("describeProject", () => {
116116
]);
117117
});
118118

119+
it("reports each half's out-of-sync offset from its source pair, at the project fps", async () => {
120+
const { rows, text } = await rowsOf(`<html><body>
121+
<div data-composition-id="main" data-duration="10" data-fps="24">
122+
<video id="v" src="t.mp4" muted data-sync-origin="lk-1" data-start="1" data-duration="4" data-track-index="0"></video>
123+
<audio id="a" src="t.mp4" data-sync-origin="lk-1" data-start="1.5" data-duration="4" data-track-index="1"></audio>
124+
<audio id="m" src="m.mp3" data-start="0" data-duration="4" data-track-index="2"></audio>
125+
</div></body></html>`);
126+
const offset = (id: string) => rows.find((r) => r.id === id)?.syncOffsetFrames;
127+
expect(offset("a")).toBe(12);
128+
expect(offset("v")).toBe(-12);
129+
expect(offset("m")).toBeNull();
130+
expect(text).toContain("out-of-sync=+12f");
131+
});
132+
119133
it("reports unreadable automation instead of showing no lanes", async () => {
120134
const bad = (await describeProject(project())).tracks
121135
.flatMap((t) => t.rows)

‎packages/cli/src/timeline/describeProject.ts‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@ import {
88
import { HF_AUDIO_FX_ATTR, parseAudioFxChain } from "@hyperframes/core/audio-fx";
99
import { HF_AUDIO_GROUP_ATTR } from "@hyperframes/core/audio-groups";
1010
import { byStart, type ClipFact, type ClipLane } from "@hyperframes/core/clip-facts";
11-
import { parseNumeric } from "@hyperframes/core";
11+
import { fpsToNumber, parseFpsWithDefault, parseNumeric } from "@hyperframes/core";
12+
import { findSyncPartner, readLinkTiming, syncOffsetFrames } from "@hyperframes/core/media-link";
13+
import { readCompositionFps } from "../utils/compositionFps.js";
1214
import {
1315
readMediaOffsetSeconds,
1416
readPlaybackRate,
@@ -60,6 +62,8 @@ export interface TimelineRow extends ClipFact {
6062
hostRow: RowPointer | null;
6163
/** Where this row's sub-composition clips are (one level only); each is a full row in its kind's `rows`. */
6264
children: RowPointer[];
65+
/** Signed frames from its `data-sync-origin` partner (+ = late); `null` unpaired or rates differ. */
66+
syncOffsetFrames: number | null;
6367
}
6468

6569
export interface RowPointer {
@@ -137,6 +141,7 @@ interface DocScope {
137141
withProbeSlot: <T>(fn: () => Promise<T>) => Promise<T>;
138142
measure: MeasureMedia;
139143
sourceOverrides: ReadonlyMap<string, string>;
144+
fps: number;
140145
}
141146

142147
/** Source length in seconds of a media file. ffprobe in production; tests pass a recorded fake. */
@@ -312,9 +317,15 @@ function describeRowFields(scope: DocScope, node: DomNode, timing: RowTiming): C
312317
warnings: [],
313318
elementId: el.id || null,
314319
hfId: el.getAttribute("data-hf-id"),
320+
syncOffsetFrames: syncOffsetOf(el, scope.fps),
315321
};
316322
}
317323

324+
function syncOffsetOf(el: Element, fps: number): number | null {
325+
const partner = findSyncPartner(el);
326+
return partner ? syncOffsetFrames(readLinkTiming(el), readLinkTiming(partner), fps) : null;
327+
}
328+
318329
async function describeRow(scope: DocScope, node: DomNode, depth: number): Promise<ClipDraft> {
319330
return describeRowFields(scope, node, await resolveRowTiming(scope, node, depth));
320331
}
@@ -343,6 +354,7 @@ async function readSubComposition(
343354
withProbeSlot: parent.withProbeSlot,
344355
measure: parent.measure,
345356
sourceOverrides: parent.sourceOverrides,
357+
fps: parent.fps,
346358
};
347359
const rows = await Promise.all(
348360
topLevelElements(toNode(root)).map((node) => describeRow(scope, node, 1)),
@@ -416,6 +428,11 @@ function flatten(top: ClipDraft[]): TimelineTrack[] {
416428
}));
417429
}
418430

431+
function projectFps(source: string): number {
432+
const parsed = parseFpsWithDefault(readCompositionFps(source) ?? undefined);
433+
return fpsToNumber(parsed.ok ? parsed.value : { num: 30, den: 1 });
434+
}
435+
419436
/** Needs a global DOMParser (`ensureDOMParser`). Reads `index.html` and one level of sub-compositions. */
420437
export async function describeProject(
421438
indexPath: string,
@@ -437,6 +454,7 @@ export async function describeProject(
437454
withProbeSlot: createProbeGate(PROBE_CONCURRENCY),
438455
measure,
439456
sourceOverrides,
457+
fps: projectFps(source),
440458
};
441459
const rows = (
442460
await Promise.all(topLevelElements(toNode(root)).map((node) => describeRow(scope, node, 0)))

‎packages/cli/src/timeline/formatTimeline.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@ function durationNote(row: TimelineRow): string | false {
2929
}
3030
}
3131

32+
function syncNote(frames: number | null): string | false {
33+
if (!frames) return false;
34+
return `out-of-sync=${frames > 0 ? "+" : ""}${frames}f`;
35+
}
36+
3237
function details(row: TimelineRow): string {
3338
const lanes = row.lanes.map(
3439
(l) => `${l.target}[${l.points.map((p) => `${n(p.t)}:${n(p.v)}`).join(" ")}]`,
@@ -38,6 +43,7 @@ function details(row: TimelineRow): string {
3843
row.volume !== null && `vol=${row.volume}`,
3944
row.playbackRate !== null && `rate=${n(row.playbackRate)}`,
4045
row.audioGroup && `group=${row.audioGroup}`,
46+
syncNote(row.syncOffsetFrames),
4147
durationNote(row),
4248
row.sourceFile && !row.children.length && "children=unread",
4349
row.laneError && `lanes unreadable: ${row.laneError}`,

‎packages/cli/src/timeline/timeExpr.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ const row: TimelineRow = {
3232
durationSource: "authored",
3333
pendingReason: null,
3434
laneError: null,
35+
syncOffsetFrames: null,
3536
};
3637

3738
describe("parseTimeExpression", () => {

‎packages/core/src/mediaLink.test.ts‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,14 @@
11
import { describe, expect, it } from "vitest";
22
import {
33
MEDIA_LINK_ATTR,
4+
SYNC_ORIGIN_ATTR,
5+
findSyncPartner,
6+
formatSyncOffset,
47
linkTimingMismatches,
8+
moveIntoSyncStart,
9+
slipIntoSyncMediaStart,
10+
sourceZeroTime,
11+
syncOffsetFrames,
512
mintLinkId,
613
readLinkTiming,
714
relinkSplitHalves,
@@ -24,6 +31,11 @@ describe("readLinkTiming", () => {
2431
it("reads data-playback-start as the media-start alias", () => {
2532
expect(readLinkTiming(attrs({ "data-playback-start": "1.5" })).mediaStart).toBe(1.5);
2633
});
34+
35+
it("lets data-playback-start win over data-media-start, as playback does", () => {
36+
const both = attrs({ "data-playback-start": "0", "data-media-start": "0.5" });
37+
expect(readLinkTiming(both).mediaStart).toBe(0);
38+
});
2739
});
2840

2941
describe("linkTimingMismatches", () => {
@@ -86,6 +98,17 @@ describe("relinkSplitHalves", () => {
8698
expect(link("i-split")).toBeNull();
8799
});
88100

101+
it("keeps the source origin when only one member of an unlinked pair is cut", () => {
102+
const d = doc(`
103+
<video id="v" data-sync-origin="lk-1" data-start="0" data-duration="10"></video>
104+
<audio id="a" data-sync-origin="lk-1" data-start="0" data-duration="4"></audio>
105+
<audio id="a-split" data-sync-origin="lk-1" data-start="4" data-duration="6"></audio>`);
106+
relinkSplitHalves(d, ["a-split"]);
107+
const right = d.getElementById("a-split");
108+
expect(right?.getAttribute(SYNC_ORIGIN_ATTR)).toBe("lk-1");
109+
expect(right && findSyncPartner(right)?.id).toBe("v");
110+
});
111+
89112
it("mints distinct ids for distinct groups", () => {
90113
const d = doc(`
91114
<video id="v" data-link="lk-1"></video><video id="v2" data-link="lk-1"></video>
@@ -97,3 +120,100 @@ describe("relinkSplitHalves", () => {
97120
expect(ids).not.toContain("lk-7");
98121
});
99122
});
123+
124+
describe("relinkSplitHalves sync origin", () => {
125+
it("gives right halves their own shared sync origin, even when unlinked", () => {
126+
document.body.innerHTML = `
127+
<video id="v" data-sync-origin="lk-1"></video><video id="v2" data-sync-origin="lk-1"></video>
128+
<audio id="a" data-sync-origin="lk-1"></audio><audio id="a2" data-sync-origin="lk-1"></audio>`;
129+
relinkSplitHalves(document, ["v2", "a2"]);
130+
const origin = (id: string) => document.getElementById(id)?.getAttribute(SYNC_ORIGIN_ATTR);
131+
expect(origin("v")).toBe("lk-1");
132+
expect(origin("a")).toBe("lk-1");
133+
expect(origin("v2")).not.toBe("lk-1");
134+
expect(origin("v2")).toBe(origin("a2"));
135+
});
136+
});
137+
138+
describe("sync offset", () => {
139+
const clip = (start: number, mediaStart = 0, playbackRate = 1) => ({
140+
start,
141+
mediaStart,
142+
playbackRate,
143+
});
144+
145+
it("places source zero at start minus media-start over rate", () => {
146+
expect(sourceZeroTime(clip(4, 2, 2))).toBe(3);
147+
});
148+
149+
it("is zero for a pair in sync, whatever their shared media start", () => {
150+
expect(syncOffsetFrames(clip(2, 1), clip(2, 1), 30)).toBe(0);
151+
expect(syncOffsetFrames(clip(3, 2), clip(2, 1), 30)).toBe(0);
152+
});
153+
154+
it("is signed from the clip's side: late is positive, and the partner reads the negation", () => {
155+
expect(syncOffsetFrames(clip(2 + 10 / 30), clip(2), 30)).toBe(10);
156+
expect(syncOffsetFrames(clip(2), clip(2 + 10 / 30), 30)).toBe(-10);
157+
});
158+
159+
it("counts media start: a slipped clip is out of sync in place", () => {
160+
expect(syncOffsetFrames(clip(2, 0.5), clip(2, 0), 30)).toBe(-15);
161+
});
162+
163+
it("divides media start by the shared rate", () => {
164+
expect(syncOffsetFrames(clip(2, 1, 2), clip(2, 0, 2), 30)).toBe(-15);
165+
});
166+
167+
it("has no offset when the rates differ", () => {
168+
expect(syncOffsetFrames(clip(2, 0, 1.5), clip(2, 0, 1), 30)).toBeNull();
169+
});
170+
171+
it("formats frames, then seconds:frames past one second", () => {
172+
expect(formatSyncOffset(10, 30)).toBe("+10");
173+
expect(formatSyncOffset(-51, 30)).toBe("-1:21");
174+
expect(formatSyncOffset(30, 30)).toBe("+1:00");
175+
expect(formatSyncOffset(-3, 24)).toBe("-3");
176+
});
177+
178+
it("moves the clip onto its partner, or refuses before zero", () => {
179+
expect(moveIntoSyncStart(clip(2.5, 0.5), clip(1, 0))).toBeCloseTo(1.5);
180+
expect(syncOffsetFrames(clip(1.5, 0.5), clip(1, 0), 30)).toBe(0);
181+
expect(moveIntoSyncStart(clip(0, 2), clip(0, 0))).toBeCloseTo(2);
182+
expect(moveIntoSyncStart(clip(1, 0), clip(0, 2))).toBeNull();
183+
});
184+
185+
it("slips the clip's media in place, scaled by rate, or refuses before the file start", () => {
186+
expect(slipIntoSyncMediaStart(clip(3, 0), clip(2, 0))).toBeCloseTo(1);
187+
expect(slipIntoSyncMediaStart(clip(3, 0, 2), clip(2, 0, 2))).toBeCloseTo(2);
188+
expect(syncOffsetFrames(clip(3, 2, 2), clip(2, 0, 2), 30)).toBe(0);
189+
expect(slipIntoSyncMediaStart(clip(1, 0), clip(2, 0))).toBeNull();
190+
});
191+
});
192+
193+
describe("findSyncPartner", () => {
194+
it("finds the opposite kind sharing the origin, preferring the most shared timeline", () => {
195+
document.body.innerHTML = `<div data-composition-id="m">
196+
<video id="v" data-sync-origin="lk-1" data-start="0" data-duration="4"></video>
197+
<video id="v2" data-sync-origin="lk-1" data-start="4" data-duration="4"></video>
198+
<audio id="a" data-sync-origin="lk-1" data-start="4.2" data-duration="4"></audio>
199+
<audio id="x" data-sync-origin="lk-9" data-start="4" data-duration="4"></audio></div>`;
200+
const byId = (id: string) => document.getElementById(id);
201+
const a = byId("a");
202+
const x = byId("x");
203+
expect(a && findSyncPartner(a)?.id).toBe("v2");
204+
expect(x && findSyncPartner(x)).toBeNull();
205+
});
206+
207+
it.each(['data-composition-id="child"', 'data-composition-file="child.html"'])(
208+
"never takes a partner from a nested composition (%s) reusing the origin",
209+
(host) => {
210+
document.body.innerHTML = `<div data-composition-id="m">
211+
<video id="v" data-sync-origin="lk-1" data-start="0" data-duration="4"></video>
212+
<div ${host}>
213+
<audio id="ca" data-sync-origin="lk-1" data-start="0" data-duration="4"></audio>
214+
</div></div>`;
215+
const v = document.getElementById("v");
216+
expect(v && findSyncPartner(v)).toBeNull();
217+
},
218+
);
219+
});

0 commit comments

Comments
 (0)