[David Elner (delner)]

Based on #2724

Why

Every voice integration has to describe its session the same way under audio schema v3: sources, a shared timeline, participants, speakers, recorded segments, and each span's place on that timeline. AudioSessionRecorder owns that shape once, so integrations only translate their framework's events into calls.

• Provides common implementation for session shape to reduce duplication per integration.
• Provides session-local IDs (participant-N, speaker-N) derived from native values.
• With audio capture disabled, it can still describes the session, so timing and identities can still be relayed on the trace.

In sum, this enables LiveKit's move to schema v3 (next PR), and future voice integrations.

Example

How an integration uses it:

const audio = new AudioSessionRecorder({
  origin: Date.now(),
  sources: {
    "session-input": { label: "Agent input audio" },
    "agent-output": { label: "Agent audio" },
  },
  audio: { sampleRate: 48_000, segmentBytes }, // omit to describe without recording
  async onFile({ id, data, mimeType, metadata }) {
    // Upload `data` as input.audio[id], then log it with its audio.segments entry.
    session.log({ input: { audio: { [id]: await upload(data, mimeType) } }, metadata });
  },
});

session.log({ metadata: audio.session() }); // schema version, sources, timeline

const { id, added } = audio.participant("PA_x7Kq", {
  type: "user",
  attributes: { "contrib.livekit.participant_id": "PA_x7Kq" },
});
if (added) session.log({ metadata: added }); // audio.participants, first time only
span.log({ metadata: { "audio.participant_ids": [id] } });

audio.record("session-input", pcm, { sampleRate: 16_000, channelCount: 1, offsetMs });
span.log({ metadata: { "audio.selections": audioSelections([{ start, end }]) } });

await audio.finish();

Design notes

integration ──calls──► AudioSessionRecorder ──returns──► metadata to log
                            │
                            └─ AudioRecorder (when recording) ─► onFile(segment + entry)

• The AudioSessionRecorder is responsible for managing the session, resolving references, etc but it is not responsible for instrumentation or trace data itself... it merely returns metadata that instrumentation can log.
• It tracks commonly referenced objects like participant and speaker, and helps spans map to a common, PII-less ID. (e.g. participant-1)