Skip to content

Qwen3-TTS PR 2: ramped chunk policy and window geometry for streaming consumers - #299

Open
merceod wants to merge 9 commits into
mainfrom
engine/streaming-chunk-schedule
Open

merceod wants to merge 9 commits into
mainfrom
engine/streaming-chunk-schedule

Conversation

@merceod

@merceod merceod commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Two small pieces for streaming vocoders (Qwen3-TTS, Qwen3-Omni Code2Wav, Orpheus SNAC, Chatterbox).

  1. ScheduledLeftContextChunkPolicy(schedule, chunk, left_context). Left-context chunking whose chunk sizes follow a ramp, so the first audio leaves after a few frames and later chunks grow. LeftContextChunkPolicy needs chunk > left_context and pops 300 frames before the first audio in the Qwen3-TTS deployment (24 s of audio). With schedule=(1, 3, 8, 16), chunk=25, left_context=25 the windows are 1 | 1+3 | 4+8 | 12+16 | 25+25 | ...
  2. The consumer gets told the geometry of each popped window instead of re-deriving the producer's schedule. StreamChunk.context_items (leading items an earlier chunk already delivered), num_items and start_offset ride the synthetic streaming edge and land in the consuming request's step_metadata["stream_chunks"][edge], readable from prepare_inputs and cg_key_info (so a consumer can pick its CUDA-graph bucket before its inputs are built). The existing policies report their context too (sliding window window - stride, left context left_context after the first pop, fixed 0), so overlap trimming works for the terminal flush and for any policy.

No behaviour change for existing models. context_items is informational, the new edge fields default to None, and step_metadata["stream_chunks"] is only set for streamed inputs. Nine commits, one per file.

Tests: test/modular/test_stream_chunk_schedule.py (ramped window/context sequence, every item delivered once as new data for 11 stream lengths and two consumer timings, one terminal chunk, context values of the existing policies). Full test/modular passes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant