Conversation
This was referenced Sep 22, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two small pieces for streaming vocoders (Qwen3-TTS, Qwen3-Omni Code2Wav, Orpheus SNAC, Chatterbox).
ScheduledLeftContextChunkPolicy(schedule, chunk, left_context). Left-context chunking whose chunk sizes follow a ramp, so the first audio leaves after a few frames and later chunks grow.LeftContextChunkPolicyneedschunk > left_contextand pops 300 frames before the first audio in the Qwen3-TTS deployment (24 s of audio). Withschedule=(1, 3, 8, 16), chunk=25, left_context=25the windows are 1 | 1+3 | 4+8 | 12+16 | 25+25 | ...StreamChunk.context_items(leading items an earlier chunk already delivered),num_itemsandstart_offsetride the synthetic streaming edge and land in the consuming request'sstep_metadata["stream_chunks"][edge], readable fromprepare_inputsandcg_key_info(so a consumer can pick its CUDA-graph bucket before its inputs are built). The existing policies report their context too (sliding windowwindow - stride, left contextleft_contextafter the first pop, fixed 0), so overlap trimming works for the terminal flush and for any policy.No behaviour change for existing models.
context_itemsis informational, the new edge fields default toNone, andstep_metadata["stream_chunks"]is only set for streamed inputs. Nine commits, one per file.Tests:
test/modular/test_stream_chunk_schedule.py(ramped window/context sequence, every item delivered once as new data for 11 stream lengths and two consumer timings, one terminal chunk, context values of the existing policies). Fulltest/modularpasses.