fix: decouple microphone recording rate from TTS sample rate#503
Open
GrowDev1 wants to merge 1 commit into
Open
fix: decouple microphone recording rate from TTS sample rate#503GrowDev1 wants to merge 1 commit into
GrowDev1 wants to merge 1 commit into
Conversation
…#491) SAMPLE_RATE (24000) was a single hardcoded constant used for BOTH TTS playback (where 24kHz is correct -- it's what the TTS providers emit) AND microphone recording (where it's wrong -- a mic's native rate is whatever the hardware is, commonly 44.1kHz or 48kHz for USB mics, never 24kHz). Requesting a rate the device doesn't natively support isn't universally safe: some backends resample transparently, some don't, producing corrupted/aliased recordings on affected hardware/OS/driver combinations. Adds RECORDING_SAMPLE_RATE, a new config constant defaulting to SAMPLE_RATE (so behavior is unchanged unless a user opts in via VOICEMODE_RECORDING_SAMPLE_RATE), and switches every recording-side usage in converse.py to it: prepare_audio_for_stt's AudioSegment construction, the WAV archival writes, record_audio's sd.rec() call, and record_audio_with_silence_detection's VAD chunk-size math + sd.InputStream + resampling math. All TTS/playback-side usage (streaming.py, core.py's chime functions) is untouched. Also fixes docs that referenced a VOICEMODE_SAMPLE_RATE variable the code never actually read (a documented no-op) -- including one that recommended it as a WSL2 microphone troubleshooting step, which would have silently done nothing. Pointed these at the new, working variable instead. Tests: 2 new files covering the config default/override behavior, that prepare_audio_for_stt/record_audio honor the actual capture rate (verified via real WAV byte output, not just mocked call args), and that the VAD resampling math lands on an exact 480-sample (30ms/16kHz) frame across every standard rate (8k/16k/24k/32k/44.1k/48kHz) -- WebRTC VAD requires an exact frame size or it raises.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #491.
Summary
SAMPLE_RATE(voice_mode/config.py,24000) was a single hardcoded constant used for both TTS playback and microphone recording. 24kHz is correct for TTS (it's what the providers emit), but wrong as a hardcoded mic capture rate — a microphone's native rate is whatever the hardware is, commonly 44.1kHz or 48kHz for USB mics, never 24kHz. Requesting a rate the device doesn't natively support isn't universally safe: some audio backends resample transparently, some don't, producing corrupted/aliased recordings on affected hardware/OS/driver combinations — exactly what #491 reports with a 48kHz-native mic.The fix
Added
RECORDING_SAMPLE_RATE = int(os.getenv("VOICEMODE_RECORDING_SAMPLE_RATE", str(SAMPLE_RATE)))— a new, independent config constant. Defaults toSAMPLE_RATE, so behavior is unchanged unless a user explicitly opts in.Switched every recording-side usage in
voice_mode/tools/converse.pyto the new constant:prepare_audio_for_stt()'sAudioSegmentconstruction + the Whisper-downsample comparisonrecord_audio()'ssd.rec()call, sample-count math, and debug/error loggingrecord_audio_with_silence_detection()'s VAD chunk-size math,sd.InputStream()call, and the VAD-downsampling resample-length mathsample_rate_hz,bitrate_kbps)All TTS/playback-side usage (all of
streaming.py,core.py's chime-playback functions) is untouched — verified by checking every remainingSAMPLE_RATEreference in the repo is genuinely playback-side, not a missed recording site.Docs fix, found while verifying this:
docs/reference/environment.mdanddocs/guides/configuration.mdalready documented aVOICEMODE_SAMPLE_RATEvariable — including recommending it as a fix for exactly this kind of mic-rate mismatch — but the code never actually reads that variable anywhere (confirmed via a repo-wide grep). It was a documented no-op that would have silently done nothing for anyone following the docs. Updated those entries (and one indocs/.archive/troubleshooting/wsl2-microphone-access.md) to point at the new, workingVOICEMODE_RECORDING_SAMPLE_RATEinstead.Test plan
tests/test_recording_sample_rate_config.py— new, 3 tests: default equalsSAMPLE_RATE(24000, unchanged behavior), override is decoupled from TTS rate, no cross-test pollutiontests/test_recording_sample_rate_prepare_audio.py— new, 5 tests:prepare_audio_for_stt()produces WAV bytes whose actual duration is correct at an overridden rate (a real check viawavemodule on real exported bytes — this would fail ifframe_rateweren't threaded correctly, unlike a mocked-call-args check)record_audio()requests the overridden rate fromsounddevicetests/test_stt_audio_saving.py,test_stt_local_compression_skip.py,test_silence_detection.py,test_vad_aggressiveness.py,test_history_buffer.py,test_audio_format_config.py): 61 passed, 2 failed (confirmed pre-existing on unmodifiedmastertoo — missingffmpegin my environment, unrelated to this change), 5 skipped (pre-existing, unrelated)