fix(mps): prevent Metal command buffer double-commit crash on 5B quanto models#1773
Closed
huangyebiaoke wants to merge 1 commit into
Closed
fix(mps): prevent Metal command buffer double-commit crash on 5B quanto models#1773huangyebiaoke wants to merge 1 commit into
huangyebiaoke wants to merge 1 commit into
Conversation
…to models Root cause: Wan 2.2 5B uses quanto mbf16 quantization whose ops lack native MPS kernels and fallback to CPU via PYTORCH_ENABLE_MPS_FALLBACK=1. These CPU fallback ops corrupt the MPS command buffer when mixed with native MPS SDPA attention, triggering: IOGPUMetalCommandBuffer validate:213: failed assertion 'commit an already committed command buffer'. Fixes (2 changes): 1. device_patch.py: Add torch.mps.synchronize() before SDPA to flush pending fallback operations before entering the MPS attention kernel. 2. attention.py: Add .contiguous() after .to() and .expand() calls in pay_attention() to ensure tensor buffer independence and prevent shared buffer references from being committed twice. Tested by: NOT YET (needs M3/M4 Mac with 36GB to test 5B quanto model) Ref: deepbeepmeep#1750 (comment)
This was referenced May 8, 2026
Author
3 tasks
Author
|
Superseded by #1776 which adds manual fallback for macOS 26.x Metal bugs on top of these fixes. Closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root Cause
Wan 2.2 5B uses
quantombf16 quantization. Quando ops lack native MPS kernels and fallback to CPU viaPYTORCH_ENABLE_MPS_FALLBACK=1. These CPU fallback ops corrupt the MPS command buffer when mixed with native MPS SDPA attention.The crash happens immediately after first inference step because the first quantized weight access + attention operation triggers the corruption.
Why 14B works but 5B doesn't
quantombf16 -> weight access ops fallback to CPU -> buffer corruptionFix (2 changes)
torch.mps.synchronize()before SDPA to flush pending fallback operations.contiguous()after.to()and.expand()to ensure tensor buffer independenceTesting Needed
@SquishedSquirrel could you re-test Wan2.2 5B (with and without fastwan) on your M3 Max 36GB?
Ref: #1750 (comment)