Stabilize WAN MPS generation on Apple Silicon#1868
Conversation
|
Thx for this work but in order to integrate in WanGP I need to minimize the potential impact on the existence not only in terms of compatibility but also for future maintenance:
|
|
There are two forms of sdpa, native and manual attention. Native sdpa was causing higher memory usage, and giving slower generations as well as errors. I will see what can be done. |
|
@deepbeepmeep I have to bow out here. I understand what it is doing, but not at a code level, so I really don't think it wise that I keep trying to make patches that just "work" rather than patches that take into account the whole picture instead of just WAN. For anyone who wants to pick up the torch (pun intended) the primary issue was that the code to support the gradio UI runs in another thread, and the monkey patches combined with MPS caused a race condition between the two threads which generates numerous command buffer and uncommitted encoder errors. This wasn't found until very late in trying to solve the issue, so even though Codex thinks it is a bad idea to start over and target that issue first, my gut says that might be the better solution.... solve the thread race condition first, then clean up what is left. Test by running queues in headless mode (create the queue in the UI, exit, and then run them headless and then compare completion with running the same task within the ui.) |
This PR improves WAN generation stability on Apple Silicon/MPS, especially for WAN 2.2 5B and WAN 2.1 1.3B.
Changes:
WAN2GP_MPS_NATIVE_SDPA=1.Validation:
Let me be clear that these patches were done with Codex and GPT 5.5. I can't speak to the quality or durability of the patches. They center around WAN, as that was my primary goal. Memory usage is quite high, so a 36GB Mac or better is really needed if you plan to do anything with 14B, and even then the frame size has to be pretty small.
Manual fallback SDPA was chosen for lower memory impact and slightly faster speed.
WebUI worker-thread note
During testing, the quantized WAN transformer path behaved differently in headless queue processing versus the Gradio/WebUI path.
The true quantized WAN 2.2 5B checkpoint could complete repeated headless generations, but the same model/configuration would intermittently crash from the WebUI with Metal command-buffer assertions such as:
commit an already committed command buffercommit command buffer with uncommitted encoderThe instability appeared when generation was dispatched through the extra WebUI
async_run_in("generation", ...)worker thread. Running the queued WAN generation inline in the Gradio request path for quantized WAN transformer tasks avoided those crashes in local testing.This is intentionally scoped narrowly:
quanto,int8, orfp8)WAN2GP_MPS_WEBUI_INLINE_WORKERThis keeps the WebUI behavior unchanged for CUDA and for non-quantized/non-WAN tasks.