Skip to content

Integrate with MoonEP and MXFP4 experts on the DeepGEMM runner - #18

Closed
lyppg wants to merge 1 commit into
wirybeaver:sg-moonep-upstream-draft-20260802from
bytedance-iaas:jxp/moonep_mxfp4_deepgemm_runner_integration
Closed

Integrate with MoonEP and MXFP4 experts on the DeepGEMM runner#18
lyppg wants to merge 1 commit into
wirybeaver:sg-moonep-upstream-draft-20260802from
bytedance-iaas:jxp/moonep_mxfp4_deepgemm_runner_integration

Conversation

@lyppg

@lyppg lyppg commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

MoonEP: MXFP4 experts on the DeepGEMM runner

Builds on #33249, this PR adds initial support for MoonEP a2a backend and DeepGemm runner for MXFP4 experts K3.

Validation

Validated end to end on Kimi-K3, 4x GB200, TP16/EP16, MXFP4 weights.
gsm8k test:

python -m sglang.test.run_eval --eval-name gsm8k --port 30000 --num-examples 500 --num-shots 5  --max-tokens 2048
Total latency: 643.858 s
Score: 0.986
Output throughput: 157.187 token/s
[METRIC] gsm8k_score=0.986 labels={"model": "Kimi-K3", "eval": "gsm8k"}
[METRIC] gsm8k_latency=643.8576721129939 labels={"model": "Kimi-K3", "eval": "gsm8k"}

CI States

Latest PR Test (Base): ❌ Run #31830931356
Latest PR Test (Extra): ❌ Run #31830931201

@lyppg
lyppg requested a review from wirybeaver August 13, 2026 22:26
@wirybeaver
wirybeaver force-pushed the sg-moonep-upstream-draft-20260802 branch from 7b0cdf6 to ffdf6f9 Compare August 14, 2026 14:37
@lyppg
lyppg force-pushed the jxp/moonep_mxfp4_deepgemm_runner_integration branch from 5035898 to f2d7990 Compare August 14, 2026 18:55
@lyppg

lyppg commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

moved to sgl-project#34874.

@lyppg lyppg closed this Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant