[Tracker]: afd-plugin active development

[jiangkuaixue123]

Purpose

This issue is the execution-level index for active afd-plugin work.

• Project direction, support policy, and long-term workstreams remain in [RFC]: afd-plugin project roadmap #155.
• Detailed design and acceptance criteria remain in the linked RFCs and issues.
• The linked issue or PR is the source of truth for status; this checklist is a
  maintainer-facing snapshot.

Last reviewed: 2026-09-14

Upstream alignment

• GPU backend alignment with vLLM 0.26.0 — [RFC]: Upgrade the GPU backend to vLLM 0.26.0 #167
• GPU backend alignment with vLLM 0.28.0 — [RFC]: Upgrade the GPU backend to vLLM 0.28.0 #310
• NPU backend alignment with vLLM/vLLM-Ascend 0.26.0 — [RFC]: Upgrade the NPU backend to a vLLM-Ascend nightly compatible with vLLM 0.26.0 #168
• NPU backend alignment with vLLM 0.28.0 and a pinned compatible vLLM-Ascend runtime — [RFC]: Upgrade the NPU backend to vLLM 0.28.0 #341

ModelRunnerV2

• GPU ModelRunnerV2 support — [RFC]: Add GPU ModelRunnerV2 Support for AFD #223 @jiaran-king
  • Implementation: Add AFD support for CUDA GPU ModelRunnerV2 #245
  • DBO support ([1/2][Model Runner V2] DBO support, eager mode vllm#50945) [2/2][Model Runner V2] FULL CUDA graph capture for microbatched steps (DBO) vllm#51700
• Ascend NPU ModelRunnerV2 support — [RFC]: Add Ascend NPU ModelRunnerV2 support for AFD #240 @lirx-pd
  • Implementation: add MRV2 for NPU without DBO #257
  • DBO support [Feature]: Add DBO support for Ascend NPU ModelRunnerV2 #265

Native MoERunner forward refactor

• Refactor AFD MoE forward around native MoERunner injection — [RFC]: Refactor AFD MoE forward around native MoERunner injection #225

Phase 1: Attention-side native MoERunner reuse

• Replace handwritten GPU and CAMP2p MoE forward proxies with
  role-specific runners that share one remote-experts implementation.
• Reuse native Ascend routing for CAMAsync and remove the copied routing
  selection path.
• Preserve connector payloads, ordering, FFN-driver ownership, dense-layer
  behavior, and role-aware weight allocation.
• Validate GPU P2P NCCL, NPU CAMP2p, and NPU CAMAsync with pre-/post-refactor
  parity across their currently supported execution modes.

Phase 2: Model-independent FFN MoE computation

• Introduce reusable GPU and NPU FFN MoE runners instead of
  model-specific FFN helper methods.
• Keep connector communication state in the FFN driver.
• Demonstrate reuse with at least one non-DeepSeek native FusedMoE model.
• Document and fail closed for dense, sequence-parallel, and non-factory
  MoE exceptions.

Detailed architecture, validation requirements, and acceptance criteria remain
in #225.

Model support

DeepSeek V4

• DeepSeek V4 support tracker — [RFC]: Add DeepSeek V4 support on GPU and NPU for vLLM 0.26.0 #169 @bjf-frz
  • GPU P2pNcclAFDConnector implementation — V0.26.0 support for dsv4 (gpu) #191
  • NPU CAMAsyncAFDConnector — [DeepSeek V4][NPU] Support CAMAsyncAFDConnector #227
    • Implementation: [feat][model]v0.26.0 camasync dsv4 #300
  • NPU CAMP2pAFDConnector — [DeepSeek V4][NPU] Support CAMP2pAFDConnector #228

GLM5.3-flash

• NPU CAMAsyncAFDConnector(prioritize this) @yujuancao07
• NPU CAMP2pAFDConnector
• GPU P2pNcclAFDConnector

Qwen3 MoE

• CUDA AFD adapter on vLLM 0.26.0 — [RFC]: Add Qwen3 MoE CUDA AFD support on vLLM 0.26.0 #215, [Feature] Add Qwen3 MoE CUDA AFD support #216
  • GPU PyNcclAFDConnector support — [RFC]: Add Qwen3 MoE CUDA AFD support on vLLM 0.26.0 #215, [Feature] Add Qwen3 MoE CUDA AFD support #216
  • NPU CAMP2PConnector support (help wanted)
  • NPU CAMAsyncConnector support (help wanted)

Qwen3.5 / Qwen3.6 MoE

• CUDA text-only adapter for the native
  Qwen3_5MoeForConditionalGeneration family — [RFC] Experimental Qwen3.5 MoE AFD support: CUDA correctness baseline #179, Add Qwen3.6 MoE CUDA AFD implementation support #181, test(e2e): add Qwen3.6 MoE CUDA coverage #256
  • GPU PyNcclAFDConnector support — [RFC] Experimental Qwen3.5 MoE AFD support: CUDA correctness baseline #179, Add Qwen3.6 MoE CUDA AFD implementation support #181, test(e2e): add Qwen3.6 MoE CUDA coverage #256
  • NPU CAMP2PConnector support (help wanted)
  • NPU CAMAsyncConnector support (help wanted)

Model performance

• DeepSeek V3.2 prefill performance and token-balanced TP/SP dual batching
  — [RFC]: DeepSeek V3.2 prefill performance study and token-balanced TP/SP dual batching #170 @ShwStone

Connectors and runtime optimization

• Async GPU connector — [RFC]: Async GPU Connector #233 @specture724
  • Implementation: [Feat]: first impliment for async GPU connector #239
• CAMAsync multi-stream execution — [Feature]: Add CAMAsync multi-stream execution on a shared NPU stream pipeline #150
• CAMP2p A2E/E2A multi-stream execution on the FFN side — [Feature]: Enable multi-stream parallelism for CAMP2P A2E/E2A on FFN side #68
  • Implementation: [Feat]: support multistream for CAMP2p #161
• Derive CAM HCCL buffer sizes per role — Derive CAM HCCL buffer sizes per role #234

Reliability and CI

Merged reliability fixes (FFN graph replay, NPU teardown, and multi-host
initialization): #270, #279, #328.

CI now includes L4 Kubernetes jobs, weekly CUDA E2E, and strengthened
pre-commit checks: #299, #268, #271.

NPU runtime qualification for vLLM 0.28.0 remains tracked in #341.

Platform and contributor ecosystem

• Experimental Moore Threads MUSA backend — [RFC]: Add Moore Threads MUSA backend support for AFD #173
• Repository-backed agent workflow — [RFC]: Initialize repository-backed agent workflow for afd-plugin #196
• Maintain the Attention–FFN disaggregation paper index — [Doc]: important papers of attention-ffn disaggregation #32