[Bug]: FFN worker initialization fails when FFN ranks span multiple hosts

[swjeong9]

Current environment

Python: 3.12.13
OS: Ubuntu 22.04.5 LTS in the container / Amazon Linux 2023 on the host (kernel 6.12.100-125.179.amzn2023.x86_64)
afd-plugin commit: 117ebf0 (unmodified)
vLLM version/checkout: v0.26.0
CUDA/GPU: NVIDIA L4, one per process. Driver 580.178.04 (open kernel module), CUDA 13.0, NCCL 2.28.9+cuda13.0
Install command: an image built on vllm/vllm-openai:v0.26.0, with the vLLM v0.26.0
                 checkout and afd-plugin each installed through `pip install -e`
Deployment: AWS EKS 1.33, hostNetwork, one process per pod
Hosts: one g6.12xlarge (4x L4) and two g6.xlarge (1x L4 each), all in us-west-2a
Network: TCP over ENA. No EFA is attached, so NCCL falls back to the socket transport
Run: 2A2F, TP 1, DP 2 per role, expert parallel enabled, eager, DBO disabled

Reproduction

We deployed 2A2F across three hosts.

Placement	g6.12xlarge	g6.xlarge#1	g6.xlarge#2	Split role
B	F0, F1	A0	A1	Attention
C	A0, A1	F0	F1	FFN

• In placement B the Attention workers are scheduled onto separate hosts.
• In placement C the FFN workers are scheduled onto separate hosts.

The commands are below.

A=2; F=2
MODEL=deepseek-ai/DeepSeek-V2-Lite
COMMON=(--served-model-name deepseek-v2-lite --trust-remote-code --enable-expert-parallel
        --max-model-len 4096 --max-num-seqs 128 --max-num-batched-tokens 128
        --gpu-memory-utilization 0.9 --host 0.0.0.0 --data-parallel-size-local 1 --enforce-eager)
afd() { printf '{"afd":{"role":"%s","connector":"P2pNcclAFDConnector","host":"%s","port":6269,"num_attention_ranks":%d,"num_ffn_ranks":%d}}' "$1" "$FFN0_IP" "$A" "$F"; }

# FFN nodes, rank r = 0..F-1
vllm serve $MODEL "${COMMON[@]}" \
  --port 18400 --data-parallel-size $F \
  --data-parallel-address $FFN0_IP --data-parallel-rpc-port 13346 \
  --additional-config "$(afd ffn)"

# Attention nodes, rank r = 0..A-1
vllm serve $MODEL "${COMMON[@]}" \
  --port 18305 --data-parallel-size $A \
  --data-parallel-address $ATTN0_IP --data-parallel-rpc-port 13345 \
  --additional-config "$(afd attention)"

# For either role, ranks with r > 0 also pass --headless --data-parallel-start-rank $r

Expected behavior

Both subgroups complete initialization and the Attention endpoint begins accepting requests, even when the FFN ranks are distributed across multiple hosts.

Actual behavior and logs

Placement B succeeds and placement C fails on FFN rank 1. The two placements use the same nodes and the same GPU distribution; only the split role differs.

Placement	Split role	117ebf0
B	Attention	succeeds
C	FFN	fails

The pod placement for C:

rdv-attn-0   ip-192-168-10-252   (g6.12xlarge)
rdv-attn-1   ip-192-168-10-252   (g6.12xlarge)
rdv-ffn-0    ip-192-168-10-129   ← afd.host
rdv-ffn-1    ip-192-168-10-146   ← fails

An excerpt from the rdv-ffn-1 log:

(Worker_DP1_EP1 pid=209) ERROR 09-09 07:02:11 Traceback (most recent call last):
  File "/workspace/theseus/afd-plugin/afd_plugin/v1/worker/ffn_worker.py", line 154, in start_ffn_server_loop
    self.model_runner.initialize_afd_connector()
  File "/workspace/theseus/afd-plugin/afd_plugin/v1/worker/ffn_model_runner.py", line 106, in initialize_afd_connector
    self.connector.init_afd_connector()
  File "/workspace/theseus/afd-plugin/afd_plugin/connectors/gpu/p2p.py", line 270, in init_afd_connector
    self.a2e_group = StatelessProcessGroup.create(
  File "/workspace/theseus/vllm/vllm/distributed/utils.py", line 496, in create
    listen_socket.bind((host, port))
OSError: [Errno 99] Cannot assign requested address

rdv-ffn-1 fails because it attempts to bind an address that is not local to its host.

Each subgroup brings up a store server of its own. The rank that opens that server is the subgroup's rank 0, and under 2A2F the rank 0 of subgroup 0 is F0 while the rank 0 of subgroup 1 is F1.

# vllm/distributed/utils.py:492
launch_server = rank == 0
if launch_server and listen_socket is None:
    listen_socket.bind((host, port))

Both subgroups, however, are given the same afd_config.host as their bind address. Only the port is derived per subgroup.

# afd_plugin/connectors/gpu/p2p.py:272
self.a2e_group = StatelessProcessGroup.create(
    host=self.afd_config.host,
    port=base_port + self.mapping.subgroup_index + 1,
    ...
)

afd_config.host is F0's address. F0 therefore binds its own address and succeeds, whereas F1 attempts to bind a non-local address and fails. bind accepts only addresses assigned to the calling host.

Initialization succeeds as long as every FFN rank resides on a single host, and fails once they are distributed across multiple hosts.

We will open a separate PR proposing a fix for this issue.

AFD configuration

afd:
  connector: P2pNcclAFDConnector
  host: <node IP of F0>
  port: 6269
  num_attention_ranks: 2
  num_ffn_ranks: 2
  role: <ffn or attention>

Before submitting

• I searched existing issues for related reports.
• I confirmed whether this reproduces against vLLM v0.26.0 or explained why not.

[swjeong9]

Fixed in #328. Closing this issue as resolved.