P2P collaborative inference across browser tabs via WebRTC

[sauravpanda]

Context

Wavefy Network splits model layers across devices via P2P. 60 layers divided among 4 devices. Similar to Petals.

Proposal

• Use WebRTC data channels to connect browser tabs/devices
• Each browser holds N layers of the model
• Pipeline-parallel inference: activations flow through the chain
• Could run 70B+ models across 4-8 browser tabs on different machines
• Expert routing for MoE: different peers hold different experts

Wild but feasible

• WebRTC data channels are low-latency and widely supported
• The main challenge is activation transfer bandwidth (~16KB per token per layer)
• Best for batch/offline scenarios, not interactive chat