Skip to content

fix: DHT bootstrap + hostname redirect fallback - #231

Open
shkarlsson wants to merge 2 commits into
QuixThe2nd:mainfrom
shkarlsson:fix/bun-dht-peer-discovery
Open

fix: DHT bootstrap + hostname redirect fallback#231
shkarlsson wants to merge 2 commits into
QuixThe2nd:mainfrom
shkarlsson:fix/bun-dht-peer-discovery

Conversation

@shkarlsson

@shkarlsson shkarlsson commented Jul 8, 2026

Copy link
Copy Markdown

Cherry-picked bluejorts's DHT fixes from #232 into this branch so CI passes.

Changes

DHT bootstrap fixes (from #232, bluejorts)

  • dht.listen() — now called explicitly so bittorrent-dht's internal bucket-refresh / re-bootstrap-when-isolated maintenance loop fires.
  • k-rpc timeout — moved to krpcSocket({ timeout: 5000 }) (k-rpc ignores the timeout option, k-rpc-socket respects it).
  • Cold-bootstrap loop — re-seeds routers and re-runs lookup(ownId) every 10s until the table reaches 20 nodes, instead of a single shallow lookup.
  • Concurrency — raised to 32/16 for faster DHT traversal.
  • Cache persistence — lowered threshold from 50→8 nodes so warm tables survive restarts.
  • Bootstrap list — replaced dead/misconfigured routers with empirically verified ones.

Hostname redirect fallback (src/backend/networking/http.ts)

When a peer's /auth response advertises a different hostname (common with dynamic IPs / NAT), fall back to verifying the server signature against the advertised identity while keeping the original route.

Bootstrap nodes (src/backend/config.ts)

Aligned with bluejorts's verified list (correct port for dht.libtorrent.org:25401, removed dead routers).

Testing

@bluejorts

Copy link
Copy Markdown

Thank you for this fix a ton! I don't know a ton about DHT internals so wasn't comfy looking into it myself.

@shkarlsson
shkarlsson force-pushed the fix/bun-dht-peer-discovery branch 2 times, most recently from 6793516 to 58f2322 Compare July 9, 2026 07:20
@bluejorts

Copy link
Copy Markdown

Context from digging into the "DHT stuck at 1 node under Bun" issue, in case it's useful (related: #232).

The bencode Uint8ArrayBuffer shim in bunCompat.ts turns out to be a no-op for the DHT decode path in this dependency tree. k-rpc-socket resolves its own nested copy of bencode (node_modules/k-rpc-socket/node_modules/bencode, v2.0.3), not the top-level bencode@4 that the shim patches. I verified that nested v2.0.3 already returns a real Buffer under Bun 1.3.14 (Buffer.isBuffer(msg.t) === true, working readUInt16BE), so the Buffer.isBuffer(msg.t) guard in k-rpc-socket was never actually failing — DHT responses decode and parse correctly without the shim.

The real reason the routing table stalls at one node is usage, not bencode:

  • dht.listen() is never called, so bittorrent-dht's internal bucket-refresh / re-bootstrap-when-isolated maintenance loop (gated on the socket listening event) never starts.
  • The intended 5s query timeout is passed to krpc(), which ignores it, so the socket uses its 2s default and drops slow-but-live routers.
  • Only a single shallow bootstrap lookup runs, then the DHT idles until the 15-minute reannounce.
  • The bootstrap list is mostly dead (dht.libtorrent.org on the wrong port :6881 vs :25401, plus defunct routers).

On the same machine/network, libtorrent reaches ~67 DHT nodes while bittorrent-dht gets 1 — under both Node and Bun — which rules out a Bun/UDP root cause.

#232 addresses the usage issues (calls listen(), adds a cold-bootstrap lookup loop, fixes the timeout/cache/bootstrap list) and takes the routing table from 1 → ~50–76 nodes, verified in a bridge-networked container, with no bencode shim needed.

Not a knock on this PR — the hostname-redirect fallback here is a separate, orthogonal improvement. Just flagging that the bencode piece can likely be dropped.

When a peer's /auth response advertises a different hostname than what
was connected to (common with dynamic IPs/NAT), the previous code
redirected auth to the advertised hostname. If that hostname was
unreachable, the connection failed entirely — even though the original
hostname was responding fine.

Fix: if advertised hostname auth fails, fall back to verifying the server
signature against the advertised identity while keeping the original
route. Also adds extra DHT bootstrap nodes for better resilience.
@shkarlsson
shkarlsson force-pushed the fix/bun-dht-peer-discovery branch from 58f2322 to 0aeddca Compare July 9, 2026 18:19
@shkarlsson

Copy link
Copy Markdown
Author

Thanks for the thorough investigation @bluejorts — you're right on all counts.

I verified the dependency chain myself:

  • k-rpc-socket does require('bencode') → resolves to its nested bencode@2.0.3 (in node_modules/k-rpc-socket/node_modules/bencode), which uses Buffer.prototype.slice() throughout and returns real Buffer objects for byte strings.
  • bittorrent-dht/client.js does import bencode from 'bencode' (ESM) but only uses bencode.encode, never decode.
  • bencode@4.0.0 decode returns new Uint8Array(data.slice(...)) → Uint8Array, but it's never called in the DHT decode path.

So bunCompat.ts was indeed a complete no-op. I've stripped it from this PR, along with the import './bunCompat' in index.ts.

This PR now contains only the two orthogonal fixes:

  1. Hostname redirect fallback (http.ts) — if the advertised hostname auth fails, fall back to verifying the server signature against the advertised identity while keeping the original route.
  2. Extra DHT bootstrap nodes (config.ts) — more bootstrap entries for better initial discovery.

#232 is the real fix for the 1-node DHT issue (listen() never called, k-rpc timeout ignored, bootstrap loop missing). I'll leave that entirely to you — let me know if there's anything I can help with on that front.

@shkarlsson shkarlsson changed the title fix: DHT peer discovery broken under Bun + hostname redirect fallback fix: hostname redirect fallback when advertised hostname is unreachable Jul 9, 2026
@shkarlsson shkarlsson changed the title fix: hostname redirect fallback when advertised hostname is unreachable fix: DHT bootstrap + hostname redirect fallback Jul 9, 2026
@shkarlsson
shkarlsson force-pushed the fix/bun-dht-peer-discovery branch from ba49850 to 3c224bc Compare July 9, 2026 20:15
The DHT routing table was stuck at a single node (itself) on cold start,
while libtorrent reached 67 nodes on the same network — so this was a
usage problem, not the network. Five compounding issues, all fixed:

- dht.listen() was never called, so the socket's 'listening' event never
  fired and bittorrent-dht's internal bucket-refresh / re-bootstrap-when-
  isolated maintenance loop never started. Now bind explicitly on a free
  ephemeral port (node.port belongs to the UTP transport).
- The 5s query timeout was passed to k-rpc, which ignores it; the socket
  used its 2s default and dropped slow-but-live routers. Moved to
  krpcSocket({ timeout: 5000 }).
- Only a single shallow bootstrap lookup ran, then the DHT idled until the
  15-minute reannounce. Added a cold-bootstrap loop that re-seeds routers
  and re-runs lookup(ownId) every 10s until the table reaches 20 nodes.
- Raised k-rpc concurrency (16→32) and backgroundConcurrency (4→16) for
  faster traversal; added backgroundConcurrency to the KRPCOptions type.
- Cache persistence only fired at >50 nodes (never reached), so restarts
  were always cold. Now persists whenever the table reaches a new high of
  >=8 nodes, seeding cacheSize from the loaded cache.

Bootstrap list: replaced the mostly-dead/mis-configured set (wrong
dht.libtorrent.org port :6881, defunct utorrent/bitcomet/aelitis routers,
and Hydrabase peer hostnames that don't speak the DHT) with the routers
empirically verified live: dht.transmissionbt.com:6881,
dht.libtorrent.org:25401 (correct port), the canonical bittorrent/utorrent
routers, and a transmissionbt IP fallback.

Verified: real app now reaches "Ready with 53 nodes" on startup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VjgRYtdZZk6LuvTmsmpSh

# Conflicts:
#	src/backend/config.ts
@shkarlsson
shkarlsson force-pushed the fix/bun-dht-peer-discovery branch from 3c224bc to 68ccbf3 Compare July 9, 2026 20:22
@e4779 e4779 mentioned this pull request Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants