fix(daemon): honour HTTP(S)_PROXY when dialing the wakeup WebSocket - #6279
Merged
Conversation
runTaskWakeupConnection built its dialer by hand:
dialer := websocket.Dialer{HandshakeTimeout: 10 * time.Second}
A zero-value Proxy field means "dial direct and ignore the
environment". websocket.DefaultDialer sets Proxy to
http.ProxyFromEnvironment; a dialer built this way gets nothing, and
gorilla skips the CONNECT wrapper entirely.
This is the daemon's control connection to the Multica server, so in
the SaaS deployment it dials out to the public internet. On a machine
whose only egress is a corporate proxy the handshake can never succeed.
Nothing points at the cause either: the loop logs "task wakeup
websocket unavailable; polling fallback remains active" at debug level
and never mentions a proxy. The daemon then runs permanently degraded —
task pickup waits for the HTTP poll (PollInterval, 30s by default)
instead of a server push, heartbeats stay on HTTP, and the WS-first
batch claim path (MUL-4257) falls back to HTTP on every task.
Set Proxy: http.ProxyFromEnvironment. gorilla rewrites wss:// to
https:// on the parsed URL before it calls Proxy, so HTTPS_PROXY — and
NO_PROXY — apply to this dial the same way they apply to every other
HTTPS client in the process. The lark connector fixed the same defect
the same way in multica-ai#4165; this was the last bare dialer left in non-test
code.
No regression where no proxy is configured: ProxyFromEnvironment
returns a nil URL, gorilla leaves netDial untouched, and the dial is
byte-for-byte the direct dial it was before. Everything downstream of
the handshake — headers, heartbeat writer, RPC attach, teardown — is
unchanged.
The regression test drives the dial from a child process. net/http
resolves the proxy environment once per process and caches the result
(envProxyOnce), so by the time a test in this package runs, an earlier
test has already primed that cache with "no proxy" and t.Setenv can no
longer reach it. The child starts with a clean environment pointing at
a stub CONNECT proxy, and the parent asserts the CONNECT for the wss
target arrived. Without the fix the stub proxy sees nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@seacen is attempting to deploy a commit to the IndexLabs Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On a machine whose only route to the internet is a corporate egress proxy, the daemon's task-wakeup WebSocket can never connect.
runTaskWakeupConnectionbuilds its dialer by hand, and a zero-valuewebsocket.DialerhasProxy == nil, which gorilla reads as "dial direct" — unlikewebsocket.DefaultDialer, it never consults the environment. The failure is silent, too: the loop logs "task wakeup websocket unavailable; polling fallback remains active" at debug level and nothing ever mentions a proxy, so the daemon just runs permanently degraded — task pickup waits on the 30s HTTP poll instead of a server push, and the WS-first claim path (MUL-4257) falls back to HTTP on every task.Every
net/httpcall in the process honoursHTTPS_PROXYthroughDefaultTransport, so this one connection behaving differently is genuinely hard to diagnose from the outside.The fix is one line — set
Proxy: http.ProxyFromEnvironment— plus a comment. gorilla rewriteswss://tohttps://before consultingProxy, soHTTPS_PROXYapplies to this dial. With no proxy variables set,ProxyFromEnvironmentreturns nil and the dial is byte-for-byte what it was before;NO_PROXYis honoured the same way it is everywhere else in the process.A note on the regression test, because its shape is unusual: it drives the real
runTaskWakeupConnectiondial through a fake CONNECT proxy, but it has to re-exec itself into a child process.net/httpcaches the proxy environment once per process (envProxyOnce), and other daemon tests have already burned that cache by the time this test runs — a plaint.Setenvversion passes in isolation and fails with the package. The child gets a scrubbed environment with only the fake proxy set, so the cache semantics are correct regardless of test order or-shuffle. The target host is deliberately non-loopback, since Go's proxy rules bypass proxies for localhost. The test fails onmain(the proxy never sees a CONNECT) and passes with the fix; it is also clean under-race.go build ./...,go vet, and the wakeup/WS/heartbeat tests are green. One pre-existing failure in the package (TestProbeAgentCLIs_QoderResolvesViaLoginShell) fails identically on a cleanmaincheckout on this machine — it picks up a locally installed qoder CLI — and is unrelated.This is the last hand-built dialer in the tree; the lark connector already falls back to the environment at dial time (#4165). Follow-up to the note at the end of my review on #5833.
cc @Bohan-J — one line of production code, hopefully quick.