Skip to content

feat: TCP keepalive for Postgres and MySQL connections#4355

Open
adriangb wants to merge 1 commit into
transact-rs:mainfrom
adriangb:feat/tcp-keepalive
Open

feat: TCP keepalive for Postgres and MySQL connections#4355
adriangb wants to merge 1 commit into
transact-rs:mainfrom
adriangb:feat/tcp-keepalive

Conversation

@adriangb

@adriangb adriangb commented Jul 24, 2026

Copy link
Copy Markdown

Adds TCP keepalive support, closing #3540. This is @xuehaonan27's #3559 rebased onto the current connect_tcp and reduced to an additive API; the design credit is theirs.

Why

Without keepalive, a connection whose server disappeared without closing the socket never finds out. A failover, a killed container, or a dropped NAT mapping leaves the client blocked reading a response that will never arrive. There is nothing left to retransmit, so no RST is ever provoked, and the read waits forever. TCP_NODELAY, which is all SQLx sets today, does not help, and a server-side statement_timeout cannot fire on a server that is gone.

We hit this in production: we have a ~ hot loop that is constantly querying so we are almost guaranteed to hit this every server restart (pgbouncer scale down in our case). It reproduces reliably by holding an ACCESS EXCLUSIVE lock so the query blocks server-side, then removing the server from the network and restarting it. Notably a restart w/o an in-flight query won't reproduce: the query has to be already acknowledged and executing, which is exactly the common case during a real failover.

@abonander, in #3559 (comment) you allowed for this case: "I suppose that's still preferable to it hanging forever on a read that will never complete." One correction on the objection raised there, that a keepalive timeout is only noticed the next time we use the socket: that is not true for a blocked reader. When the probes are exhausted the kernel sets sk_err to ETIMEDOUT and wakes anyone parked on the socket, so a pending read fails rather than waiting for someone to poke it. That is precisely the case that hangs today.

Since #3559 was opened, three other users have reported the same silent-stale-connection failure in it (Aug 2025, Dec 2025, Feb 2026), and the author of the follow-up #4159 moved to tokio-postgres over it.

What this adds

  • sqlx_core::net::TcpKeepalive, with idle / interval / retries, cfg-gated for the platforms where socket2 does not support the latter two.
  • net::connect_tcp_with_keepalive. connect_tcp keeps its signature and delegates with None, so external drivers (Compile-time support for external drivers #3889) are unaffected.
  • PgConnectOptions::tcp_keepalive() and MySqlConnectOptions::tcp_keepalive().
  • libpq-compatible URL parameters for Postgres: keepalives, keepalives_idle, keepalives_interval, keepalives_count. These previously hit the ignoring unrecognized connect parameter branch, so anyone copying a libpq DSN silently got no keepalive.

Keepalive stays off by default, so behaviour is unchanged for existing users. (libpq defaults it on with a 2h idle; I did not want to change a default in this PR, but happy to if you would prefer parity.)

Tests

Unit tests for the builder and for the URL parameters, including that keepalives=0 disables it again after another parameter turned it on. The setsockopt itself is not unit-testable without a socket; it is exercised by every connection made with keepalive configured.

I am carrying this as a patched fork in the meantime, so I am happy to iterate on the API shape or split MySQL out if that makes review easier.

@adriangb
adriangb force-pushed the feat/tcp-keepalive branch 5 times, most recently from ee406e3 to 1e8c1d9 Compare July 24, 2026 20:25
Adds `TcpKeepalive` and `net::connect_tcp_with_keepalive`, plus
`tcp_keepalive()` on `PgConnectOptions` and `MySqlConnectOptions` and
libpq-compatible URL parameters for Postgres (`keepalives`,
`keepalives_idle`, `keepalives_interval`, `keepalives_count`).

Keepalive is off by default, so nothing changes for existing users.

Motivation: without it, a connection whose server disappeared *without*
closing the socket never finds out. A failover, a killed container, or a
dropped NAT mapping leaves the client blocked reading a response that will
never arrive; there is nothing left to retransmit, so no RST is ever
provoked and the read waits forever. `TCP_NODELAY`, which is all SQLx sets
today, does not help, and a server-side `statement_timeout` cannot fire on
a server that is gone.

This is the case @abonander allowed for in
transact-rs#3559 (comment):
"I suppose that's still preferable to it hanging forever on a read that
will never complete." Worth adding on the objection raised there, that a
keepalive timeout is only noticed the next time the socket is used: that
is not true of a blocked reader. When the probes are exhausted the kernel
sets `sk_err` to `ETIMEDOUT` and wakes anyone parked on the socket, so the
pending read fails rather than waiting for someone to poke it.

We hit this in production: a maintenance loop that had a query in flight
when its Postgres instance went away stopped doing work permanently, while
other loops in the same process recovered in seconds. Reproduced by holding
an `ACCESS EXCLUSIVE` lock so the query blocks server-side, then removing
the server from the network and restarting it.

Implementation follows transact-rs#3559 by @xuehaonan27, rebased onto the current
`connect_tcp` and reduced to an additive API: `connect_tcp` keeps its
signature and delegates, so external drivers are unaffected.
@adriangb
adriangb force-pushed the feat/tcp-keepalive branch from 1e8c1d9 to be545ed Compare July 24, 2026 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant