fix(lifecycle): install signal handlers before startup work

[stephane-segning]

Summary

Two fixes, both found by running the software rather than reading it.

1. A SIGTERM arriving early in startup killed vpay-server outright, bypassing graceful shutdown and dropping any in-flight request.
2. just ci failed roughly one run in several on a testcontainers concurrency flake — pre-existing on master, unrelated to (1), but a gate that fails intermittently is not a gate.

Source of truth: found while running the stack locally at the maintainer's request after #3 merged. Bears on docs/flows/crash-safety.md, whose whole subject is not losing work at process boundaries.

Intent

The signal race

$ target/debug/vpay-server & P=$!; sleep 0.05; kill -TERM $P; wait $P; echo $?
143          # killed outright — no graceful shutdown ran

$ target/debug/vpay-server & P=$!; sleep 1.0;  kill -TERM $P; wait $P; echo $?
0            # "received SIGTERM, starting graceful shutdown"

Root cause. tokio::signal::ctrl_c() is an async fn — it registers its handler on first poll. The shutdown future was not polled until axum::serve(..).with_graceful_shutdown(..) ran, i.e. after CLI parsing, tracing init, adapter-registry logging and TcpListener::bind. Until that moment SIGTERM retained its default disposition: immediate termination. tokio::signal::unix::signal(kind), by contrast, is a plain function that registers synchronously inside its body.

Under CPU contention the window widens, which is why vpay-worker-bin showed it too when both binaries were started together.

The flake

Each integration test starts its own postgres:16-alpine testcontainer, so nextest's default parallelism raced 13+ container starts simultaneously, intermittently failing with Error: container port. The same test passed 3/3 in isolation.

Scope

Signal handling. Both binaries now install SIGTERM and SIGINT via signal() as the first statement in main(), before tracing init, through a shared vpay_config::ShutdownSignals (that crate already hosts the shared CommonArgs, so there was precedent). Installation failure is now a hard startup error rather than a logged warning — it means the process has no graceful shutdown for its entire lifetime, not for a brief window, which is a misconfiguration rather than a degradation. --shutdown-grace-seconds and the bounded-drain logic are untouched; non-unix keeps the previous ctrl_c() fallback.

Test concurrency. A nextest test group bounds vpay-tests-integration to one concurrent test. Each test keeps its own container and its existing Drop-based cleanup — only the rate of container starts changes.

Verification

Check	Before	After
SIGTERM at 50ms, vpay-server	exit 143, no shutdown line	6/6 exit 0 + shutdown line
SIGTERM at 50ms, vpay-worker-bin	intermittent 143	6/6 exit 0 + shutdown line
cargo nextest run --workspace ×5	~1 in several failed	5/5 clean, 80 passed / 3 skipped
Leaked containers after 5 runs	—	0
just ci	exit 100	exit 0

Screenshots / Evidence

INFO vpay_config::signal: received SIGTERM, starting graceful shutdown
INFO vpay_worker_bin: graceful shutdown complete, exiting
EXIT=0

Risk Assessment

Low, with two disclosures I want visible rather than buried.

The regression test is weaker than its name suggests. A genuine signal exists at DELAY = 2ms (~90-98% pass fixed vs ~68% unfixed, 250+ trials per side, macOS and a Linux container). That delay does not survive full-suite contention, which widens the window for fixed code too — no delay was found that is simultaneously safe under load and sensitive to this exact bug. It settled at 50ms, which is stable across repeated full runs but primarily guards against total removal of signal handling rather than narrow reintroduction of this race. This is stated in the test's own doc comment.

A shared-container approach was tried first and rejected on evidence, destructively. Rust does not drop statics at process exit, so a ContainerAsync held in a OnceCell never runs Drop and leaks its container on every invocation. This was not theoretical — it orphaned hundreds of containers, drove the host's Docker VM into memory pressure, and OOM-killed an unrelated container on the development machine. All orphans were cleaned up and the affected container restarted. The finding is real and worth keeping: do not hold a testcontainer in a static.

max-threads = 3 was also measured and still failed ~1 run in 14; the binding constraint is the Docker VM's 4 vCPUs, not the host's 12 cores. max-threads = 1 may be more conservative than a well-resourced Linux CI runner needs — it is a fixed bound, not an auto-detect, so it leaves some speed on the table there. Cost is wall-clock only: ~7s → ~15s for the full workspace run.

AI Usage Declaration

AI (Claude Opus 5 via Claude Code) performed this work, delegating to Sonnet sub-agents. The orchestrating session re-ran every claim rather than accepting agent reports — which mattered here: my own first reproduction script reported 0/6 and was wrong (it reused one log file across iterations and mangled the exit-code check); re-measured properly it is 6/6. The machine state after the container leak was independently verified clean before committing.

• A human is accountable for this change and has reviewed it.
• Every claim here was verified by running the command, not inferred.
• Limitations stated explicitly — see Risk Assessment.
• No functionality was fabricated; no test was weakened to go green.

Reviewer Focus

1. The hard-fail decision in ShutdownSignals::install() — a process that cannot install a handler now refuses to start. Agree?
2. max-threads = 1 — right bound for your CI, or should it be relaxed there?
3. The regression test's honest limitation under load, disclosed above.

[changeset-bot]

⚠️ No Changeset found

Latest commit: e32f66b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR