perf(job): unthrottle autovacuum on job_executions - #142
Merged
Conversation
The job_executions table already had aggressive autovacuum triggers (scale_factor 0.01, threshold 50), but under a 10 loans/s stress-test soak in lana-bank it still accumulated 13.9k dead tuples against 114 live rows, and the poller's WITH due AS (...) query degraded from ~9ms at startup to ~30ms within an hour. The trigger thresholds were firing; the default 2ms vacuum cost delay throttled each run so reclaim could not keep up with the churn rate. Set autovacuum_vacuum_cost_delay = 0 for this table. It is tiny (~100 live rows), so unthrottled vacuums are cheap and keep dead tuples near zero, holding the poll query's cost flat. Mirrors GaloyMoney/lana-bank#7675.
nicolasburtey
marked this pull request as ready for review
July 29, 2026 22:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
During a 1h, 10-loans/s-target lana-bank stress-test soak (
sb-shard10benchsandbox, 2026-07-29),job_executionsaccumulated 13,893 dead tuples against 114 live rows despite the aggressive autovacuum triggers already in this migration (scale_factor = 0.01,threshold = 50). The poller'sWITH due AS (...)query degraded from ~9 ms at startup to ~30 ms and stayed there.The trigger thresholds were firing — the bottleneck was the default 2 ms
vacuum_cost_delaythrottle: each autovacuum run reclaimed dead tuples slower than the workload created them.What
Adds
autovacuum_vacuum_cost_delay = 0to the existing per-table settings. The table is tiny (~100 live rows), so unthrottled vacuum runs are cheap and keep dead tuples near zero, holding the poll query's cost flat.Mirrors GaloyMoney/lana-bank#7675, which carries the same change in lana-bank's vendored copy of this migration (files verified identical).
Evidence
pg_stat_user_tables:n_live_tup = 114,n_dead_tup = 13,893pg_stat_statements(job poll, queryid-7391300573923152793): mean 9.2 ms (startup) → 30.1 ms (T+25min) → 27.5 ms (T+1h)