feat(db): Database connection multiplexing for serverless environments (#600)#651
feat(db): Database connection multiplexing for serverless environments (#600)#651jadonamite wants to merge 1 commit into
Conversation
|
@jadonamite Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| - | - | Generic Password | 7167381 | docker-compose.yml | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
…evs17#600) Route serverless DB access through a transaction-pooling proxy (RDS Proxy or PgBouncer) so a small set of backend connections is multiplexed across many concurrent function invocations, preventing connection exhaustion. - backend/shared/db/serverlessPool.ts: transaction-pooling adapter with IAM/SCRAM-256 auth, credential refresh, withClient/withTransaction helpers, and abandoned-connection leak detection (>30s force-close). - backend/serverless/dbConfig.ts + withDatabase.ts: env-driven pool config (RDS IAM token provider) and a Lambda wrapper that releases the client in a finally block after every invocation. - backend/monitoring/connectionPoolMetrics.ts: Prometheus pool/leak metrics and structured leak alerting. - infra/terraform/{rds_proxy,pgbouncer}.tf: proxy provisioning (max ~50 pooled connections serving 500+ functions, transaction pooling). - docker-compose.yml + .env.example: local PgBouncer + Postgres for parity; all credentials read from a gitignored .env (no hardcoded secrets). Closes Smartdevs17#600
615f491 to
7167381
Compare
Summary
Implements database connection multiplexing for serverless workloads. Serverless functions (webhook handlers, auth callbacks, scheduled jobs) currently open a new DB connection per invocation, exhausting the connection limit during traffic spikes. This routes all connections through a transaction-pooling proxy (AWS RDS Proxy or self-hosted PgBouncer), so a small set of backend connections is multiplexed across hundreds of concurrent functions.
Changes
backend/shared/db/serverlessPool.ts—ServerlessConnectionPool: transaction-pooling adapter over the existingpgpool, withwithClient/withTransaction/queryhelpers, IAM and SCRAM-256 auth modes, per-connect credential refresh (rotating RDS tokens), and connection-leak detection that force-closes any client checked out longer than 30s.backend/serverless/dbConfig.ts— env-driven pool configuration, including an RDS IAM auth-token credential provider (@aws-sdk/rds-signer, lazily imported).backend/serverless/withDatabase.ts— Lambda handler wrapper that hands each invocation a pooled client and callsrelease()in afinallyblock, regardless of success/throw. Pool singleton is reused across warm invocations.backend/monitoring/connectionPoolMetrics.ts— Prometheus pool gauges (total/idle/waiting/checked-out), a*_leaked_totalcounter, and structured leak alerting.infra/terraform/rds_proxy.tf— RDS Proxy with IAM auth, transaction pooling (~50 backend connections), TLS required, 30s idle timeout,EXCLUDE_VARIABLE_SETSpinning filter.infra/terraform/pgbouncer.tf— self-hosted PgBouncer alternative (transaction mode, SCRAM-256,MAX_PREPARED_STATEMENTS, 500 client cap / 50 server pool).docker-compose.yml— local Postgres + PgBouncer for dev/test parity.Acceptance criteria
MAX_PREPARED_STATEMENTS/ RDS Proxy)release()in afinallyblock per invocationCloses #600