fix(profiles,users): serialize backfill setup with OIDC provisioning transactions

[willgriffin]

Closes #2352

Root cause (verified, not assumed)

The issue's hypothesis is correct, and I confirmed it two ways.

1. Adapter-level. The DuckDB adapter guards transaction() / beginTransaction() with an internal connectionLock, but top-level query() takes no lock. A statement left in flight when BEGIN TRANSACTION runs on the same native connection fails immediately:

probe3a commit-then-concurrent-root-reads:  ok            (400 rounds)
probe3b root-stmt-in-flight-at-BEGIN:       FAILED
  sql: 'SELECT 1 FROM t LIMIT 1'
  originalError: 'Failed to execute prepared statement'

So concurrent root reads are not the hazard (the post-commit rebind's Promise.all is fine); a root statement overlapping an open transaction on the same connection is.

2. In the failing test. Instrumenting oidc-account-linking.test.ts with sub-millisecond start/end markers on the shared root handle shows the exact overlap on this machine, even on a green run:

0.265 root start (inflight 1, txdepth 0) SELECT 1 FROM _smrt_backfills LIMIT 1   ← flow A, pre-lock
0.275 root start (inflight 2, txdepth 0) SELECT 1 FROM _smrt_backfills LIMIT 1   ← flow B, pre-lock
0.575 root end   (inflight 1, txdepth 0)
0.800 >>> BEGIN (depth 1, inflight 1)                    ← flow A opens its transaction
0.853 root end   (inflight 0, txdepth 1)                 ← flow B's statement ends INSIDE flow A's transaction

coordinateOidcProvisioning established the shared _smrt_backfills table before acquiring its adapter-transaction lock ("outside the provisioning transaction"). Both flows therefore issue a root statement while unserialized; whichever flow wins the lock opens BEGIN TRANSACTION on the same native connection a few hundred microseconds later, while the other flow's statement is still in flight. The window is ~50µs here, which is why the metal pods mostly get away with it and 4-vCPU hosted runners do not.

Everything else the coordinator does on the root handle — the transaction and the post-commit rebind — was already inside the lock, which is why the reported stack surfaced in rebindOidcProfileResult: it is a victim of the poisoned connection, not the cause.

Fix

Move the tracker initialization inside withProvisioningLocks, still outside the provisioning transaction. Every root-handle statement the coordinator owns — tracker setup, the transaction, the post-commit rebind — is now serialized per database URL on SQLite/DuckDB. Transaction-bound callers are unchanged: they still never run _smrt_backfills DDL.

@happyvertical/smrt-profiles and @happyvertical/smrt-users both route through this one coordinator (@happyvertical/smrt-profiles/internal/oidc-provisioning), so the single change covers both failing suites — there is no second copy in smrt-users. No assertion was weakened: maxActiveLookups/maxActiveResolvers stay 1 and every row-count assertion is untouched.

Postgres root handles are unaffected in substance: they never take the adapter-transaction key, so initialization simply moves under the identity/email locks, and the shared initialization promise makes repeat calls a single cached check.

Regression guard

packages/profiles/src/__tests__/oidc-provisioning-coordinator.test.ts asserts the ordering contract deterministically rather than by racing: it blocks one flow inside its provisioning transaction, starts a second flow, and asserts the second issues no statement on the shared DuckDB root handle. Reverting only the fix turns it red with the precise offender:

AssertionError: expected [ Array(1) ] to deeply equal []
+ [ "SELECT 1 FROM _smrt_backfills LIMIT 1" ]

Validation

• @happyvertical/smrt-profiles full suite — 17 files, 209 passed (was 208; +1 new test)
• @happyvertical/smrt-users — oidc-provisioning-safety.test.ts 93 passed; whole node suite 22 files / 421 passed. The 7 Svelte component files cannot load in an agent worktree (vite8 + rolldown Could not resolve 'node:module'), so CI is the gate for those.
• 20/20 local repeats of oidc-account-linking.test.ts -t "independent DuckDB handles"
• 20/20 local repeats of oidc-provisioning-safety.test.ts -t "one DuckDB root handle"
• test(oidc): stabilize DuckDB transaction fixtures #2191-style --sequence.shuffle probe: 6/6 green on each OIDC file (43 and 93 tests)
• pnpm typecheck (profiles, users), biome check on touched files, pnpm check:agents-chain, pnpm smrt dev:knowledge-check — clean
• The real acceptance test is this PR's own hosted affected-package-tests shards.

No changeset is committed — release automation generates them on merge.

---

Update: hosted CI found a second, dominant hazard

Run 31994388100 on 5edce0dbd (lock-ordering fix only) shows the fix above was necessary but not sufficient:

• shard 3/3 — green
• shard 1/3 — @happyvertical/smrt-users#test, all 3 retries, Failed to execute raw query at UserCollection.ts:726 inside withProvisioningLock
• shard 2/3 — @happyvertical/smrt-profiles#test, silent worker abort (16/17 files, 201/209 tests) — the issue's other reported signature

Both failures are inside the coordinator lock, where no other flow can run. So the remaining overlap is a flow racing itself: rebindOidcProvisioningResult / rebindOidcProfileResult issued their two or three primary-key reads through Promise.all on one native DuckDB connection. That is why the originally reported profiles stack pointed at SELECT * FROM profiles WHERE id = $1 in the rebind — it was the direct cause, not a downstream victim. My local probe passed 400 rounds of that shape on macOS/arm64; the hosted x86 runners do not tolerate it.

Second fix (c7bca9561): the rebind reads and the LIMIT 2 owner/email candidate reads are now sequential in both packages. They are primary-key lookups — one or two extra round trips on a first login, immaterial on pooled Postgres.

Second guard: never overlaps two statements on one shared DuckDB connection during concurrent provisioning counts in-flight statements across the root handle and its transaction handle while two real createProfileFromOidc flows run. It fails on structure, not on a race — restoring either Promise.all turns it red naming the duplicated statement:

AssertionError: expected [ …(2) ] to deeply equal []
+ "SELECT * FROM oidc_identities WHERE id = $1",
+ "SELECT * FROM oidc_identities WHERE id = $1",

This guard would have caught the original defect on any machine, which the timing-dependent suites could not.

Revalidation: profiles 17 files / 210 passed; users node suite 22 files / 421 passed (7 Svelte files need CI — vite8+rolldown cannot load the plugin in an agent worktree); 20/20 repeats of the users DuckDB test; 6/6 shuffle on the guard file; typecheck, biome, agents-chain, knowledge-check clean.

{"schema":"hv-agent-run:v1","policy_revision":"1.0.0","runtime":"claude","session":"bb4c7a21-74b8-4fce-972c-107c341951d9","issue":"https://github.com/happyvertical/smrt/issues/2352"}

[Copilot]

Pull request overview

This PR fixes a DuckDB/SQLite shared-connection race in OIDC provisioning by ensuring _smrt_backfills initialization is serialized under the same coordinator locks as the provisioning transaction and post-commit rebind, and adds a deterministic regression test to enforce that ordering contract.

Changes:

• Move _smrt_backfills tracker initialization inside withProvisioningLocks (but still outside the provisioning transaction) to prevent root-handle statements from overlapping a concurrent flow’s open transaction.
• Add a new coordinator ordering regression test for DuckDB shared root handles.
• Update package AGENTS.md documentation in Profiles and Users to reflect the expanded “serialize every coordinator-owned root-handle statement” contract.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
packages/profiles/src/auth/oidcProvisioningCoordinator.ts	Serializes backfill tracker initialization under coordinator locks to prevent overlapping root-handle statements with open transactions.
packages/profiles/src/tests/oidc-provisioning-coordinator.test.ts	Adds deterministic regression coverage ensuring no root-handle statements occur while another flow holds the provisioning transaction.
packages/profiles/AGENTS.md	Documents the updated coordinator serialization contract for SQLite/DuckDB.
packages/users/AGENTS.md	Mirrors the updated coordinator serialization contract documentation for Users.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.