Repository navigation
Conversation
Add a node_modules_dir config option that installs the PGlite npm packages once and links them into each work directory, so parallel test workers (for example pytest-xdist) no longer race to install the same packages in their own directories. Installs are serialised with a file lock that the OS releases if the installer dies, and the shared install is refreshed automatically when the dependencies change. The npm pins move to the latest compatible releases (pglite 0.5.8, pglite-socket 0.2.11, pglite-pgvector 0.0.9), which means pgvector now comes from its own package, and the generated server script waits for db.waitReady before accepting connections. The readiness probe now performs a real PostgreSQL startup handshake instead of a bare connect-and-close, because the newer socket server drops the next connection after an abrupt probe disconnect.
The readiness probe performs a PostgreSQL startup handshake, so tests that mock socket.socket and call start() must stub _postgres_startup_probe as well: a mocked socket cannot speak the protocol.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Parallel test workers (for example pytest-xdist with
-n 8) each install the PGlite npm packages into their own work directory today, so a test run does onenpm installper worker. This adds anode_modules_dirconfig option that installs them once, guarded by a cross-process file lock, and links the result into each work directory:flock/msvcrt.locking) is released by the OS if the installer dies, so a crashed process cannot leave a stale lock.node_modules_diris not set (install into the work directory).Package alignment
The branch moves the npm pins to the latest compatible releases:
@electric-sql/pglite^0.5.8@electric-sql/pglite-socket^0.2.11@electric-sql/pglite-pgvector^0.0.9Since pglite 0.5.x moved vector out of the main bundle,
SUPPORTED_EXTENSIONS["pgvector"]now points at@electric-sql/pglite-pgvector, matching #52's approach.Readiness fix
With the newer socket server, the existing readiness probe (bare connect + immediate close) causes the next connection to be dropped. Two changes make startup deterministic:
db.waitReadybeforeserver.start(), so the socket only accepts once PGlite can serve queries;Without this,
tests/test_extensions.py::test_pgvector_extensionfails withserver closed the connection unexpectedlyon the new packages.Overlap
update-pglite) already brings the 0.5.0 migration and pgvector package; this branch carries the newer 0.5.8 pin set plus the readiness changes. Happy to rebase once fix: update pglite 0.5.0, allow multiple concurrent connections #52 lands.package.jsoncontent to be configurable;_package_json_content()here is the single place that content is generated, which composes cleanly with that.Testing
tests/test_shared_node_modules.py: shared install reuse, stale-install invalidation, linking over an existing directory, lock timeout, package pins, startup probe, generated-script ordering.tests/test_extensions.py::test_pgvector_extensionpasses with the new packages.