Skip to content

indexer: add Postgres-backed Database implementation#585

Open
Meet-hybrid wants to merge 2 commits into
Epta-Node:mainfrom
Meet-hybrid:feat/social-graph-materialization
Open

indexer: add Postgres-backed Database implementation#585
Meet-hybrid wants to merge 2 commits into
Epta-Node:mainfrom
Meet-hybrid:feat/social-graph-materialization

Conversation

@Meet-hybrid

Copy link
Copy Markdown

Closes #568

Add Postgres-backed Database implementation for the indexer (services/indexer/src/postgres-db.ts) implementing the existing Database interface.
SQL-backed implementations cover profiles, follows, posts, likes, tips, and pools, enabling materialized data access for Express routes without contract reads.
Update TODO.md to mark Phase 1 Step 1 as complete.
Regenerate/update services/indexer/package-lock.json.

@devJaja devJaja self-requested a review June 21, 2026 21:01

@devJaja devJaja left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @Meet-hybrid — implementing the Database interface is the right first step. The code is clean, all queries are parameterised, and the TODO.md roadmap gives a clear picture of what remains. But this PR does not close #568 as claimed, and there are a few code issues that need to be fixed.

Does this close #568?
No. Issue #568 requires all three phases: social graph materialization (Phase 1), the /api/feed/following endpoint (Phase 2), and the weighted Explore feed (Phase 3). This PR delivers only the PostgresDatabase class — a data-access layer with no tests, no API endpoints, no follow_counts table, no event subscriptions, and no feed queries. The Closes #568 in the PR description is incorrect. Please update the description to say this is Phase 1 Step 1 of a multi-PR series, and only link the issue once the full acceptance criteria are met.

Blocking issues

  1. insertPost discards the post content — hardcodes ''
    await this.pool.query( INSERT INTO posts (id, author, content, tip_total, like_count, ...) VALUES ($1, $2, '', $3, $4, ...) -- content is always empty string )

The Post type presumably has a content field. This silently drops all post text on insert. Any search or display feature that reads content from the DB will see empty strings. This looks like an incomplete refactor.

  1. insertFollow stores follow.ledger in a created_at TIMESTAMPTZ column using to_timestamp() INSERT INTO follows (follower, followee, created_at)
    VALUES ($1, $2, $3) -- follow.ledger is a ledger sequence number, not a Unix timestamp
    A ledger sequence number (e.g. 1234567) passed to to_timestamp() becomes a timestamp in mid-January 1970. This is a data type mismatch. The follows table schema in the issue specifies followed_at TIMESTAMPTZ. If the source data is a ledger sequence, it should be stored in a ledger integer column, not converted to a timestamp. insertPost and upsertLike have the same issue.

  2. addPoolAdmin uses $1::text to append a single string to a text array
    SET admins = (
    SELECT ARRAY(
    SELECT DISTINCT a
    FROM unnest(admins || $1::text) AS a
    )
    )

admins || $1::text attempts to concatenate a text[] column with a scalar text. PostgreSQL will reject this at runtime — the operator is || for array concatenation, which requires text[] on both sides. It should be admins || ARRAY[$1::text]. This will produce a runtime error every time addPoolAdmin is called.

Non-blocking
- No tests. The TODO.md defers tests to Step 5, but a PostgresDatabase implementation without a single integration or unit test (even with a mock pool) is hard to validate. At minimum, add a few mock-pool unit tests for the critical paths (insertPost, insertFollow, addPoolAdmin) before this lands.

  • listPosts uses offset pagination, not keyset pagination. The issue specifically requires keyset pagination on the feed endpoints for consistency under concurrent inserts. The underlying listPosts using OFFSET is fine for now as a general-purpose method, but document that the feed endpoints must not use this method directly.
  • getPost maps created_ledger via extract(epoch from created_at)::bigint — this round-trips through the timestamp conversion bug mentioned in point 2. If ledger sequence numbers are stored as timestamps, the recovered value will be wrong.
  • package-lock.json included — pnpm workspace. Remove it.

@vercel

vercel Bot commented Jun 22, 2026

Copy link
Copy Markdown

@Meet-hybrid is attempting to deploy a commit to the Jaja's projects Team on Vercel.

A member of the Team first needs to authorize it.

@Meet-hybrid Meet-hybrid requested a review from devJaja June 22, 2026 01:19

@devJaja devJaja left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Request Changes ⚠️

Thanks for the contribution @Meet-hybrid — the structure is right and the Database interface implementation is a solid foundation. But there are three real bugs that need fixing, and the Closes #568 claim is incorrect.

Does this close #568?
No. Issue #568 requires all three phases: social graph materialization with follow_counts, the /api/feed/following endpoint, and the weighted Explore feed. This PR delivers only the PostgresDatabase class — no API endpoints, no follow_counts table, no event subscriptions, no feed queries, no tests. Please update the PR description to say this is Phase 1 Step 1 of a multi-PR series and remove the Closes #568 link.

Blocking
1. insertPost hardcodes content as an empty string
INSERT INTO posts (id, author, content, tip_total, like_count, ...) VALUES ($1, $2, '', $3, $4, ...) -- content always empty
The Post type has a content field. This silently drops all post text on every insert. Any search or display feature reading content from the DB will see empty strings. Fix: pass post.content as $3 and shift the remaining params.

  1. insertFollow stores follow.ledger (a ledger sequence number) into a created_at TIMESTAMPTZ column via to_timestamp()
    INSERT INTO follows (follower, followee, created_at)
    VALUES ($1, $2, $3) -- follow.ledger is e.g. 1234567, not a Unix timestamp to_timestamp(1234567) produces a date in January 1970. insertPost and upsertLike have the same mismatch. Ledger sequences should be stored in an integer column (created_ledger BIGINT), not converted to a timestamp. Fix the schema mapping or use $3::bigint into a ledger sequence column.

  2. addPoolAdmin uses invalid SQL — admins || $1::text won't compile

SET admins = (
SELECT ARRAY(
SELECT DISTINCT a
FROM unnest(admins || $1::text) AS a -- ERROR: cannot concat text[] with text scalar
)
)

PostgreSQL rejects text[] || text. It must be admins || ARRAY[$1::text]. This will throw a runtime error every time addPoolAdmin is called.

Non-blocking

  • No tests. Even basic mock-pool unit tests for insertPost, insertFollow, and addPoolAdmin would catch the bugs above before review. Please add them before the next iteration.
  • getPost round-trips through the timestamp bug — it reads back created_ledger via extract(epoch from created_at)::bigint, which will return a wrong value because the stored timestamp was wrong to begin with.
  • listPosts uses OFFSET pagination — fine as a general query, but document that feed endpoints must not use this method directly since the issue requires keyset pagination for feed consistency under concurrent inserts.
  • package-lock.json included — pnpm workspace. Remove it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(indexer): build materialized social graph view with follower-count aggregations and weighted feed ranking for Explore and Following feeds

2 participants