Skip to content

Persist indexer webhook replay protection in Postgres for cross-instance idempotency #187

Description

@greatest0fallt1me

Description

EventIngestionService in src/services/eventIngestionService.ts deduplicates eventId values using an in-process private readonly processedEventIds = new Set<string>(). docs/SECURITY.md and README.md advertise replay protection, but this set is lost on restart and not shared across instances, so a redelivered event after a deploy or on another replica would be reprocessed. This issue moves the idempotency record into Postgres so duplicate deliveries remain safe no-ops fleet-wide.

Requirements and context

  • Add a processed_indexer_events table (Drizzle schema + migration) keyed by eventId with a receivedAt timestamp and a unique constraint.
  • Replace the in-memory Set with a repository-backed check: on ingest, attempt an insert and treat a unique-violation as a duplicate (duplicate: true), preserving the existing IngestionResult contract and 202/200 semantics in src/routes/webhooks/indexer.ts.
  • Keep reset() working for tests (e.g. truncate or delegate to an injectable store) so the existing suite stays deterministic.
  • Non-functional: the duplicate check must be atomic (rely on the DB unique constraint, not a read-then-write race); failures must not silently accept unverified events.

Acceptance criteria

  • A new processed_indexer_events table and migration exist with a unique eventId.
  • First delivery of an eventId returns duplicate: false; any later delivery returns duplicate: true even across fresh service instances.
  • Signature/payload validation still runs before the idempotency check.
  • src/indexerWebhook.test.ts duplicate-delivery expectations pass against the persistent path (with the store mocked/faked).
  • npm run build and npm test pass.
  • docs/SECURITY.md documents that replay protection is now durable.

Suggested execution

1. Fork the repo and create a branchgit checkout -b feature/persistent-replay-protection.
2. Implement changes — add the table to src/db/schema.ts, a repository for it, and rewire EventIngestionService to use it (constructor-injected per docs/TESTING.md).
3. Write/extend tests — this repo uses Jest + Supertest. Extend src/indexerWebhook.test.ts and add a service-level test injecting a fake store to verify unique-violation handling.
4. Test and commit

npm run lint -- --max-warnings 0
npm run build
npm test

Example commit message

feat(security): persist indexer event idempotency in Postgres

Guidelines

Target >=90% coverage on the new repository/service paths (repo standard 95% per jest.config.js). Update docs/SECURITY.md and docs/data-model.md. Timeframe: 96 hours.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions