Skip to content

feat(#279): cross-language contract test — backend client ↔ AI service Pydantic schemas#332

Merged
kilodesodiq-arch merged 2 commits into
ChainForgee:mainfrom
samsolo247:feat/issue-279-ai-service-contract-tests
Jul 19, 2026
Merged

feat(#279): cross-language contract test — backend client ↔ AI service Pydantic schemas#332
kilodesodiq-arch merged 2 commits into
ChainForgee:mainfrom
samsolo247:feat/issue-279-ai-service-contract-tests

Conversation

@samsolo247

Copy link
Copy Markdown
Contributor

Summary

Closes #279

Adds the contract test layer described in issue #279 (backlog item #71) to catch JSON schema drift between the NestJS backend client and the FastAPI AI service Pydantic schemas before it reaches production.

Changes

app/backend/scripts/openapi-validate.ts (new)

Shared utility and standalone CLI that:

  • Fetches the AI service OpenAPI JSON document over HTTP/HTTPS
  • Resolves $ref pointers within the document
  • Extracts response schemas by path + method + status code
  • Structurally validates a plain object against a schema (required fields, types, enum membership)

Run standalone to check spec drift without tests:

AI_SERVICE_URL=http://localhost:8000 ts-node --transpile-only scripts/openapi-validate.ts

app/backend/test/contract/ai-service.contract.spec.ts (new)

Contract test suite with five assertion groups:

Group What it checks
Document structure openapi version string, info, components.schemas
Required v1 paths All 6 endpoints the backend depends on exist with correct HTTP method + 200 response
Enum correctness provider_preference values (auto/test/openai/groq); TaskStatus lifecycle values
Required fields Every response schema's required[] array matches the TypeScript interfaces the backend compiles against
Fixture validation Real app/ai-service/fixtures/*.json payloads are validated against the declared schemas
Cross-field consistency Optional vs required fields (e.g. verification/error on humanitarian response)

Live vs CI mode — when AI_SERVICE_URL is set the spec is fetched live; otherwise the test falls back silently to an embedded snapshot so CI runs without a live AI service.

Compile-time type guards — TypeScript asserts functions for all five response interfaces cause a build failure if the backend's internal types drift from the schema shapes.

How to run

cd app/backend
# unit test runner (uses embedded snapshot, no AI service needed)
npm test -- --testPathPattern="test/contract/ai-service.contract.spec"

# with a live AI service
AI_SERVICE_URL=http://localhost:8000 npm test -- --testPathPattern="test/contract/ai-service.contract.spec"

Files affected

  • app/backend/scripts/openapi-validate.ts — new
  • app/backend/test/contract/ai-service.contract.spec.ts — new

…service schemas

Implements issue ChainForgee#279 (backlog item ChainForgee#71).

New files:
- app/backend/scripts/openapi-validate.ts
    Shared helpers to fetch the AI service OpenAPI document, resolve $refs,
    and structurally validate a plain object against a schema.  Also runnable
    as a standalone CLI for quick spec drift checks.

- app/backend/test/contract/ai-service.contract.spec.ts
    Contract test that:
    • Obtains the AI service OpenAPI spec (live fetch when AI_SERVICE_URL is
      set; falls back to an embedded snapshot so CI always runs).
    • Asserts all six v1 endpoints the backend depends on exist in the spec.
    • Verifies provider_preference and TaskStatus enum values.
    • Checks required fields on every response schema against the TypeScript
      interfaces the backend compiles against.
    • Validates real AI service fixture payloads (app/ai-service/fixtures/)
      against those schemas — catching enum drift or missing fields before
      they reach production.
    • Compile-time type guards ensure backend interfaces stay in sync with
      the schema shapes (fails the build on shape divergence).

Copy link
Copy Markdown
Contributor

Hi 👋 Appreciate this PR! The CI is flagging a failing check — could you take a peek at the failed job and push a fix? Happy to help if you need a hand 🙏

ESLint rule @typescript-eslint/no-unused-vars (error) was failing CI:
  'SchemaObject' is defined but never used.
  Allowed unused vars must match /^_/u

SchemaObject was imported but never referenced in the test file.
Removing it is the clean fix — it can be re-added with an _ prefix
if it becomes needed in future.
@samsolo247

Copy link
Copy Markdown
Contributor Author

Hi 👋 Appreciate this PR! The CI is flagging a failing check — could you take a peek at the failed job and push a fix? Happy to help if you need a hand 🙏

Done.
Kindly review again.

@kilodesodiq-arch kilodesodiq-arch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kilodesodiq-arch
kilodesodiq-arch merged commit 66ae6e3 into ChainForgee:main Jul 19, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cross-language contract test: backend client ↔ AI service Pydantic schemas

3 participants