feat(litrag): Section 12 infrastructure — Dockerfile, docker-compose, k8s, Helm chart, CI workflow - #8
Conversation
…r-compose, k8s manifests, Helm chart, CI workflow - Multi-stage Dockerfile (python:3.11-slim builder→runtime, port 8001) - docker-compose.yml: litrag-svc, torchserve, postgres+pgvector, mlflow, prometheus — all services start healthy (validated) - k8s/: 6 manifests — litrag-deployment (2 replicas, CPU 1 core, Mem 2Gi), torchserve-deployment (1 replica, CPU 2 cores, Mem 4Gi, PVC), litrag-service (ClusterIP:8001), litrag-hpa (min=2 max=6, CPU>70%, queue>10), litrag-configmap, litrag-secret (template) - helm/: Chart.yaml, values.yaml (all Section 12.3 values), templates/ (_helpers.tpl + 6 templates) - .github/workflows/ci.yml: pytest+coverage, Docker build+push-to-GHCR, docker compose integration smoke test - requirements.txt: all runtime deps pinned (fastapi, sqlalchemy, anthropic, openai, loguru, prometheus-client, spacy, tiktoken, aiohttp, numpy, etc.) - monitoring/prometheus.yml: litrag-svc + torchserve scrape targets - serving/config.properties: TorchServe config (token auth disabled for dev) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e216a7fe2f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| pip install httpx pytest pytest-asyncio pydantic pydantic-settings | ||
| pytest tests/litrag/test_api_contracts.py -v --tb=short |
There was a problem hiding this comment.
On a clean GitHub runner this step only installs httpx, pytest, pytest-asyncio, pydantic, and pydantic-settings, but tests/litrag/test_api_contracts.py imports api.main.create_app, which pulls in fastapi, loguru, prometheus_client, sqlalchemy, and the rest of the app stack. Because jobs do not share the earlier test environment, pytest tests/litrag/test_api_contracts.py will fail during import instead of exercising the compose deployment.
Useful? React with 👍 / 👎.
| --set-string litrag.secrets.openaiApiKey="sk-..." \ | ||
| --set-string litrag.secrets.databaseUrl="postgresql+asyncpg://..." | ||
| */}} | ||
| {{- if or .Values.litrag.secrets.anthropicApiKey .Values.litrag.secrets.openaiApiKey .Values.litrag.secrets.databaseUrl }} |
There was a problem hiding this comment.
DATABASE_URL when the bundled Postgres chart is enabled
With the current defaults (postgres.enabled: true and litrag.secrets.databaseUrl: ""), this condition prevents the LitRAG Secret from being rendered, and no other Helm template injects DATABASE_URL. The app therefore falls back to shared.config.BaseConfig.database_url (...@localhost:5432/onco_lens), which points at the LitRAG pod itself rather than the PostgreSQL subchart service, so a default helm install leaves the API stuck failing its database health check.
Useful? React with 👍 / 👎.
| # Download the scispaCy NER model into the installed prefix | ||
| RUN PYTHONPATH=/install/lib/python3.11/site-packages \ | ||
| python -m spacy download en_core_sci_sm --target /install/lib/python3.11/site-packages || true |
There was a problem hiding this comment.
The image downloads en_core_sci_sm, but ingestion/pipeline.py constructs NERExtractor() and litrag/ner_extractor.py defaults that to en_ner_bc5cdr_md. In other words, the Docker image never contains the model the ingestion pipeline asks spaCy to load, and the trailing || true keeps the build green, so the first containerized ingestion run will fail with a “model not installed” error.
Useful? React with 👍 / 👎.
1 participant
Summary
Services in docker-compose.yml
Kubernetes manifests (k8s/)
Helm chart (helm/)
CI ()
Three jobs:
Validation
{"status":"healthy","components":{"database":{"ok":true,"latency_ms":21.7,"detail":null},"torchserve":{"ok":true,"latency_ms":25.1,"detail":null},"openai":{"ok":true,"latency_ms":164.7,"detail":null}},"timestamp":"2026-03-18T10:37:58.378342Z"}# HELP python_gc_objects_collected_total Objects collected during gc
TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 11519.0
python_gc_objects_collected_total{generation="1"} 3669.0
python_gc_objects_collected_total{generation="2"} 1068.0
HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
HELP python_gc_collections_total Number of times this generation was collected
TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 389.0
python_gc_collections_total{generation="1"} 35.0
python_gc_collections_total{generation="2"} 3.0
HELP python_info Python platform information
TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="11",patchlevel="15",version="3.11.15"} 1.0
HELP process_virtual_memory_bytes Virtual memory size in bytes.
TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 1.1418624e+09
HELP process_resident_memory_bytes Resident memory size in bytes.
TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 1.31760128e+08
HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
TYPE process_start_time_seconds gauge
process_start_time_seconds 1.77383010766e+09
HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 5.78
HELP process_open_fds Number of open file descriptors.
TYPE process_open_fds gauge
process_open_fds 16.0
HELP process_max_fds Maximum number of open file descriptors.
TYPE process_max_fds gauge
process_max_fds 1.048576e+06
HELP litrag_http_requests_total Total HTTP requests processed, labelled by method, endpoint, and status code.
TYPE litrag_http_requests_total counter
litrag_http_requests_total{endpoint="/internal/health",method="GET",status_code="200"} 11.0
litrag_http_requests_total{endpoint="/metrics",method="GET",status_code="200"} 12.0
HELP litrag_http_requests_created Total HTTP requests processed, labelled by method, endpoint, and status code.
TYPE litrag_http_requests_created gauge
litrag_http_requests_created{endpoint="/internal/health",method="GET",status_code="200"} 1.7738301136853943e+09
litrag_http_requests_created{endpoint="/metrics",method="GET",status_code="200"} 1.7738301165035996e+09
HELP litrag_http_request_duration_seconds HTTP request latency in seconds, labelled by method and endpoint.
TYPE litrag_http_request_duration_seconds histogram
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.005",method="GET"} 0.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.01",method="GET"} 0.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.025",method="GET"} 0.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.05",method="GET"} 0.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.075",method="GET"} 0.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.1",method="GET"} 0.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.25",method="GET"} 11.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.5",method="GET"} 11.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="0.75",method="GET"} 11.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="1.0",method="GET"} 11.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="2.5",method="GET"} 11.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="5.0",method="GET"} 11.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="7.5",method="GET"} 11.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="10.0",method="GET"} 11.0
litrag_http_request_duration_seconds_bucket{endpoint="/internal/health",le="+Inf",method="GET"} 11.0
litrag_http_request_duration_seconds_count{endpoint="/internal/health",method="GET"} 11.0
litrag_http_request_duration_seconds_sum{endpoint="/internal/health",method="GET"} 2.0382909380007277
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.005",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.01",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.025",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.05",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.075",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.1",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.25",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.5",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="0.75",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="1.0",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="2.5",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="5.0",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="7.5",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="10.0",method="GET"} 12.0
litrag_http_request_duration_seconds_bucket{endpoint="/metrics",le="+Inf",method="GET"} 12.0
litrag_http_request_duration_seconds_count{endpoint="/metrics",method="GET"} 12.0
litrag_http_request_duration_seconds_sum{endpoint="/metrics",method="GET"} 0.013048805999915203
HELP litrag_http_request_duration_seconds_created HTTP request latency in seconds, labelled by method and endpoint.
TYPE litrag_http_request_duration_seconds_created gauge
litrag_http_request_duration_seconds_created{endpoint="/internal/health",method="GET"} 1.7738301136853173e+09
litrag_http_request_duration_seconds_created{endpoint="/metrics",method="GET"} 1.7738301165035522e+09
HELP litrag_active_requests Number of HTTP requests currently being processed.
TYPE litrag_active_requests gauge
litrag_active_requests 1.0
HELP litrag_rag_stage_duration_seconds RAG pipeline stage latency in seconds. Stages: embed, retrieve, rerank, score, synthesize, contradiction, gap.
TYPE litrag_rag_stage_duration_seconds histogram
HELP litrag_query_cache_hits_total Total number of query response cache hits (QRY-010).
TYPE litrag_query_cache_hits_total counter
litrag_query_cache_hits_total 0.0
HELP litrag_query_cache_hits_created Total number of query response cache hits (QRY-010).
TYPE litrag_query_cache_hits_created gauge
litrag_query_cache_hits_created 1.7738301103781059e+09
HELP litrag_query_cache_misses_total Total number of query response cache misses (QRY-010).
TYPE litrag_query_cache_misses_total counter
litrag_query_cache_misses_total 0.0
HELP litrag_query_cache_misses_created Total number of query response cache misses (QRY-010).
TYPE litrag_query_cache_misses_created gauge
litrag_query_cache_misses_created 1.773830110378118e+09
HELP litrag_contradiction_timeouts_total Total number of contradiction detection tasks that exceeded the 5-second timeout (CON-003).
TYPE litrag_contradiction_timeouts_total counter
litrag_contradiction_timeouts_total 0.0
HELP litrag_contradiction_timeouts_created Total number of contradiction detection tasks that exceeded the 5-second timeout (CON-003).
TYPE litrag_contradiction_timeouts_created gauge
litrag_contradiction_timeouts_created 1.7738301103781302e+09
HELP litrag_ingestion_runs_total Total ingestion pipeline runs triggered, labelled by terminal status.
TYPE litrag_ingestion_runs_total counter
HELP litrag_scorer_invocations_total Total evidence scorer invocations, labelled by whether the fallback was used.
TYPE litrag_scorer_invocations_total counter
HELP litrag_ner_chunk_lookups_total Total GET /internal/chunks/{id}/ner requests, labelled by whether the chunk was found.
TYPE litrag_ner_chunk_lookups_total counter
HELP litrag_evidence_grade_last Evidence grade of the most recent query response encoded as an integer (A=4, B=3, C=2, D=1).
TYPE litrag_evidence_grade_last gauge
litrag_evidence_grade_last 0.0
🤖 Generated with Claude Code