From 51578cdee8434c0300d58ad889d18cfcf8f31961 Mon Sep 17 00:00:00 2001 From: Ori Nachum Date: Fri, 3 Apr 2026 07:58:40 +0300 Subject: [PATCH 1/2] docs: add CLAUDE.md with project guidance for Claude Code Provides architecture overview, test commands, CI markers, and contributor workflow so Claude Code sessions start with full context. Co-Authored-By: Claude Opus 4.6 (1M context) --- CLAUDE.md | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..75c1250 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,84 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +Open Bedrock Server is a unified, provider-agnostic chat completions API server. It exposes a single `/v1/chat/completions` endpoint that auto-detects request format (OpenAI, Bedrock Claude, Bedrock Titan), routes to the appropriate provider based on model ID, and converts between formats seamlessly. Built with FastAPI + Pydantic, uses boto3 for AWS Bedrock and the openai SDK for OpenAI. + +## Commands + +```bash +# Install dependencies +uv sync --dev + +# Run all safe tests (no real API calls) — default for development +uv run python run_tests.py --mode all-safe -v + +# Run only unit tests +uv run python run_tests.py --mode unit -v + +# Run a single test file +uv run pytest tests/test_chat.py -v + +# Run a single test by name +uv run pytest -k "test_openai_format_detection" -v + +# Run tests by marker +uv run pytest -m "unit" -v +uv run pytest -m "integration and not real_api" -v + +# Run real API tests (requires credentials, costs money) +uv run python run_tests.py --mode real-api -v + +# Lint +black src tests +isort src tests +flake8 src tests +mypy src + +# Start the server +bedrock-chat serve --host 0.0.0.0 --port 8000 + +# Interactive chat +bedrock-chat chat --model gpt-4o-mini --stream +``` + +## Architecture + +### Source layout: `src/open_bedrock_server/` + +**Request flow:** Client → FastAPI routes (`api/routes/chat.py`) → `RequestFormatDetector` (detects OpenAI/Bedrock Claude/Bedrock Titan) → `LLMServiceFactory.get_service_for_model()` (routes by model ID prefix) → provider service → response conversion → client. + +**Key layers:** + +- **`api/`** — FastAPI app, routes (chat, files, health, models, knowledge_bases), auth + logging middleware, request/response schemas. +- **`services/`** — `AbstractLLMService` (ABC in `llm_service_abc.py`) defines the contract. `OpenAIService` and `BedrockService` implement it. `LLMServiceFactory` resolves provider from model ID (prefix-based: `gpt-*` → OpenAI, `anthropic.*`/`amazon.*`/`meta.*`/etc. → Bedrock). File services handle S3-backed file upload/query. +- **`adapters/`** — Format conversion layer. `BaseLLMAdapter` (ABC) defines convert-to/from-provider methods. `OpenAIAdapter` handles OpenAI. `BedrockToOpenAIAdapter` accepts Bedrock format and delegates to OpenAI. The `bedrock/` subdirectory uses a **strategy pattern**: `BedrockAdapterStrategyABC` base with per-vendor strategies (`claude_strategy.py`, `titan_strategy.py`, `nova_strategy.py`, `mistral_strategy.py`, `ai21_strategy.py`, `cohere_strategy.py`, `meta_strategy.py`, `stability_strategy.py`, `writer_strategy.py`). +- **`core/`** — Pydantic models (`models.py` for OpenAI-compatible types, `bedrock_models.py` for Bedrock types including `RequestFormat` enum), custom exceptions. +- **`utils/`** — `RequestFormatDetector` (auto-detects input format), `config_loader` (env-based config via pydantic-settings), `knowledge_base_detector`. +- **`cli/`** — Click-based CLI (`bedrock-chat` entry point) for server management, interactive chat, config. + +### Adding a new Bedrock model family + +1. Create a new strategy in `adapters/bedrock/` implementing `BedrockAdapterStrategyABC` +2. Register the model prefix in `LLMServiceFactory.get_service_for_model()` and `BaseLLMAdapter._get_default_param()` +3. Add to `get_supported_models()` in `llm_service_factory.py` + +## Test Markers + +Defined in `pytest.ini`. CI runs `unit` + `integration and not real_api`. Key markers: + +- `unit` — fast, no external deps +- `integration` — may use mocks +- `real_api` / `external_api` — real API calls, skipped in CI +- `openai_integration` — needs `OPENAI_API_KEY` +- `aws_integration` / `bedrock_integration` — needs AWS credentials + +## CI + +GitHub Actions (`ci-tests.yml`) runs on push/PR to main/develop with Python 3.12 + 3.13. Runs unit → mocked integration → all-safe tests. Real API tests are a separate manual workflow (`real-api-tests.yml`). + +## Configuration + +Uses `.env` file with pydantic-settings. Key vars: `API_KEY` (server auth), `OPENAI_API_KEY`, AWS credentials (`AWS_PROFILE`/`AWS_ACCESS_KEY_ID`+`AWS_SECRET_ACCESS_KEY`/`AWS_ROLE_ARN`), `AWS_REGION`, `S3_FILES_BUCKET` (for file query feature), `DEFAULT_OPENAI_MODEL`. From 041507cbc6848f581c14b0eac874857ae810c887 Mon Sep 17 00:00:00 2001 From: Ori Nachum Date: Fri, 3 Apr 2026 08:08:33 +0300 Subject: [PATCH 2/2] fix: address PR review comments on CLAUDE.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Clarify primary endpoint vs additional routes - Prefix lint commands with uv run - Fix BedrockAdapterStrategyABC → BedrockAdapterStrategy - Fix pydantic-settings → python-dotenv/os.getenv - Remove nonexistent DEFAULT_OPENAI_MODEL env var Co-Authored-By: Claude Opus 4.6 (1M context) --- CLAUDE.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 75c1250..33e6589 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -Open Bedrock Server is a unified, provider-agnostic chat completions API server. It exposes a single `/v1/chat/completions` endpoint that auto-detects request format (OpenAI, Bedrock Claude, Bedrock Titan), routes to the appropriate provider based on model ID, and converts between formats seamlessly. Built with FastAPI + Pydantic, uses boto3 for AWS Bedrock and the openai SDK for OpenAI. +Open Bedrock Server is a unified, provider-agnostic chat completions API server. Its primary `/v1/chat/completions` endpoint auto-detects request format (OpenAI, Bedrock Claude, Bedrock Titan), routes to the appropriate provider based on model ID, and converts between formats seamlessly. Additional endpoints cover files, models, health, and knowledge bases. Built with FastAPI + Pydantic, uses boto3 for AWS Bedrock and the openai SDK for OpenAI. ## Commands @@ -32,10 +32,10 @@ uv run pytest -m "integration and not real_api" -v uv run python run_tests.py --mode real-api -v # Lint -black src tests -isort src tests -flake8 src tests -mypy src +uv run black src tests +uv run isort src tests +uv run flake8 src tests +uv run mypy src # Start the server bedrock-chat serve --host 0.0.0.0 --port 8000 @@ -54,14 +54,14 @@ bedrock-chat chat --model gpt-4o-mini --stream - **`api/`** — FastAPI app, routes (chat, files, health, models, knowledge_bases), auth + logging middleware, request/response schemas. - **`services/`** — `AbstractLLMService` (ABC in `llm_service_abc.py`) defines the contract. `OpenAIService` and `BedrockService` implement it. `LLMServiceFactory` resolves provider from model ID (prefix-based: `gpt-*` → OpenAI, `anthropic.*`/`amazon.*`/`meta.*`/etc. → Bedrock). File services handle S3-backed file upload/query. -- **`adapters/`** — Format conversion layer. `BaseLLMAdapter` (ABC) defines convert-to/from-provider methods. `OpenAIAdapter` handles OpenAI. `BedrockToOpenAIAdapter` accepts Bedrock format and delegates to OpenAI. The `bedrock/` subdirectory uses a **strategy pattern**: `BedrockAdapterStrategyABC` base with per-vendor strategies (`claude_strategy.py`, `titan_strategy.py`, `nova_strategy.py`, `mistral_strategy.py`, `ai21_strategy.py`, `cohere_strategy.py`, `meta_strategy.py`, `stability_strategy.py`, `writer_strategy.py`). +- **`adapters/`** — Format conversion layer. `BaseLLMAdapter` (ABC) defines convert-to/from-provider methods. `OpenAIAdapter` handles OpenAI. `BedrockToOpenAIAdapter` accepts Bedrock format and delegates to OpenAI. The `bedrock/` subdirectory uses a **strategy pattern**: `BedrockAdapterStrategy` base with per-vendor strategies (`claude_strategy.py`, `titan_strategy.py`, `nova_strategy.py`, `mistral_strategy.py`, `ai21_strategy.py`, `cohere_strategy.py`, `meta_strategy.py`, `stability_strategy.py`, `writer_strategy.py`). - **`core/`** — Pydantic models (`models.py` for OpenAI-compatible types, `bedrock_models.py` for Bedrock types including `RequestFormat` enum), custom exceptions. -- **`utils/`** — `RequestFormatDetector` (auto-detects input format), `config_loader` (env-based config via pydantic-settings), `knowledge_base_detector`. +- **`utils/`** — `RequestFormatDetector` (auto-detects input format), `config_loader` (env-based config via `python-dotenv` + `os.getenv`), `knowledge_base_detector`. - **`cli/`** — Click-based CLI (`bedrock-chat` entry point) for server management, interactive chat, config. ### Adding a new Bedrock model family -1. Create a new strategy in `adapters/bedrock/` implementing `BedrockAdapterStrategyABC` +1. Create a new strategy in `adapters/bedrock/` implementing `BedrockAdapterStrategy` 2. Register the model prefix in `LLMServiceFactory.get_service_for_model()` and `BaseLLMAdapter._get_default_param()` 3. Add to `get_supported_models()` in `llm_service_factory.py` @@ -81,4 +81,4 @@ GitHub Actions (`ci-tests.yml`) runs on push/PR to main/develop with Python 3.12 ## Configuration -Uses `.env` file with pydantic-settings. Key vars: `API_KEY` (server auth), `OPENAI_API_KEY`, AWS credentials (`AWS_PROFILE`/`AWS_ACCESS_KEY_ID`+`AWS_SECRET_ACCESS_KEY`/`AWS_ROLE_ARN`), `AWS_REGION`, `S3_FILES_BUCKET` (for file query feature), `DEFAULT_OPENAI_MODEL`. +Uses environment variables loaded from a `.env` file via `python-dotenv`, with values read in `utils/config_loader.py`. Key vars: `API_KEY` (server auth), `OPENAI_API_KEY`, AWS credentials (`AWS_PROFILE`/`AWS_ACCESS_KEY_ID`+`AWS_SECRET_ACCESS_KEY`/`AWS_ROLE_ARN`), `AWS_REGION`, `S3_FILES_BUCKET` (for file query feature).