security(api): server exposes all endpoints with no authentication (critical)

[jeremylongshore]

Severity: CRITICAL

Filing as a Design Issue per the repo's prefer-issue-over-PR norm so you can red/green the approach before I PR.

What

agent-brain-server's FastAPI app has zero authentication on any endpoint:

• POST /index, POST /index/add, DELETE /index, GET /query, POST /query, GET /query/count, GET /index/folders, GET /index/jobs, GET /index/cache are all open.
• agent-brain-server/agent_brain_server/api/main.py instantiates FastAPI() with no auth dependency. Grep for HTTPBearer, APIKeyHeader, Depends(verify_*) returns zero hits in the API surface.
• The 127.0.0.1 default bind is NOT an authorization control. Any local process on the same machine (any browser tab with the right Origin, any rogue subprocess, any other agent) can read the entire index or wipe it.

Threat model

Single dev box, multiple projects, multiple LLM agents running concurrently (which is the actual agent-brain deployment pattern based on your multi-instance architecture):

1. Agent in project A queries /query and reads embedded chunks of project B's source.
2. Any local script can curl -X DELETE http://127.0.0.1:8000/index and nuke the index.
3. Combined with the CORS wildcard (separate issue), a malicious web page can do all of the above from the browser via DNS rebinding.

OWASP API1:2023 (BOLA) + API2:2023 (Broken Auth).

Proposed fix

Two-layer, opt-out friendly for local-only dev:

1. Add API_KEY: SecretStr | None = None to config/settings.py.
2. Add app/security.py with a verify_api_key FastAPI dependency reading X-API-Key header. If settings.API_KEY is None, the dependency is a no-op (preserves the current local-dev experience for solo users); if set, it 401s on missing/wrong.
3. Apply the dependency to every router except /health, /health/status, /docs, /redoc, /openapi.json via APIRouter(dependencies=[Depends(verify_api_key)]) per file.
4. CLI generates a key on agent-brain init (32 bytes urlsafe), writes to .agent-brain/runtime.json mode 600 (see also #state-dir-perms issue), and reads it for all CLI calls via X-API-Key.
5. .env.example documents API_KEY= and the README gets a one-liner: "for shared hosts, set API_KEY to require auth on the server."
6. Optional hardening: refuse to start if API_HOST is anything other than 127.0.0.1 and API_KEY is unset. Loud warn-and-continue if both 127.0.0.1 and no key (default dev case).

Test strategy

• New fixture in tests/conftest.py: app_with_api_key that sets API_KEY and returns a test client with the right header
• Per-router: 401 on missing header, 401 on wrong key, 200 on correct key
• Existing tests stay green via the no-API_KEY default path
• Contract tests for both modes against Chroma and Postgres backends

Files touched (estimate)

• agent_brain_server/config/settings.py (+1 field)
• agent_brain_server/api/security.py (new, ~40 LOC)
• agent_brain_server/api/routers/{health,query,index,jobs,folders,cache}.py (router-level dependency)
• agent_brain_cli/agent_brain_cli/client/*.py (send X-API-Key)
• agent_brain_cli/agent_brain_cli/commands/init.py (generate + store key)
• .env.example, README.md (docs)
• Tests as above

Open questions for you

1. OK with the "no key = no auth" default? Preserves your single-user dev experience. Alternative: refuse to start without a key (breaking change, but stronger default).
2. OK with X-API-Key header vs Authorization: Bearer? Header is simpler; Bearer is more conventional.
3. Should /docs be gated too in non-DEBUG mode, or left open? (Filed as a separate issue.)

Happy to do the PR end-to-end once you green-light the approach. Will run task before-push + task pr-qa-gate before pushing. Plan to branch security/issue-NNN-api-key-auth to match your <type>/issue-<N>-<slug> convention.

• Jeremy Longshore
  intentsolutions.io

[RichardHightower]

Canary PR #182 (closes #180) merged at 10c2681. The path-containment fix is in main; AGENT_BRAIN_ALLOW_EXTERNAL_PATHS is the operator-controlled trust posture going forward.

This one is unblocked whenever you and I get a chance to settle the design direction. @jeremylongshore

[RichardHightower]

Thanks for the deep dive, Jeremy — and the threat model is exactly right. "127.0.0.1 is not authorization" is the core insight; the multi-agent same-box case is the actual deployment pattern, not the edge case.

Green-lighting with three adjustments to your proposal:

Q1 — Secure-by-default, not opt-in. Instead of "no key = no auth," auto-generate the key on agent-brain init (32 bytes urlsafe → .agent-brain/runtime.json mode 600, as you proposed in step 4) and have the server refuse to start without one. CLI handles the key transparently, so the single-user dev UX stays frictionless. For users who explicitly want unauthenticated localhost dev, require an explicit --insecure flag with a loud warning at startup. This mirrors how Postgres requires --allow-empty-password and Redis requires protected-mode no to be unauthenticated — opt-out, not opt-in.

Existing installs upgrade smoothly because agent-brain init is idempotent; add a one-time migration on server start: if .agent-brain/runtime.json exists without an api_key field, generate one and rewrite the file (mode 600).

Q2 — `Authorization: Bearer ` over `X-API-Key`. Forward-compat with the MCP v4 OAuth 2.1 work tracked in #188 — one auth scheme that evolves from static-key to OAuth-issued bearer, rather than two coexisting header schemes. Standard tooling (curl `--bearer`, HTTP clients) just works.

Q3 — Gate `/docs`, `/redoc`, `/openapi.json` when `DEBUG=False`, open when `DEBUG=True`. FastAPI one-liner via `docs_url=None if not settings.DEBUG`. Local dev keeps the browse-to-docs loop; production / shared hosts don't leak the API surface.

Everything else in your proposal lands as written:

• `API_KEY: SecretStr | None` in `config/settings.py` (now functionally required unless `--insecure`)
• `app/security.py` with `verify_api_key` dependency
• Router-level `Depends(verify_api_key)` on every router except health
• CLI sends `Authorization: Bearer` automatically from `runtime.json`
• `.env.example` + README docs

Branch convention `security/issue-179-bearer-auth` is fine. Reminder to run `task before-push` and `task pr-qa-gate` before the PR (you already said you would — just flagging that they fail on <50% coverage and we'd rather catch that locally than in CI).

Looking forward to the PR. Thanks for offering to do this end-to-end.

[jeremylongshore]

Implemented and up as #193 — all three adjustments from your green-light:

• Q1 — secure by default: key auto-generated on agent-brain init into runtime.json (mode 600), server refuses to start without one, --insecure opt-out with a loud warning, one-time backfill on first start for existing installs. CLI sends it transparently.
• Q2 — Authorization: Bearer (forward-compat with the MCP v4: OAuth 2.1 for remote Agent Brain instances #188 MCP v4 OAuth work).
• Q3 — /docs /redoc /openapi.json gated unless DEBUG=true.

verify_bearer_token does constant-time comparison, 401s on missing/bad credentials, and fails closed (503) if misconfigured. task server:pr-qa-gate is green locally (1219 passed, 79.26% coverage). Happy to adjust anything on review.

• Jeremy Longshore
  intentsolutions.io