llm-hello-py

A tiny FastAPI + httpx app that exists for one reason: to prove, end to end, that c2a daari secrets bind actually wires Daari LLM Gateway credentials into a running pod's environment. It's a sibling demo to pg-hello-py, covering the LLM Gateway credential shape instead of Postgres.

What this is

• GET / — friendly HTML landing page, links to /llm-check
• GET /llm-check — POSTs a trivial "reply with pong" prompt to {OPENAI_BASE_URL}/v1/chat/completions using OPENAI_API_KEY as a Bearer token and LLM_MODEL (default llama-3.1-8b-instant) as the model, and returns:
  • success: {"ok": true, "model": "...", "reply": "..."}
  • failure: {"ok": false, "error": "..."} with HTTP 500

The app fails fast at startup (SystemExit(1) with a clear stderr message) if either required env var is missing. That's deliberate — a silently-missing binding is exactly the failure mode this app exists to catch, so we'd rather crash loudly on boot than serve requests against vars that were never set.

Paketo auto-detects the Procfile (web: uvicorn app:app --host 0.0.0.0 --port $PORT) — no server.py/Node concerns, this is a pure Python buildpack app.

Prereqs

• c2a CLI installed and authenticated (c2a login)
• An active project set (c2a project use <name> — check with c2a project show)
• A Daari LLM Gateway endpoint + API key for your project (see the Daari console / c2a llm commands)

Manual walkthrough

1. Confirm your active project:

   c2a project show

2. Create the Daari Secret holding the LLM Gateway credentials:

   c2a daari secrets create llm-hello-creds \
     --from-literal OPENAI_API_KEY=<gateway-api-key> \
     --from-literal OPENAI_BASE_URL=<gateway-base-url> \
     --from-literal LLM_MODEL=llama-3.1-8b-instant

   Note the k8sName printed in the output (in parentheses) — that's what daari secrets bind takes as its second argument, not the display name you typed.

3. Deploy the app from this directory's git remote (or any fork/mirror of it):

   c2a app create llm-hello-py -g <git-url-for-this-repo-or-a-fork>

4. Bind the secret to the app:

   c2a daari secrets bind llm-hello-py <k8sName-from-step-2>

   This writes the llm-hello-py-service-binding Secret's C2A_SYSTEM_ENV JSON (the platform's runtime binding contract — variables / secrets / bindings all serialized together) and additive-patches the ksvc's envFrom to reference it. The platform init step unpacks that JSON into plain env vars before app.py starts, so this app never parses C2A_SYSTEM_ENV itself — it just reads OPENAI_API_KEY etc. from os.environ as shown above.

5. Wait for the new revision to become Ready, then verify:

   c2a app show llm-hello-py -f url
   curl "$(c2a app show llm-hello-py -f url)/llm-check"

   Expect something like {"ok":true,"model":"llama-3.1-8b-instant","reply":"pong"} (the exact reply text depends on the model).

Namespace gotcha

c2a daari secrets create uses the CLI's active project's namespace with no override flag and no warning. If you're not sure which project is active, secrets can land in the wrong namespace and go undetected until a bind silently fails to find them. Always run c2a project show first.

No automated e2e script

Unlike pg-hello-py, this demo has no e2e.sh. Exercising it for real requires a live Daari LLM Gateway deployment and API key; see NOTE.md for details. Use the manual walkthrough above.