mlarsen-source · mlarsen-source · Jun 4, 2026 · Jun 1, 2026
diff --git a/AGENTS.md b/AGENTS.md
diff --git a/frontend/CLAUDE.md → CLAUDE.md b/frontend/CLAUDE.md → CLAUDE.md
diff --git a/README.md b/README.md
@@ -1,34 +1,38 @@
 # Interview Coach
 
-AI-powered interview coaching platform. Users record a response to an interview question and receive structured feedback on delivery, tone, and answer quality.
+AI-powered interview coaching platform. Users select an interviewer, listen to a spoken question, record their answer, and receive structured feedback on delivery, tone, and answer quality.
 
 ## How it works
 
-1. User records an answer in the browser
-2. Audio → Whisper API → timestamped transcript
-3. Audio + transcript → Audeering wav2vec2 → delivery scores (arousal, dominance, valence)
-4. Transcript + scores + question → LLM → structured feedback
-5. Frontend renders a scorecard
+1. User selects an interviewer voice and presses **Start Interview**
+2. Interviewer introduces themselves via TTS (`POST /speech/tts`)
+3. Interviewer reads the question aloud; recording starts automatically
+4. User answers and presses **I'm Done Answering**
+5. Audio → Groq Whisper → timestamped transcript segments (`POST /speech/transcribe`)
+6. Each segment → local wav2vec2 model → arousal / dominance / valence scores
+7. Scores + transcript displayed per segment on the interview screen
+8. *(Coming)* Transcript + scores + question → LLM → structured feedback (`POST /feedback/generate`)
+9. *(Coming)* Frontend renders full scorecard
 
 ## Prerequisites
 
 Install these once at the OS level before running setup:
 
 - **Python 3.9+**
 - **pnpm** — `npm install -g pnpm`
-- **ffmpeg** — decodes audio files, installed via your OS package manager:
+- **ffmpeg** — required to decode audio files:
 
 | Platform | Command                                          |
 | -------- | ------------------------------------------------ |
 | Windows  | `choco install ffmpeg` (requires admin terminal) |
 | macOS    | `brew install ffmpeg`                            |
 | Linux    | `sudo apt install ffmpeg`                        |
 
-Verify ffmpeg with `ffmpeg -version` after installing. On Windows, restart the terminal first so the PATH updates.
+Verify with `ffmpeg -version` after installing. On Windows, restart the terminal first.
 
 ## Setup
 
-Run these once from the project root after cloning.
+Run once from the project root after cloning.
 
 ### 1. Python environment
 
@@ -39,27 +43,46 @@ source venv/Scripts/activate   # Windows (Git Bash)
 pip install -r backend/requirements.txt
 ```
 
-The prompt shows `(venv)` when the environment is active. You must activate it in every new terminal session — packages stay installed, only activation is per-session.
+`(venv)` appears in the prompt when active. Re-activate in every new terminal — packages stay installed.
 
 ### 2. Environment variables
 
-Create `backend/.env` and add your API keys (required once Whisper and LLM services are wired up):
+```bash
+cp backend/.env.example backend/.env
+```
+
+Open `backend/.env` and fill in your keys:
 
 ```
-OPENAI_API_KEY=your_key_here
-ANTHROPIC_API_KEY=your_key_here
+GROQ_API_KEY=your_key_here       # https://console.groq.com  (speech-to-text + TTS)
+ANTHROPIC_API_KEY=your_key_here  # https://console.anthropic.com  (LLM feedback, coming)
+OPENAI_API_KEY=your_key_here     # Reserved
 ```
 
-This file is gitignored — never commit it.
+`backend/.env` is gitignored — never commit it.
+
+### 3. Accept Groq Orpheus TTS terms ⚠️ Required
+
+The text-to-speech feature uses Groq's Orpheus model. **Each Groq account must accept the model terms once before the TTS endpoint will work.** Without this step the backend starts fine but every call to `POST /speech/tts` returns a 400 error and the interviewer will be silent.
+
+1. Log in to [console.groq.com](https://console.groq.com)
+2. Visit this URL directly:
+   ```
+   https://console.groq.com/playground?model=canopylabs%2Forpheus-v1-english
+   ```
+3. Click **Accept** on the terms banner
+4. Done — acceptance is permanent for your account, no need to repeat it
+
+> If you skip this step and try to run the project, the interviewer will not speak and the frontend will show "Could not load audio — check the backend is running." The backend itself will still start correctly.
 
-### 3. Frontend
+### 4. Install frontend dependencies
 
 ```bash
 cd frontend
 pnpm install
 ```
 
-This also installs git hooks via lefthook. **Pre-commit** auto-formats staged frontend/backend files; **pre-push** runs ESLint, Prettier check, and TypeScript on the frontend (plus Ruff on the backend).
+Installs dependencies and git hooks (lefthook). Pre-commit auto-formats staged files; pre-push runs ESLint, Prettier, and TypeScript checks.
 
 ## Running the project
 
@@ -68,12 +91,13 @@ Open two terminals from the project root.
 **Terminal 1 — Backend:**
 
 ```bash
-source venv/Scripts/activate   # Windows; use venv/bin/activate on macOS/Linux
+source venv/Scripts/activate   # Windows; venv/bin/activate on macOS/Linux
 cd backend
 uvicorn app:app --reload
 ```
 
-Server starts at **http://localhost:8000**. Interactive API docs at **http://localhost:8000/docs**. The emotion model loads on startup; first run downloads ~1 GB of model weights.
+Starts at **http://localhost:8000**. API docs at **http://localhost:8000/docs**.
+First run downloads ~1 GB of emotion model weights — this is normal.
 
 **Terminal 2 — Frontend:**
 
@@ -82,36 +106,60 @@ cd frontend
 pnpm dev
 ```
 
-App opens at **http://localhost:3000**.
+App at **http://localhost:3000**.
 
 ## Repo structure
 
 ```
-backend/    — FastAPI backend (Python)
-  app.py              — entry point; registers all routers
-  requirements.txt    — Python dependencies
+backend/
+  app.py                        — FastAPI entry point; registers routers, loads emotion model
+  requirements.txt              — Python dependencies
   services/
-    tone_delivery_analyzer/ — Audeering wav2vec2 emotion model (local)
-    speech_to_text/         — Whisper API transcription (todo)
-    llm/                    — LLM feedback generation (todo)
-frontend/   — Next.js frontend (React)
-docs/       — project proposal and reference documents
+    tone_delivery_analyzer/     — local wav2vec2 emotion model (arousal/dominance/valence)
+    speech_to_text/             — Groq Whisper transcription + TTS
+    llm/                        — LLM feedback generation (coming)
+    text_analysis/              — transcript scoring (coming)
+
+frontend/
+  app/
+    page.tsx                    — interview UI (home page)
+    InterviewClient.tsx         — full interview state machine
+    layout.tsx / globals.css    — root layout and styles
+    scorecard/                  — scorecard display components
+    dev/
+      flow/                     — pipeline visualization dev page
+      transcribe/               — transcription dev/debug page
+  lib/
+    prompts/
+      questions.ts              — question bank (20 questions + intro)
+      interviewers.ts           — interviewer voices and personas
+    interview-coach/
+      types.ts                  — shared TypeScript types
+      mocks.ts                  — mock data for UI development
+      pipelineStages.ts         — pipeline stage definitions
+
+docs/
+  architecture.md               — system architecture and data flow
+  agent-workflows/              — AI agent workflow guides
 ```
 
-**Complete the Setup section above before consulting any of the links below.**
+## Reference
 
-- Backend endpoint reference: [backend/README.md](backend/README.md)
-- Frontend scripts reference: [frontend/README.md](frontend/README.md)
-- Tone/delivery model details: [backend/services/tone_delivery_analyzer/README.md](backend/services/tone_delivery_analyzer/README.md)
+- Architecture and data flow: [docs/architecture.md](docs/architecture.md)
+- Backend endpoints: [backend/README.md](backend/README.md)
+- Agent conventions: [AGENTS.md](AGENTS.md)
+- Tone/delivery model: [backend/services/tone_delivery_analyzer/README.md](backend/services/tone_delivery_analyzer/README.md)
 
 ## Stack
 
-| Layer           | Technology                                                    |
-| --------------- | ------------------------------------------------------------- |
-| Frontend        | Next.js (React)                                               |
-| Backend         | FastAPI (Python)                                              |
-| Speech-to-text  | OpenAI Whisper API                                            |
-| Tone/delivery   | audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim (local) |
-| Feedback        | Claude or GPT-4o (API)                                        |
-| Frontend deploy | Vercel                                                        |
-| Backend deploy  | Render or Fly.io                                              |
+| Layer            | Technology                                                     |
+| ---------------- | -------------------------------------------------------------- |
+| Frontend         | Next.js 16, React 19, TypeScript, Tailwind CSS 4               |
+| Backend          | FastAPI (Python)                                               |
+| Text-to-speech   | Groq Orpheus (`canopylabs/orpheus-v1-english`)                 |
+| Speech-to-text   | Groq Whisper (`whisper-large-v3-turbo`)                        |
+| Tone/delivery    | `audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim` (local)|
+| LLM feedback     | Anthropic Claude (coming)                                      |
+| Package manager  | pnpm                                                           |
+| Frontend deploy  | Vercel (planned)                                               |
+| Backend deploy   | Render or Fly.io (planned)                                     |
diff --git a/backend/.env.example b/backend/.env.example
@@ -1,8 +1,12 @@
-# Copy this file to .env and fill in your keys.
+# Copy this file to .env and fill in your keys before running the backend.
 # .env is gitignored — never commit it.
+# All three keys are required for the full pipeline to function.
 
-# Required for speech-to-text (Whisper)
-OPENAI_API_KEY=
+# Required for speech-to-text (Groq Whisper API — https://console.groq.com)
+GROQ_API_KEY=
 
-# Required for LLM feedback (Claude)
+# Required for LLM feedback (Claude — https://console.anthropic.com)
 ANTHROPIC_API_KEY=
+
+# Reserved for future use (OpenAI fallback or embeddings)
+OPENAI_API_KEY=
diff --git a/backend/README.md b/backend/README.md
@@ -4,39 +4,99 @@ For setup and running instructions see the [root README](../README.md).
 
 ## Endpoints
 
-| Method | Path | Status | Description |
-|--------|------|--------|-------------|
-| GET | `/health` | done | liveness check |
-| GET | `/emotion/health` | done | confirms emotion model is loaded |
-| POST | `/emotion/analyze` | done | upload audio → arousal / dominance / valence |
-| POST | `/speech/transcribe` | todo | upload audio → transcript + word timestamps |
-| POST | `/feedback/generate` | todo | transcript + scores → structured feedback |
+| Method | Path                 | Status | Description                                              |
+|--------|----------------------|--------|----------------------------------------------------------|
+| GET    | `/health`            | done   | Liveness check                                           |
+| GET    | `/emotion/health`    | done   | Confirms emotion model is loaded                         |
+| POST   | `/speech/tts`        | done   | Text → WAV audio via Groq Orpheus TTS                    |
+| POST   | `/speech/transcribe` | done   | Audio file → per-segment transcript + emotion scores     |
+| POST   | `/emotion/analyze`   | done   | Audio file → single arousal / dominance / valence score  |
+| POST   | `/feedback/generate` | todo   | Transcript + scores + question → structured LLM feedback |
 
 ## Structure
 
 ```
-app.py              — entry point; registers all routers, loads emotion model at startup
-requirements.txt    — Python dependencies
+app.py                          — entry point; registers all routers, loads emotion model at startup
+requirements.txt                — Python dependencies
 services/
-  tone_delivery_analyzer/
-    router.py       — POST /emotion/analyze
-    emotion_model.py — model class definitions (do not modify)
-    run_emotion.py  — standalone CLI for testing the model directly
   speech_to_text/
-    router.py       — POST /speech/transcribe (not yet implemented)
+    router.py                   — POST /speech/tts and POST /speech/transcribe
+  tone_delivery_analyzer/
+    router.py                   — POST /emotion/analyze and GET /emotion/health
+    emotion_model.py            — model class definitions (do not modify)
+    run_emotion.py              — standalone CLI for testing the model directly
   llm/
-    router.py       — POST /feedback/generate (not yet implemented)
+    router.py                   — POST /feedback/generate (not yet implemented)
+  text_analysis/
+    router.py                   — reserved for transcript scoring (not yet implemented)
+```
+
+## POST /speech/tts
+
+Converts text to speech using Groq Orpheus and returns WAV audio.
+
+**Request body (JSON):**
+```json
+{ "text": "Hello, welcome to your interview.", "voice": "hannah" }
 ```
 
-## Test /emotion/analyze
+Available voices: `autumn`, `diana`, `hannah`, `austin`, `daniel`, `troy`
+
+`voice` defaults to `hannah` if omitted or invalid.
+
+**Response:** `audio/wav` binary
 
 ```bash
-curl -X POST http://localhost:8000/emotion/analyze \
-  -F "file=@path/to/audio.mp3"
+curl -X POST http://localhost:8000/speech/tts \
+  -H "Content-Type: application/json" \
+  -d '{"text": "Tell me about yourself.", "voice": "hannah"}' \
+  --output question.wav
+```
+
+Requires `GROQ_API_KEY` in `backend/.env` and Orpheus terms accepted at `https://console.groq.com/playground?model=canopylabs%2Forpheus-v1-english`.
+
+## POST /speech/transcribe
+
+Accepts an audio file. Transcribes via Groq Whisper, slices audio at segment boundaries, and scores each slice with the local emotion model.
+
+**Request:** `multipart/form-data`, field name `file`
+
+**Response:** array of segments
+
+```json
+[
+  { "start": 0.0,  "end": 2.4, "text": "I think the best approach...", "arousal": 0.61, "dominance": 0.55, "valence": 0.32 },
+  { "start": 2.4, "end": 5.1, "text": "would be to first consider...", "arousal": 0.48, "dominance": 0.52, "valence": 0.41 }
+]
+```
+
+```bash
+curl -X POST http://localhost:8000/speech/transcribe \
+  -F "file=@path/to/audio.webm"
 ```
 
-Expected response:
+Requires `GROQ_API_KEY` and the emotion model loaded (backend running).
+
+## POST /emotion/analyze
+
+Scores a full audio file as a single arousal / dominance / valence reading. Used for whole-recording analysis; `/speech/transcribe` is preferred for per-segment scores.
 
+**Request:** `multipart/form-data`, field name `file`
+
+**Response:**
 ```json
-{"arousal": 0.61, "dominance": 0.55, "valence": 0.32}
+{ "arousal": 0.61, "dominance": 0.55, "valence": 0.32 }
+```
+
+```bash
+curl -X POST http://localhost:8000/emotion/analyze \
+  -F "file=@path/to/audio.mp3"
 ```
+
+## Score interpretation
+
+| Dimension  | Low (→ 0)          | High (→ 1)              |
+|------------|--------------------|-------------------------|
+| Arousal    | Calm, flat         | Excited, energetic      |
+| Dominance  | Hesitant, passive  | Assertive, confident    |
+| Valence    | Negative, stressed | Positive, enthusiastic  |
diff --git a/backend/app.py b/backend/app.py
@@ -25,7 +25,7 @@ async def lifespan(app: FastAPI):
 app.add_middleware(
     CORSMiddleware,
     allow_origins=["http://localhost:3000"],
-    allow_methods=["GET", "POST"],
+    allow_methods=["GET", "POST", "OPTIONS"],
     allow_headers=["*"],
 )
 

diff --git a/backend/requirements.txt b/backend/requirements.txt
@@ -16,7 +16,10 @@ transformers>=4.38.0
 librosa>=0.10.0
 numpy>=1.24.0
 
-# Speech-to-text — OpenAI Whisper API (requires OPENAI_API_KEY)
+# Speech-to-text — Groq Whisper API (requires GROQ_API_KEY)
+groq>=0.9.0
+
+# OpenAI SDK — reserved for fallback / embeddings if needed
 openai>=1.0.0
 
 # LLM feedback — Anthropic Claude API (requires ANTHROPIC_API_KEY)