Last updated: March 6, 2026 Purpose: Inform default model assignments, roster recommendations, and smart-mode routing across all Hydra positions.
- Provider Profiles
- Head-to-Head Benchmark Tables
- Hydra Position Mapping
- Recommended Defaults
- Model Selection Guide
- Pricing Reference
| Attribute | Value |
|---|---|
| Model ID | claude-opus-4-6 |
| Released | February 5, 2026 |
| Context | 200K (1M beta) |
| Max Output | 128K tokens |
| Pricing | $5 / $25 per 1M (in/out) |
| Speed | ~66 tok/s (API), ~79 tok/s (Bedrock) |
| TTFT | ~1.52s |
| Thinking | Adaptive (effort: low/medium/high/max) |
Key Benchmarks:
| Benchmark | Score | Rank |
|---|---|---|
| SWE-bench Verified | 80.8% | #1 (tied) |
| HumanEval | 97.6% | #1 |
| Terminal-Bench 2.0 | 65.4% | #2 |
| GPQA Diamond | 91.3% | #2 |
| ARC-AGI-2 | 68.8% | #1 (1.8x runner-up) |
| BrowseComp | 84.0% | #1 |
| OSWorld | 72.7% | #1 |
| TAU2-bench Retail | 91.9% | #1 |
| TAU2-bench Telecom | 98.2% (4.5) | #1 |
| Humanity's Last Exam | 40.0% | #1 |
| LiveCodeBench | 76% | -- |
| MMLU | 91% | -- |
| AIME 2025 | 100% | Tied #1 |
| Long-Context Retrieval | 76% | #1 (vs 18.5% predecessor) |
| Chatbot Arena Coding | ~1,510 ELO | #1 (4.5 data) |
Strengths:
- Best abstract reasoning (ARC-AGI-2 nearly 2x competition)
- Best agentic tool use (TAU2-bench, BrowseComp, OSWorld)
- Best code generation quality (SWE-bench, HumanEval, Arena Coding)
- Largest output capacity (128K) for whole-file generation
- 1M context beta for whole-codebase analysis
- Adaptive thinking auto-scales reasoning to complexity
- Agent Teams for multi-agent orchestration (experimental)
- Best long-context retrieval (76% vs 18.5% predecessor)
Weaknesses:
- Moderate latency (slowest flagship)
- Expensive ($5/$25 per 1M)
- Cascading calculation errors on multi-step math
- Spatial reasoning gaps
- Multi-constraint instruction following can degrade under overload
- Prefilling removed (must use structured outputs)
Best For: Architecture, planning, code review, complex reasoning, agentic tasks, long-context analysis.
| Attribute | Value |
|---|---|
| Model ID | claude-sonnet-4-6 |
| Released | February 2026 |
| Context | 200K (1M beta) |
| Max Output | 64K tokens |
| Pricing | $3 / $15 per 1M (in/out) |
| Speed | ~82 tok/s |
| TTFT | ~1.50s |
| Thinking | Adaptive (same generation as Opus 4.6) |
Key Benchmarks (estimates):
| Benchmark | Score |
|---|---|
| SWE-bench Verified | ~79.2% |
| GPQA Diamond | ~87.4% |
| LiveCodeBench | ~72% |
Strengths:
- Same generation as Opus 4.6 with adaptive thinking
- Strong coding at 40% lower cost than Opus
- Good balance of speed and quality for agentic tasks
- 1M context (beta)
Weaknesses:
- Lower max output than Opus (64K vs 128K)
- Not for highest-complexity abstract reasoning (use Opus 4.6)
Best For: General agentic coding, balanced workloads, architect/implementer role, cost-conscious pipelines.
| Attribute | Value |
|---|---|
| Model ID | claude-sonnet-4-5-20250929 |
| Released | September 29, 2025 |
| Context | 200K (1M beta) |
| Max Output | 64K tokens |
| Pricing | $3 / $15 per 1M (in/out) |
| Speed | ~63-72 tok/s |
| TTFT | ~1.80s |
| Thinking | Extended (budget_tokens) |
Key Benchmarks:
| Benchmark | Score |
|---|---|
| SWE-bench Verified | 77.2% (82% w/ parallel compute) |
| HumanEval | 94% |
| GPQA Diamond | 83.4% |
| Terminal-Bench | 50-51% |
| OSWorld | 61.4% |
| LiveCodeBench | 68% |
| Aider Polyglot | 78.8% |
| MMLU | 89% |
Strengths:
- Best price/performance in Anthropic lineup
- Strong coding at 40% lower cost than Opus
- 1M context (beta)
- Good balance of speed and quality
- Extended thinking for correctness-critical tasks
Weaknesses:
- Extended thinking adds ~157s mean latency
- No adaptive thinking (Opus 4.6 only)
- Physics/simulation reasoning gaps
- Thinking tokens billed at output rate
Best For: General coding, balanced workloads, cost-conscious complex tasks, concierge fallback.
| Attribute | Value |
|---|---|
| Model ID | claude-haiku-4-5-20251001 |
| Released | October 15, 2025 |
| Context | 200K |
| Max Output | 64K tokens |
| Pricing | $1 / $5 per 1M (in/out) |
| Speed | ~108-200+ tok/s |
| TTFT | ~0.50s |
| Thinking | Extended (budget_tokens) |
Key Benchmarks:
| Benchmark | Score |
|---|---|
| SWE-bench Verified | 73.3% |
| Terminal-Bench | 41.0% |
| OSWorld | 50.7% |
Strengths:
- Blazing fast (lowest latency of any Claude)
- Near-Sonnet-4 coding at 1/3 the cost
- SWE-bench 73.3% exceeds earlier Sonnet 4
- Extended thinking available
- Ideal for high-volume and multi-agent pipelines
Weaknesses:
- No 1M context (200K only)
- No adaptive thinking
- Weaker on complex reasoning benchmarks
- Not for hardest tasks
Best For: Economy tier, high-volume processing, worker agents, real-time chat, budget coding.
| Attribute | Value |
|---|---|
| Model ID | gpt-5.2-codex |
| Released | February 5, 2026 |
| Context | 400K (272K in + 128K out) |
| Max Output | 128K tokens |
| Pricing | ~$1.75 / ~$14 per 1M (est.) |
| Speed | ~339 tok/s (high effort) |
| Reasoning | low, medium, high, xhigh |
Key Benchmarks:
| Benchmark | Score | Rank |
|---|---|---|
| Terminal-Bench 2.0 | 77.3% | #1 |
| SWE-bench Pro (Public) | 56.8% | #1 |
| SWE-Lancer IC Diamond | 81.4% | #1 |
| OSWorld-Verified | 64.7% | #2 |
| GDPval | 70.9% | -- |
Strengths:
- Fastest flagship model (339 tok/s, 25% faster than predecessor)
- Best terminal/agentic coding (Terminal-Bench 77.3% SOTA)
- Achieves results with fewer tokens than any prior model
- 128K output capacity
- Codex CLI with OS-enforced sandbox
- JSONL output for CI/automation
- Self-developing (first model instrumental in creating itself)
Weaknesses:
- API access still rolling out
- SWE-bench Pro improvement over 5.2-Codex is marginal
- Expensive output tokens
- Responses API only (no Chat Completions)
- No standard SWE-bench Verified score reported
Best For: Implementation, refactoring, agentic terminal tasks, file editing, headless workers.
| Attribute | Value |
|---|---|
| Model ID | gpt-5.4 |
| Released | March 5, 2026 |
| Context | 1,050,000 tokens (1.05M) |
| Max Output | 128K tokens |
| Pricing | $2.50 / $15 per 1M (in/out) |
| Speed | ~78 tok/s (xhigh reasoning) |
| Reasoning | none (default), low, medium, high, xhigh |
Key Benchmarks:
| Benchmark | Score | Rank |
|---|---|---|
| SWE-bench Pro | 57.7% | #1 |
| GPQA Diamond | 84.2% | -- |
| AIME 2025 | 100% | Tied #1 |
| MMLU | 91% | -- |
Strengths:
- Largest context of any OpenAI model (1.05M tokens)
- Best SWE-bench Pro score (57.7%)
- Full reasoning control with 5 effort levels including
nonefor speed - General-purpose flagship (not CLI-specific like 5.3-Codex)
- Strong computer use and code generation capabilities
- 128K output capacity
Weaknesses:
- Slower than GPT-5.3-Codex (78 tok/s vs 424 tok/s)
- More expensive than 5.3-Codex ($2.50/$15 vs $1.75/$14)
- Reasoning default is
none(must set explicitly for deep reasoning)
Best For: Implementation, refactoring, long-context analysis, computer use, agentic tasks.
| Attribute | Value |
|---|---|
| Model ID | gpt-5.2 |
| Released | December 11, 2025 |
| Context | 400K (272K in + 128K out) |
| Pricing | $1.75 / $14 per 1M (in/out) |
| Speed | ~50 tok/s |
| Reasoning | none, low, medium, high, xhigh |
Key Benchmarks:
| Benchmark | Score | Rank |
|---|---|---|
| SWE-bench Verified (Thinking) | 80.0% | #2 |
| GPQA Diamond (Pro) | 93.2% | #1 |
| AIME 2025 (no tools) | 100% | Tied #1 |
| FrontierMath T1-3 | 40.3% | #1 (SOTA) |
| ARC-AGI-2 (Pro) | 54.2% | #2 |
| MMMU Pro | 86.5% | #1 |
| MRCR 4-needle (256K) | ~100% | #1 |
Strengths:
- Best science/math reasoning (GPQA, FrontierMath, AIME)
- Excellent long-context (near-perfect 256K needle retrieval)
xhighreasoning effort for maximum depth- Concise reasoning summaries
- Context compaction for long-running agents
Weaknesses:
- 40% more expensive output than GPT-5/5.1
- Instant mode shows regressions in some categories
- SimpleBench ranked only 9th
- Still hallucinates on legal/medical content
Best For: Investigator, deep analysis, math/science reasoning, failure diagnosis.
| Attribute | Value |
|---|---|
| Model ID | gpt-5 |
| Released | August 7, 2025 |
| Context | 400K (272K in + 128K out) |
| Pricing | $1.25 / $10 per 1M (in/out) |
| Speed | ~50+ tok/s |
| Reasoning | minimal, low, medium (default), high |
Key Benchmarks:
| Benchmark | Score |
|---|---|
| SWE-bench Verified | 74.9% |
| Aider Polyglot | 88% |
| GPQA (no tools) | 88.4% (Pro) |
| AIME 2025 (no tools) | 94.6% |
| TAU2-bench Telecom | 96.7% |
Strengths:
- Unified reasoning model (replaces o-series + GPT-series split)
- Strong coding + tool calling + math in one model
minimaleffort provides o3-level speed at better quality- Front-end coding beats o3 ~70% of the time
- Good streaming speed for chat UX
Weaknesses:
- Superseded by 5.2 for deep reasoning
- Higher latency than GPT-4.1 for simple tasks
- Some tasks don't benefit from reasoning overhead
Best For: Concierge, general chat, broad-spectrum tasks, streaming.
| Attribute | Value |
|---|---|
| Model ID | o4-mini |
| Released | April 16, 2025 |
| Context | 200K (input) / 100K (output) |
| Pricing | $1.10 / $4.40 per 1M (in/out) |
| Speed | High throughput |
| Reasoning | low, medium, high |
Key Benchmarks:
| Benchmark | Score |
|---|---|
| SWE-bench Verified | 68.1% |
| AIME 2025 (no tools) | 92.7% |
| AIME 2025 (w/ tools) | 99.5% |
| Codeforces Elo | 2,719 |
| GPQA Diamond | 81.4% |
Strengths:
- Best math reasoning per dollar
- Outperforms o3 on AIME at o3-mini pricing
- Near-o3 coding at much lower cost
- High throughput for volume workloads
- Supports caching and batching
Weaknesses:
- Smaller context than GPT-5 family (200K vs 400K)
- Reasoning tokens hidden and billed as output
- Superseded by GPT-5 mini for many use cases
Best For: Budget reasoning, nightly handoff, math-heavy tasks, competitive programming.
| Attribute | Value |
|---|---|
| Model ID | gpt-4.1-mini |
| Released | April 14, 2025 |
| Context | 1,000,000 tokens (1M) |
| Pricing | $0.40 / $1.60 per 1M (in/out) |
| Speed | ~50% lower latency than GPT-4o |
Strengths:
- 1M context at rock-bottom pricing
- Beats GPT-4o on many benchmarks at 83% lower cost
- No reasoning token overhead (predictable costs)
Weaknesses:
- No reasoning mode
- Less capable than GPT-5 nano for reasoning tasks
Best For: Quick summaries, simple refactors, high-volume chat, long-document processing.
| Model | Input | Output | Context | Speed |
|---|---|---|---|---|
| GPT-5 mini | $0.25/1M | $2.00/1M | 400K | Fast |
| GPT-5 nano | $0.05/1M | $0.40/1M | 400K | Fastest |
- GPT-5 mini: Good reasoning at 5x cheaper than GPT-5. GPQA 82.8%.
- GPT-5 nano: Ultra-cheap. For classification, routing, simple extraction.
| Attribute | Value |
|---|---|
| Model ID | gemini-3-pro-preview |
| Released | November 2025 |
| Context | 1M input / 65K output |
| Pricing | $2 / $12 per 1M (in/out) |
| Speed | ~128-131 tok/s |
| Thinking | Always-on (thinking_level: LOW/HIGH) |
Key Benchmarks:
| Benchmark | Score | Rank |
|---|---|---|
| SWE-bench Verified | 76.2% | -- |
| LiveCodeBench v6 | 91.7% | #1 |
| LiveCodeBench Elo | 2,439 | #1 (Grandmaster) |
| GPQA Diamond | 91.9% (93.8% DT) | #1 (Deep Think) |
| AIME 2025 | 95% (100% w/ tools) | -- |
| MMLU-Pro | 90.1% | #1 |
| Aider Polyglot | 82.2% | -- |
| Terminal-Bench 2.0 | 54.2% | -- |
| Chatbot Arena Overall | 1,492 ELO | #1 |
Strengths:
- Best competitive/algorithmic coding (Grandmaster Codeforces, LiveCodeBench #1)
- Best overall Arena ELO (1,492)
- Best science reasoning with Deep Think (GPQA 93.8%)
- 1M native context (production, not beta)
- Strong multimodal (video, image, audio, PDF)
- Deep Think mode for significant reasoning uplift
Weaknesses:
- Hallucination: 88% rate in some evaluations
- Inconsistent code quality (sometimes bloated)
- File editing issues ("old_string not found" errors in agent workflows)
- Slower than Claude Sonnet for coding tasks (10-15s responses)
- Instruction following can overreach (sweeping edits vs surgical)
- Preview instability
Best For: Analysis, code review, algorithmic problems, long-context review, competitive programming.
| Attribute | Value |
|---|---|
| Model ID | gemini-3-flash-preview |
| Released | December 2025 |
| Context | 1M input / 65K output |
| Pricing | $0.50 / $3 per 1M (cheapest frontier) |
| Speed | ~218 tok/s |
| Thinking | Always-on (thinking_level: MINIMAL/LOW/MEDIUM/HIGH) |
Key Benchmarks:
| Benchmark | Score | Rank |
|---|---|---|
| SWE-bench Verified | 78% | Beats Gemini 3 Pro! |
| Aider Polyglot | 95.2% | Exceptional |
| GPQA Diamond | 90.4% | Near-flagship |
| MMMU Pro | 81.2% | -- |
Strengths:
- Best value in AI: 78% SWE-bench at $0.50/$3 pricing
- Actually beats Gemini 3 Pro on SWE-bench (78% vs 76.2%)
- Exceptional code editing (95.2% Aider Polyglot)
- Near-Pro reasoning at 75% lower cost
- 4-level thinking granularity (MINIMAL/LOW/MEDIUM/HIGH)
- 1M context window
- 3x faster than 2.5 Pro
Weaknesses:
- 91% hallucination rate in some evaluations (highest of all)
- Shallow reasoning on complex tasks
- Instruction following inferior to Pro
- Overconfidence on uncertain claims
- Long-context reliability lower than Pro (22.1% vs 26.3% at 1M)
- Preview instability
Best For: Economy tier, high-volume coding, fast triage, iterative development loops, agent workflows.
| Model | Input/1M | Output/1M | Context | Speed | Status |
|---|---|---|---|---|---|
| 2.5 Pro | $1.25 | $10 | 1M | ~150 tok/s | Stable GA |
| 2.5 Flash | $0.30 | $2.50 | 1M | ~237 tok/s | Stable GA |
- 2.5 Pro: SWE-bench 63.8%, GPQA 84%, AIME 2024 92%. Stable but outperformed by 3.x.
- 2.5 Flash: Fastest stable model with reasoning. Best for budget/high-volume workloads requiring GA stability.
- Both use
thinkingBudget(NOTthinking_level). Different API from 3.x models.
| Benchmark | Opus 4.6 | Sonnet 4.6 | GPT-5.3-Codex | GPT-5.2 | Gemini 3 Pro | Gemini 3 Flash |
|---|---|---|---|---|---|---|
| SWE-bench Verified | 80.8% | ~79.2% | -- | 80.0% | 76.2% | 78.0% |
| SWE-bench Pro | -- | -- | 56.8% | 55.6% | -- | -- |
| Terminal-Bench 2.0 | 65.4% | -- | 77.3% | -- | 54.2% | -- |
| HumanEval | 97.6% | -- | -- | -- | -- | -- |
| LiveCodeBench Elo | -- | -- | -- | -- | 2,439 | -- |
| LiveCodeBench v6 | -- | ~72% | -- | -- | 91.7% | -- |
| Aider Polyglot | 89.4% (4.5) | -- | -- | -- | 82.2% | 95.2% |
| Chatbot Arena Coding | ~1,510 | -- | -- | ~1,480 | ~1,470 | -- |
| Benchmark | Opus 4.6 | GPT-5.2 | Gemini 3 Pro |
|---|---|---|---|
| GPQA Diamond | 91.3% | 93.2% (Pro) | 91.9% (93.8% DT) |
| AIME 2025 (no tools) | 100% | 100% | 95% |
| ARC-AGI-2 | 68.8% | 54.2% (Pro) | 45.1% (DT) |
| FrontierMath T1-3 | -- | 40.3% | -- |
| Humanity's Last Exam | 40.0% | -- | 37.5% |
| MMLU | 91% | -- | 91.8% |
| MMLU-Pro | -- | -- | 90.1% |
| MMMU Pro | 73.9% | 86.5% | 81.2% |
| Benchmark | Opus 4.6 | GPT-5.3-Codex | GPT-5 |
|---|---|---|---|
| TAU2-bench Retail | 91.9% | -- | -- |
| TAU2-bench Telecom | 98.2% (4.5) | -- | 96.7% |
| OSWorld | 72.7% | 64.7% | -- |
| BrowseComp | 84.0% | -- | -- |
| SWE-Lancer IC Diamond | -- | 81.4% | -- |
| Finance Agent | 60.7% | -- | -- |
| Benchmark | Haiku 4.5 | o4-mini | Gemini 3 Flash |
|---|---|---|---|
| SWE-bench Verified | 73.3% | 68.1% | 78.0% |
| GPQA Diamond | -- | 81.4% | 90.4% |
| AIME 2025 | -- | 99.5% | -- |
| Codeforces Elo | -- | 2,719 | -- |
| Price (in/out per 1M) | $1/$5 | $1.10/$4.40 | $0.50/$3 |
| Position | Current Default | Key Requirements | Primary Capability |
|---|---|---|---|
| Architect | Claude Opus 4.6 | Planning, decomposition, system design | Abstract reasoning, code quality |
| Analyst | Gemini 3 Pro | Review, critique, risk identification | Analytical depth, long context |
| Implementer | GPT-5.4 | Code generation, refactoring, execution | Long context, reasoning, code generation |
| Concierge | GPT-5 | Fast streaming chat, broad knowledge | Low latency, good enough quality |
| Investigator | GPT-5.2 | Failure diagnosis, root-cause analysis | Deep reasoning, code understanding |
| Nightly Handoff | o4-mini | Budget batch work, simple tasks | Cost efficiency, decent quality |
| Phase | Current Agent | Model Selection Rationale |
|---|---|---|
| Propose | Claude (Opus 4.6) | Best at architectural thinking, ARC-AGI-2 #1 |
| Critique | Gemini (3 Pro) | Strong analysis, different perspective, GPQA near-#1 |
| Refine | Claude (Opus 4.6) | Synthesizes feedback into coherent plan |
| Implement | Codex (GPT-5.4) | SWE-bench Pro #1, 1.05M context |
| Phase | Current Agent | Model Selection Rationale |
|---|---|---|
| ANALYZE | Gemini | Broad analysis, codebase scanning |
| DESIGN | Claude | Architectural design, spec creation |
| CRITIQUE | Gemini | Independent review, gap identification |
| REFINE | Claude | Integration of feedback, finalization |
| TEST | (any) | Validation |
| Phase | Typical Agent | Model Selection Rationale |
|---|---|---|
| RESEARCH | Gemini | Codebase scanning, broad analysis |
| DELIBERATE | Council (all) | Multi-perspective evaluation |
| PLAN | Claude | Detailed implementation planning |
| IMPLEMENT | Codex | Fast code generation |
| TEST | Codex | Test execution, verification |
| ANALYZE | Gemini | Quality assessment, scoring |
| Tier | Prompt Complexity | Optimization Target |
|---|---|---|
| Economy | Simple (score < 0.3) | Minimize cost, maximize speed |
| Balanced | Medium (0.3-0.6) | Balance quality/cost/speed |
| Performance | Complex (>= 0.6) | Maximize quality |
| Preset | Model | Rationale |
|---|---|---|
| default | claude-sonnet-4-6 |
Same generation as Opus 4.6, adaptive thinking, 40% cheaper |
| fast | claude-sonnet-4-5-20250929 |
Proven, SWE-bench 77.2%, extended thinking |
| cheap | claude-haiku-4-5-20251001 |
5x cheaper, SWE-bench 73.3%, blazing fast |
| performance | claude-opus-4-6 |
Best reasoning, ARC-AGI-2 #1 — select via model picker |
| Preset | Model | Rationale |
|---|---|---|
| default | gpt-5.4 |
SWE-bench Pro 57.7%, 1.05M context, 5-level reasoning |
| fast | o4-mini |
$1.10/$4.40, SWE-bench 68.1%, great math |
| cheap | o4-mini |
Same model, best budget reasoning |
| alt-default | gpt-5.2 |
When deep reasoning > speed (GPQA 93.2%) |
| Preset | Model | Rationale |
|---|---|---|
| default | gemini-3-pro-preview |
Grandmaster coding, GPQA 91.9%, Arena #1 |
| fast | gemini-3-flash-preview |
SWE-bench 78% at $0.50/$3 (!) |
| cheap | gemini-3-flash-preview |
Same model — already cheapest frontier |
| alt-cheap | gemini-2.5-flash |
GA stable, $0.30/$2.50, if preview instability is a concern |
| Role | Agent | Model | Reasoning | Rationale |
|---|---|---|---|---|
| architect | claude | claude-sonnet-4-6 (default) |
adaptive | Strong agentic coding at 40% lower cost; Opus available for highest-complexity |
| analyst | gemini | gemini-3-pro-preview |
thinking_level: HIGH | Best Arena ELO, GPQA 91.9%, 1M context |
| implementer | codex | gpt-5.4 |
none | SWE-bench Pro 57.7%, 1.05M context |
| concierge | codex | gpt-5 |
low | Fast streaming, broad knowledge, $1.25/$10 |
| investigator | codex | gpt-5.2 |
xhigh | Best deep reasoning (GPQA 93.2%, FrontierMath 40.3%) |
| nightlyHandoff | codex | o4-mini |
low | Budget-friendly, decent SWE-bench 68.1%, great math |
| Priority | Provider | Model | Rationale |
|---|---|---|---|
| 1 | openai | gpt-5 |
Best streaming UX, broad knowledge |
| 2 | anthropic | claude-sonnet-4-5-20250929 |
Strong fallback, good coding |
| 3 | gemini-3-flash-preview |
Cheapest, fast, good quality |
| Tier | Claude | Codex | Gemini |
|---|---|---|---|
| performance | opus-4-6 | gpt-5.4 | gemini-3-pro-preview |
| balanced | sonnet-4-5 | o4-mini | gemini-3-flash-preview |
| economy | haiku-4-5 | o4-mini | gemini-3-flash-preview |
| Agent | Verifier | Rationale |
|---|---|---|
| claude | gemini | Different provider perspective, strong GPQA |
| gemini | claude | Best code quality check (SWE-bench #1) |
| codex | claude | Architectural validation of implementations |
| Task Type | Best Model | Runner-Up | Budget Option |
|---|---|---|---|
| Planning / Architecture | Opus 4.6 | GPT-5.2 (xhigh) | Sonnet 4.5 |
| Code Generation | Opus 4.6 | GPT-5.3-Codex | Gemini 3 Flash |
| Code Review / Analysis | Opus 4.6 | Gemini 3 Pro | Sonnet 4.5 |
| Refactoring / Editing | GPT-5.3-Codex | Gemini 3 Flash (95.2% Aider) | o4-mini |
| Implementation (bulk) | GPT-5.3-Codex | Sonnet 4.5 | Haiku 4.5 |
| Agentic / Terminal | GPT-5.3-Codex | Opus 4.6 | o4-mini |
| Competitive / Algorithmic | Gemini 3 Pro | o4-mini (2,719 CF) | Gemini 3 Flash |
| Math / Science | GPT-5.2 (xhigh) | Opus 4.6 | o4-mini |
| Long-Context Analysis | Gemini 3 Pro (1M) | Opus 4.6 (1M beta) | Gemini 3 Flash (1M) |
| Quick Triage | Gemini 3 Flash | Haiku 4.5 | GPT-4.1-mini |
| Failure Diagnosis | GPT-5.2 (xhigh) | Opus 4.6 | GPT-5 |
| Chat / Concierge | GPT-5 (minimal) | Sonnet 4.5 | Gemini 3 Flash |
| Documentation | Sonnet 4.5 | Gemini 3 Pro | Haiku 4.5 |
| Security Review | Opus 4.6 | GPT-5.2 | Gemini 3 Pro |
| Testing | GPT-5.3-Codex | Sonnet 4.5 | Haiku 4.5 |
| Priority | Best Choice | Why |
|---|---|---|
| Maximum Quality | Claude Opus 4.6 | Highest SWE-bench, ARC-AGI, agentic benchmarks |
| Maximum Speed | GPT-5.3-Codex (339 tok/s) | Fastest flagship by a wide margin |
| Maximum Value | Gemini 3 Flash ($0.50/$3) | 78% SWE-bench at 1/10 Opus cost |
| Maximum Reasoning | GPT-5.2 (xhigh) | GPQA 93.2%, FrontierMath 40.3% |
| Maximum Context | Gemini 3 Pro/Flash (1M native) | Production-ready 1M tokens |
| Maximum Budget | Haiku 4.5 ($1/$5) or Gemini 3 Flash ($0.50/$3) | Both strong SWE-bench (73%/78%) |
| Model | Input | Output | Effective $/quality |
|---|---|---|---|
| Claude Opus 4.6 | $5.00 | $25.00 | Premium |
| GPT-5.4 | $2.50 | $15.00 | Mid-high |
| GPT-5.3-Codex | ~$1.75 | ~$14.00 | Mid-high |
| GPT-5.2 | $1.75 | $14.00 | Mid-high |
| GPT-5 | $1.25 | $10.00 | Mid |
| Gemini 3 Pro | $2.00 | $12.00 | Mid |
| Model | Input | Output | SWE-bench |
|---|---|---|---|
| Gemini 3 Flash | $0.50 | $3.00 | 78.0% |
| Claude Haiku 4.5 | $1.00 | $5.00 | 73.3% |
| o4-mini | $1.10 | $4.40 | 68.1% |
| GPT-5 mini | $0.25 | $2.00 | -- |
| GPT-4.1-mini | $0.40 | $1.60 | -- |
| GPT-5 nano | $0.05 | $0.40 | -- |
| Feature | Multiplier |
|---|---|
| Claude Batch API | 0.5x |
| Claude Prompt Cache reads | 0.1x input |
| Claude Fast Mode | 6x |
| OpenAI Cached Input | 0.1x input |
| OpenAI Batch API | 0.5x |
| Gemini Batch API | 0.5x |
| Gemini Cache reads | 0.1x input |
| Gemini >200K context | 2x (Gemini 3 Pro) |
| Model | tok/s | TTFT | Notes |
|---|---|---|---|
| GPT-5.4 | ~78 | -- | Flagship general-purpose |
| GPT-5.3-Codex | ~339 | -- | Fastest flagship |
| Gemini 3 Flash | ~218 | <1s | Fastest economy |
| Claude Haiku 4.5 | ~108-200 | ~0.50s | Fastest Claude |
| Gemini 2.5 Flash | ~237 | ~0.30s | Fastest GA model |
| Gemini 3 Pro | ~128-131 | varies | -- |
| Claude Sonnet 4.5 | ~63-72 | ~1.80s | -- |
| Claude Opus 4.6 | ~66-79 | ~1.52s | -- |
| GPT-5 | ~50+ | <200ms | Good TTFT |
| GPT-5.2 | ~50 | -- | Deep reasoning mode |
- Gemini 3.x: Uses
thinking_level(LOW/HIGH for Pro; MINIMAL/LOW/MEDIUM/HIGH for Flash). Always-on thinking. Thought signatures must propagate across turns. - Gemini 2.5.x: Uses
thinkingBudget(0-24576 tokens). Can disable thinking with budget=0. Different API from 3.x. - Claude 4.6: Uses adaptive thinking (
effort: low/medium/high/max).budget_tokensdeprecated. - Claude 4.5/Haiku 4.5: Uses extended thinking with
budget_tokens. No adaptive thinking. - OpenAI GPT-5.x: Uses
reasoning_effort(minimal/low/medium/highfor GPT-5; addsnone/xhighfor GPT-5.2 and GPT-5.4). - OpenAI o-series: Uses
reasoning_effort(low/medium/high). - Codex CLI: GPT-5.3-Codex requires Responses API only (not Chat Completions).
All models hallucinate. Reported rates:
- Gemini 3 Flash: ~91% (highest)
- Gemini 3 Pro: ~88%
- Claude models: Lower rates but still present
- GPT-5.2: Improved but still hallucinates on legal/medical
Human review remains essential for all automated code changes.
| Model | Native Context | Status |
|---|---|---|
| Gemini 3 Pro/Flash | 1M | Production |
| GPT-4.1/4.1-mini/4.1-nano | 1M | Production |
| Claude Opus 4.6 | 200K (1M beta) | Beta |
| Claude Sonnet 4.5 | 200K (1M beta) | Beta |
| GPT-5.4 | 1.05M | Production |
| GPT-5/5.2/5.3-Codex | 400K | Production |
| o3/o4-mini | 200K | Production |
| Claude Haiku 4.5 | 200K | Production |