Skip to content

fix(pricing): resolve MiniMax-M3 (provider prefix + pre-registration)#1186

Merged
JerrettDavis merged 3 commits into
headroomlabs-ai:mainfrom
shreyassks:fix/minimax-pricing
Jun 30, 2026
Merged

fix(pricing): resolve MiniMax-M3 (provider prefix + pre-registration)#1186
JerrettDavis merged 3 commits into
headroomlabs-ai:mainfrom
shreyassks:fix/minimax-pricing

Conversation

@shreyassks

@shreyassks shreyassks commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes the cost dashboard reporting $0.00 for every call when the upstream model is MiniMax-M3 (Anthropic-compatible endpoint served from the MiniMax provider).

Two root causes in headroom/pricing/litellm_pricing.py:

  1. resolve_litellm_model() had no minimax/ provider prefix. LiteLLM's community pricing database stores MiniMax-M3 only under minimax/MiniMax-M3. The resolver never tried that prefix, so callers in proxy/cost.py, proxy/savings_tracker.py, and perf/analyzer.py silently fell back to the unresolved name.
  2. The prefix check was case-sensitive. MiniMax's model name uses mixed case (MiniMax-M3), but every existing prefix pattern (claude-, gpt-, o1-, …) was lowercase, so even after adding "minimax-" the bare MiniMax-M3 wouldn't match.

This PR fixes both, plus adds a _register_minimax_pricing() helper that pre-populates litellm.model_cost["MiniMax-M3"] from minimax/MiniMax-M3 at module load — a safety net so estimate_cost() (which doesn't know the minimax/ prefix internally) succeeds even on a cold resolver cache or if LiteLLM drops the prefixed entry in a future release.

Net change: +97 / −1 lines across 2 files (one production file + one test file).

Closes #

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Changes Made

  • Production (headroom/pricing/litellm_pricing.py):
    • Add "minimax-": "minimax/" to the provider-prefix table in _resolve_litellm_model_uncached() so the resolver knows about the MiniMax provider.
    • Compute model_lower = model.lower() and match prefixes against it instead of model, so the mixed-case bare name MiniMax-M3 resolves correctly. The existing prefixes (claude-, gpt-, o1-, o3-, o4-, gemini-) are already lowercase patterns matched against canonical lowercase names (claude-sonnet-4-5-…, gpt-4o, gemini-2.0-flash) — no regression.
    • Add _register_minimax_pricing(): if minimax/MiniMax-M3 is in litellm.model_cost and MiniMax-M3 is not, copy the pricing dict under the bare key. No-op on older LiteLLM (entry missing) or when the user has already customised MiniMax-M3.
    • Invoke _register_minimax_pricing() once at module import.
  • Tests (tests/test_pricing_litellm.py):
    • Add test_litellm_minimax_mixed_case_with_provider_prefix — verifies resolve_litellm_model("MiniMax-M3") returns "minimax/MiniMax-M3" via the case-insensitive prefix match.
    • Add test_litellm_minimax_preregistration_safety_net — verifies the pre-registration populates the bare MiniMax-M3 key, that estimate_cost() returns the correct dollar figure (0.84 for 1M in + 100k out), and that a user-customised bare entry is never clobbered.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added for new functionality
  • Manual testing performed

Test Output

$ uv run pytest tests/test_pricing_litellm.py -v
============================= test session starts ==============================
platform darwin -- Python 3.11.15, pytest-9.0.3, pluggy-1.6.0
configfile: pyproject.toml
plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0, cov-7.0.0
collected 7 items

tests/test_pricing_litellm.py::test_litellm_helpers_when_dependency_is_unavailable PASSED [ 14%]
tests/test_pricing_litellm.py::test_litellm_model_pricing_exact_match_and_defaults PASSED [ 28%]
tests/test_pricing_litellm.py::test_litellm_model_pricing_uses_provider_prefixes PASSED [ 42%]
tests/test_pricing_litellm.py::test_litellm_model_pricing_uses_aliases_and_zero_cost_defaults PASSED [ 57%]
tests/test_pricing_litellm.py::test_litellm_model_pricing_returns_none_for_unknown_models PASSED [ 71%]
tests/test_pricing_litellm.py::test_litellm_minimax_mixed_case_with_provider_prefix PASSED [ 85%]
tests/test_pricing_litellm.py::test_litellm_minimax_preregistration_safety_net PASSED [100%]

============================== 7 passed in 1.07s ===============================

$ uv run ruff check headroom/pricing/litellm_pricing.py tests/test_pricing_litellm.py
All checks passed!

$ uv run mypy headroom/pricing/litellm_pricing.py
Success: no issues found in 1 source file

Manual reproducer (matches the PR writeup):

$ uv run python -c "
from headroom.pricing.litellm_pricing import resolve_litellm_model, estimate_cost
import litellm

print('resolve_litellm_model(MiniMax-M3):', resolve_litellm_model('MiniMax-M3'))
print('MiniMax-M3 in litellm.model_cost :', 'MiniMax-M3' in litellm.model_cost)
print('estimate_cost (1M in, 100k out): ', estimate_cost('MiniMax-M3', 1_000_000, 100_000))
"

resolve_litellm_model(MiniMax-M3): minimax/MiniMax-M3
MiniMax-M3 in litellm.model_cost : True
estimate_cost (1M in, 100k out):  0.84

Real Behavior Proof

  • Environment: macOS Darwin 25.5.0, Python 3.11.15, headroom-ai installed editable via uv from this branch, litellm pulled from PyPI on first run.
  • Exact command / steps: after git checkout fix/minimax-pricing && uv sync --all-extras --dev, run (1) uv run python -c "from headroom.pricing.litellm_pricing import resolve_litellm_model, estimate_cost; import litellm; print(resolve_litellm_model('MiniMax-M3'), 'MiniMax-M3' in litellm.model_cost, estimate_cost('MiniMax-M3', 1_000_000, 100_000))", then (2) uv run pytest tests/test_pricing_litellm.py -v, then (3) uv run ruff check headroom/pricing/litellm_pricing.py tests/test_pricing_litellm.py, then (4) uv run mypy headroom/pricing/litellm_pricing.py.
  • Observed result: (1) resolve_litellm_model('MiniMax-M3') returns minimax/MiniMax-M3 (was MiniMax-M3, unresolved); 'MiniMax-M3' in litellm.model_cost is True (proves _register_minimax_pricing() ran); estimate_cost('MiniMax-M3', 1_000_000, 100_000) returns 0.84 (matches $0.60/M in × 1M + $2.40/M out × 0.1M). (2) All 7 tests in tests/test_pricing_litellm.py pass (5 pre-existing + 2 new MiniMax-specific). (3) ruff reports All checks passed!. (4) mypy reports Success: no issues found in 1 source file.
  • Not tested: end-to-end through the running proxy against a live MiniMax-M3 endpoint — no Anthropic-compatible key configured in this environment. The reproducer exercises the exact code path the proxy's cost accumulator uses, but I did not point the proxy at a real upstream.

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation — N/A: no user-facing docs reference litellm_pricing.py directly; the only public API affected (estimate_cost) now returns correct values for a previously-unsupported model.
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the CHANGELOG.md if applicable — N/A: this repo doesn't appear to use CHANGELOG.md (not present at repo root).

Additional Notes

  • Why both fixes are needed. estimate_cost() calls get_model_pricing() directly, and get_model_pricing() has its own hardcoded prefix list ["openai/", "anthropic/", "google/", "mistral/", "deepseek/"] that does not include minimax/. So the prefix resolver alone is not enough for estimate_cost("MiniMax-M3") to return a non-None number — the pre-registration step is what makes the bare name resolve. The prefix resolver change matters for the proxy's cost/savings/perf code paths that call resolve_litellm_model() and then look up the prefixed string themselves.
  • Why case-insensitive matching is safe. All existing prefixes are lowercase patterns matched against already-lowercase canonical model names — lower-casing before startswith() is a no-op for them. Only the new "minimax-" entry uses a mixed-case input.
  • Pricing drift note. _register_minimax_pricing() mirrors upstream LiteLLM (input $0.60/M, output $2.40/M, cache read $0.12/M as of 2026-06). Re-check after LiteLLM updates; the function already short-circuits when the user has customised the entry.
  • Did not run the full test suite, only tests/test_pricing_litellm.py. Wider CI will catch anything I missed.

@github-actions

github-actions Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added the status: needs author action Pull request body or readiness checklist still needs author updates label Jun 20, 2026
The proxy's cost dashboard reports $0.00 for every MiniMax call because:
  1. resolve_litellm_model() has no 'minimax/' provider prefix
  2. The existing prefix check is case-sensitive; MiniMax uses mixed-case
     model names like 'MiniMax-M3'.

Fix both, and pre-register 'MiniMax-M3' in litellm.model_cost as
'minimax/MiniMax-M3' at module load (safety net for cold cache).

Closes: cost dashboard showing $0 savings despite real compression.
@shreyassks shreyassks force-pushed the fix/minimax-pricing branch from 1844f88 to 515963c Compare June 20, 2026 07:18
@github-actions github-actions Bot added status: ready for review Pull request body is complete and the author marked it ready for human review and removed status: needs author action Pull request body or readiness checklist still needs author updates labels Jun 20, 2026
@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 63.63636% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
headroom/pricing/litellm_pricing.py 63.63% 2 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

@github-actions github-actions Bot added status: ci failing Required or reported CI checks are failing and removed status: ready for review Pull request body is complete and the author marked it ready for human review labels Jun 22, 2026

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pricing fix itself is the right direction: the mixed-case MiniMax resolver path is covered, and the bare-name pre-registration gives estimate_cost("MiniMax-M3", ...) a direct lookup path.

The branch cannot merge while lint is red, though. The latest lint job has ruff check . passing, then fails at formatting:

Would reformat: tests/test_pricing_litellm.py

Please run ruff format tests/test_pricing_litellm.py and push the formatted file so CI can go green.

@github-actions github-actions Bot added status: ready for review Pull request body is complete and the author marked it ready for human review status: ci failing Required or reported CI checks are failing and removed status: ci failing Required or reported CI checks are failing status: ready for review Pull request body is complete and the author marked it ready for human review labels Jun 23, 2026

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The formatting blocker is resolved and the MiniMax pricing change looks good. The resolver now matches MiniMax-M3 case-insensitively through the minimax/ prefix, and the bare-name pre-registration makes estimate_cost("MiniMax-M3", ...) work even when callers do not resolve a prefixed LiteLLM key first. The tests cover both the resolver and the pre-registration safety net without clobbering a customized bare entry.

Lint/build/e2e are green. The current red test shards I inspected on similar heads are the shared HuggingFace cache issue in memory tests, not this pricing path.

@github-actions github-actions Bot added status: ready for review Pull request body is complete and the author marked it ready for human review and removed status: ci failing Required or reported CI checks are failing labels Jun 28, 2026
@JerrettDavis JerrettDavis merged commit 46dede3 into headroomlabs-ai:main Jun 30, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants