Skip to content

feat: add adaptive semantic model routing through Maxim Bifrost - #146

Draft
dporkka wants to merge 30 commits into
mainfrom
feature/adaptive-model-routing
Draft

dporkka wants to merge 30 commits into
mainfrom
feature/adaptive-model-routing

Conversation

@dporkka

@dporkka dporkka commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • make Bifrost the canonical gateway when configured while preserving the legacy local selector if Bifrost is unavailable
  • introduce stable semantic routes (auto, cheap/fast, coding, long-horizon, reasoning, security)
  • forward allowlisted Dev Plane task/difficulty routing context to Bifrost
  • record the concrete model resolved by an OpenAI-compatible gateway instead of the requested semantic alias
  • add a pinned MaximHQ Bifrost v2.2.5 deployment and OpenRouter-backed routing policy
  • use Bifrost's native semantic complexity classifier only for ambiguous route/auto traffic; deterministic task/difficulty signals take precedence
  • document OpenJev as a shadow experiment rather than a production selector

Initial policy

  • GLM-5.3-Flash: cheap/simple/default traffic
  • DeepSeek V4.1 Flash: hard/complex coding and long-horizon agent traffic
  • GLM-5.3: hard architecture/review/security/high-reasoning traffic

Model/provider mappings live in gateway configuration so they can change without application releases.

Tests

Added regression tests first for:

  • semantic route -> Bifrost alias behavior
  • route/auto when Bifrost is available
  • legacy fallback when Bifrost is unavailable
  • routing-header allowlisting
  • capturing the gateway-resolved model
  • Maxim Bifrost integration route expectations

Verification status

CI triggered, but the Lint/Test/Build jobs all terminated before executing any steps (runner_id=0). This is an Actions admission/infrastructure failure, not a compiler or test result. The PR remains draft until a real CI run validates compilation/tests and the sample Bifrost config is exercised against a real gateway.

Known blocker: cost authority

Dev Plane's run budget currently accumulates CallResult.Cost. With dynamic Bifrost routing, the resolved model may not exist in Dev Plane's static price table, producing a zero local estimate. Bifrost computes authoritative request cost internally but does not currently return it on the normal inference response.

Do not fix this by copying volatile provider prices back into Dev Plane. Before merge, choose one of these boundaries:

  1. Recommended: Bifrost virtual-key/budget policy is authoritative for provider spend; Dev Plane records usage/outcomes and reconciles cost telemetry asynchronously.
  2. If strict per-task dollar cutoffs must remain synchronous in Dev Plane, add an authoritative cost-return/reconciliation path before enabling dynamic routing for budgeted runs.

Until that boundary is resolved, cost-capped production traffic should not use the dynamic routes in this PR.

Follow-up

The next routing iteration should add task-level routing decision/outcome telemetry. Do not train from the current model_usage.success field: it represents successful provider calls, not successful software tasks. The adaptive selector should learn from verifier/build/test/security outcomes and optimize cost per successful task.

dporkka added 30 commits October 4, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant