Universal, zero-runtime-dependency NPM library and CLI that routes every query to the right model across quality, cost, and latency, using exact Bayesian Thompson sampling instead of static rules.
BayesRoute sits between your application and your pool of models. You declare the candidate models; for each query it classifies a domain, samples one value from every model's posteriors, scores the candidates by a multi-objective expected utility, and routes to the best draw. Because the draws come from posteriors, a rarely-tried model that might be excellent still gets its turn in proportion to the probability it is actually best, which is what gives Thompson sampling its bounded regret. After the real outcome is observed, the posteriors update in closed form, so the router adapts to a new model, a price change, or a silent regression on its own.
Core promise: zero required runtime dependencies, single-function setup, ergonomic TypeScript types, ESM + CJS dual distribution, a node-free core that runs in Node, edge runtimes, and the browser, SLSA provenance on every release.
This is infrastructure for the Massive Intelligence (IM) stack: model selection becomes a calibrated probabilistic decision with a credible interval, not a heuristic threshold guessed at design time.
pnpm add @takk/bayesroute
# or: npm install @takk/bayesroute
# or: yarn add @takk/bayesroute
# or: bun add @takk/bayesrouteThe core has no runtime dependencies at all. The @takk/* family packages are optional peers; install only the ones you compose with.
// src/example.ts
import { createBayesRoute } from '@takk/bayesroute';
const router = createBayesRoute({
models: ['gpt-5.5', 'sonnet-4.6', 'haiku-4.5'],
// Weights say how much each objective matters. Quality up, latency and cost down.
weights: { quality: 1, latency: 0.3, cost: 0.5 },
seed: 1, // reproducible routing
});
// 1. Route a query. The router classifies the domain and Thompson-samples the models.
const decision = router.route({ text: 'summarize this contract', domain: 'legal' });
// 2. Call the chosen model yourself and measure the real outcome.
const startedAt = Date.now();
const output = await callModel(decision.model, query);
const latencyMs = Date.now() - startedAt;
// 3. Feed the outcome back. The Beta and Gamma posteriors update in closed form.
router.observeDecision(decision, {
quality: isAcceptable(output), // boolean, from your judge, a regex, a classifier, or a human
latencyMs,
cost: output.usage.totalTokens,
});
// Inspect the calibrated ranking for a domain, best first.
console.log(router.rank('legal'));Every model starts neutral. After a handful of queries per domain, the router concentrates traffic on the model that maximizes your objective, while still exploring enough to catch a newly added or newly improved model on its own.
run collapses route, call, and observe into a single await. You return the result, its quality, and the cost; BayesRoute measures latency, folds the outcome back, and hands you the decision, the outcome, the result, and the updated arm.
const { result, decision } = await router.run({ text: query, domain: 'legal' }, async (model) => {
const out = await callModel(model, query);
return { result: out.text, quality: isAcceptable(out.text), cost: out.usage.totalTokens };
});For the quality signal, compose the built-in heuristic evaluators (lengthEvaluator, regexEvaluator, jsonValidEvaluator, combineAll, combineAny) or bring your own async LLM-as-judge. See examples/ for a runnable savings benchmark and Vercel AI SDK and Mastra integrations.
Each arm is a (model, domain) pair carrying three exact conjugate posteriors:
| Objective | Model | Posterior | Updated by |
|---|---|---|---|
| Quality | Bernoulli (acceptable output) | Beta(alpha, beta) |
each accept adds one to alpha, each reject one to beta |
| Latency | Exponential response time | Gamma(shape, rate) over the rate |
each observation adds one to shape, the latency to rate |
| Cost | Exponential token usage | Gamma(shape, rate) over the rate |
each observation adds one to shape, the cost to rate |
For each query the sampler draws quality ~ Beta, latencyMs ~ 1 / Gamma, cost ~ 1 / Gamma, scores
utility = w_quality * quality - w_latency * (latencyMs / scaleLatencyMs) - w_cost * (cost / scaleCost)
and routes to the arm with the highest sampled utility. The conjugate updates are closed form, so inference adds no latency, and the credible interval reported alongside every score is the true Beta or Gamma quantile, never an approximation. The long-run regret is logarithmic in the number of queries.
On RouterBench, a public dataset of 36,497 real queries with every model's recorded correctness and dollar cost across 11 models, BayesRoute learns per-domain routing online and lands on the cost-quality frontier:
| Strategy | Accuracy | Total cost |
|---|---|---|
| always GPT-4 | 84.3% | $120 |
| always GPT-3.5 | 66.6% | $8.9 |
| random | 55.5% | $30 |
| BayesRoute | 69.6% | $9.7 |
| BayesRoute, accuracy-tuned | 73.1% | $16 |
At the same spend as always-using GPT-3.5, BayesRoute answers more queries acceptably, 69.6% versus 66.6%, a Pareto improvement, and tuned for accuracy it reaches 73.1% at 13% of GPT-4's cost. No routing rules were written; the policy was learned from outcomes. This is a replay against precomputed real outcomes, not a live run of other routers; see benchmarks/ for the methodology, the honest caveats, and how to reproduce.
- Thompson sampling, not rules. Exploration and exploitation are balanced optimally per query, with bounded regret, instead of a static rule table or a greedy always-top-model heuristic.
- Multi-objective. Quality, cost, and latency are weighed simultaneously through one expected-utility function, not a single metric.
- Per-domain priors. The query classifier assigns a domain, and each domain keeps its own posteriors, so a model that is great at code and weak at legal is routed accordingly.
- Safe exploration. A guard caps how far a single decision may stray from the greedy incumbent, so exploration never silently ships a regression to production.
- Provider-agnostic and embeddable. No SaaS, no sidecar, no Python classifier to train. It is a library you
pnpm add, with a node-free core for edge and browser.
The facade wires everything together, and every layer is also a standalone entry point for callers who want only the math.
| Import | Surface |
|---|---|
@takk/bayesroute |
createBayesRoute facade plus re-exports of the whole node-free core |
@takk/bayesroute/beta |
Beta distribution: moments, density, CDF, quantile, credible interval, update, sampling |
@takk/bayesroute/gamma |
Gamma-Exponential model for latency and cost: priors, update, sampling, value intervals |
@takk/bayesroute/arms |
Per (model, domain) posterior bundle: init, observe, calibrated summary |
@takk/bayesroute/classify |
Query classifier: default, static, and keyword classifiers, plus any custom function |
@takk/bayesroute/sampler |
Thompson sampler and the greedy incumbent |
@takk/bayesroute/utility |
Multi-objective expected utility and the default weights and scales |
@takk/bayesroute/outcome |
Outcome constructors and validators, scorer-agnostic |
@takk/bayesroute/policy |
Safe-exploration guard |
@takk/bayesroute/store |
PosteriorStore interface, in-memory store, and JSON snapshot encode and decode |
@takk/bayesroute/audit |
Append-only SHA-256 hash-chain audit log, seal and verify, via the Web Crypto API |
@takk/bayesroute/decay |
Time-decay of evidence for non-stationary models |
@takk/bayesroute/node |
createFileStore, the only entry that touches node:fs, atomic JSON persistence |
@takk/bayesroute/edge |
The node-free core re-exported for edge runtimes and the browser |
BayesRoute also works from the shell against a JSON store file. Every command prints JSON to stdout.
# Route a query across three models.
npx @takk/bayesroute route --models gpt-5.5,sonnet-4.6,haiku-4.5 --domain legal --seed 1
# Fold an observed outcome into a persisted store.
npx @takk/bayesroute observe --store ./router.json \
--model gpt-5.5 --domain legal --quality accepted --latency 820 --cost 1300
# Rank the arms in a domain, best first.
npx @takk/bayesroute rank --store ./router.json --domain legal
# Inspect arms, or validate a persisted store.
npx @takk/bayesroute inspect --store ./router.json
npx @takk/bayesroute verify --store ./router.jsonExit codes follow the sysexits convention: 0 ok, 1 error, 64 usage, 65 data, 66 no input.
The default store lives in memory. For state that survives a restart with no database, use the file store:
import { createBayesRoute } from '@takk/bayesroute';
import { createFileStore } from '@takk/bayesroute/node';
const router = createBayesRoute({
models: ['gpt-5.5', 'haiku-4.5'],
store: createFileStore('./router.json'), // atomic write, reloads on construction
});For higher throughput or sharing across processes, implement the PosteriorStore interface over Redis, SQLite, or Postgres; the file store is the zero-dependency reference. A snapshot() is portable JSON, so state moves cleanly between an edge runtime and a worker.
Every score ships with its credible interval, so a compliance reviewer can see not just the chosen model but how confident the choice is. With auditing on, every routing decision and observation is recorded to a tamper-evident SHA-256 hash chain:
const router = createBayesRoute({ models: ['gpt-5.5', 'haiku-4.5'], audit: true });
router.route({ domain: 'legal' });
router.observe('gpt-5.5', 'legal', { quality: true, latencyMs: 800, cost: 1200 });
const seal = await router.seal(); // { algorithm: 'sha-256', root, count }
const result = await router.verify(seal); // { valid: true }Changing, reordering, inserting, or dropping any entry changes the root, so the seal proves the decision log is intact. The hash chain uses the Web Crypto API, never a Node built-in, so it works in Node, on the edge, and in the browser.
- 500 tests across 20 suites, all passing under Vitest 4.
- Coverage: statements 98.2%, branches 96.0%, functions 99.4%, lines 98.2%.
- Lint clean under Biome 2.
- Typecheck clean under TypeScript 6 in maximum strict mode (
exactOptionalPropertyTypes,useUnknownInCatchVariables,noUncheckedIndexedAccess). publintclean,attwclean across all subpaths for both ESM and CJS.- Bundle budgets enforced with
size-limit(the core is about 5 kB brotli). - A distribution smoke test exercises the compiled ESM and CJS artifacts and spawns the compiled CLI as a single Node process.
- Validated on the CI matrix across Node 20, 22, and 24.
- Published with
--provenance(SLSA attestation by GitHub Actions).
See SPEC.md for the formal specification, public surface, and stability promise.
Why not just use LiteLLM? LiteLLM is excellent but Python and gateway-style, with weighted load balancing rather than adaptive Thompson sampling. BayesRoute is TypeScript-native, embeddable in any Node, Bun, or Deno process, learns online, and runs as a library or a CLI.
Why not RouteLLM, Not Diamond, or Martian? RouteLLM requires training a separate classifier on preference data; Not Diamond and Martian are hosted services. BayesRoute needs no training run and no SaaS hop. It accumulates its own calibrated posteriors from your own traffic, which is the asset that compounds.
Does this work in Cloudflare Workers, Vercel Edge, Bun, or Deno?
Yes. The core and the edge entry are node-free, including the audit seal. Only the node file store touches node:fs.
Where does the routing state live?
In-process memory by default. Use the file store for persistence, or implement PosteriorStore over your own backend.
How does it handle a model that silently degrades?
Turn on decay. Old evidence loses weight over time, so the router re-explores and a quietly regressed model falls out of rotation on its own.
Does the router call the models for me? No. It chooses which model to call and learns from the outcome you report. You stay in control of the actual call, so it composes with any SDK or gateway.
See .github/CONTRIBUTING.md for the contributor guide. Substantive proposals open a GitHub Issue first; trivial fixes can go straight to a PR. All commits require DCO sign-off (git commit -s). Non-trivial contributions are governed by the Contributor License Agreement.
- Issues and feature requests. Open a GitHub issue at
davccavalcante/bayesroute/issues. Include the package version, a minimal reproduction, expected versus actual behavior, and where relevant therouter.rank()output or a store snapshot. - Security disclosures. Do NOT open public issues for vulnerabilities. Follow the responsible-disclosure flow in
SECURITY.md, contactdavcavalcante@proton.mewith the[SECURITY]prefix. - Code of Conduct. This project follows the Contributor Covenant 2.1. Participation in any BayesRoute space implies agreement.
Created by David C Cavalcante, davcavalcante@proton.me (preferred), say@takk.ag (Takk relay), linkedin.com/in/hellodav, x.com/davccavalcante, takk.ag.
BayesRoute is part of a broader portfolio of NPM packages building the probabilistic layer of Massive Intelligence (IM) infrastructure for 2026 to 2030, built at Takk Innovate Studio.
The architectural philosophy behind BayesRoute, decisions under uncertainty made explicit, calibrated, and auditable, echoes the author's research frameworks:
- MAIC (Massive Artificial Intelligence Consciousness), the Universe, the framework: a systemic intelligence framework that coordinates, supervises, and governs large-scale Massive Intelligence (IM) ecosystems, providing global context awareness, alignment, and orchestration across models, agents, and decision layers.
- HIM (Hybrid Entity Intelligence Model), the spirit, the model: a hybrid intelligence layer that integrates Massive Intelligence (IM) with human-defined logic, rules, and strategic intent, structuring decision-making before and after model execution.
- NHE (Noumenal Higher-order Entity), the reincarnated body, the agent: a non-human entity with a defined functional identity and operational agency, operating through coordinated intelligence layers while keeping a non-anthropomorphic identity.
These frameworks are published independently of BayesRoute and are separate works:
- Research papers: The Soul of the Machine, Beyond Consciousness in LLMs, The Cave of Silence.
- PhilPapers profile: David Cortes Cavalcante.
- Hugging Face: TeleologyHI.
- GitHub: davccavalcante, Takk8IS.
Join the journey as the portfolio continues to ship the probabilistic layer of Massive Intelligence (IM) infrastructure. Your support is the cornerstone of this work.
- Sponsor on GitHub: github.com/sponsors/davccavalcante
BayesRoute runs entirely inside your own process and infrastructure. It makes no outbound calls to the author, collects no telemetry, and ships no analytics. The only state it keeps is the posteriors you accumulate, in memory or in the store you configure. See PRIVACY.md for the full data-handling notice, including how the optional file store persists state on disk.
Licensed under the Apache License 2.0. See LICENSE for the full text and NOTICE for attribution. You may use, modify, and distribute the code under the terms of that license, including its patent grant and attribution requirements.