Production-validated dual-channel intent recognition: Embedding Recall + LLM Tree Reasoning.
A Python library that runs two complementary intent classification channels in parallel — fast embedding similarity and LLM hierarchical reasoning — then fuses their results into a single scored verdict. Built and validated in a real SaaS system, not a research prototype.
Evaluated on a production retail management SaaS — 57 intents, 7 domains, 140 labeled routing cases.
Single-model baselines (3 LLMs tested, 104 cases):
| Model | Accuracy | Avg Latency |
|---|---|---|
| Model A (fast) | 80.8% | 2.2s |
| Model B (balanced) | 82.7% | 6.0s |
| Model C (strong) | 86.5% | 15.1s |
No single model broke 87%. Dual-channel fusion on the fastest model (140 cases, extended regression):
| Metric | Best Single Model | Dual-channel Fusion |
|---|---|---|
| Overall accuracy | 86.5% (90/104) | 97.1% (136/140) |
| Analytics domain | 47–87% | 100% |
| Remaining failures | 14–20 | 4 (all context-dependent pronouns) |
The 4 remaining failures all contain pronouns like "that one" / "this" that require multi-turn conversation context — a dialogue management concern, not an intent recognition issue.
→ Full evaluation with cross-model failure analysis
| User says | Pure Embedding | Pure LLM | Fusion |
|---|---|---|---|
| "cancel order" | cancel_order | cancel_order | cancel_order |
| "I changed my mind" | ??? | cancel_order | cancel_order |
| "how did the store do today" | query_revenue (wrong) | daily_report | daily_report |
| "what's flying off the shelves" | query_product (wrong) | hot_selling | hot_selling |
- Embedding is fast and great at keyword matches, but fails on indirect phrasing.
- LLM tree reasoning excels at semantic inference, but is slower and may hallucinate intent names.
- Fusion combines both: embedding widens recall, tree reasoning provides precision.
User Message
│
├──────────────────┬──────────────────┐
│ │ │
┌────▼─────┐ ┌─────▼──────┐ │
│ Embedding │ │ Intent Tree│ parallel
│ Channel │ │ Channel │ execution
│ (5-15ms) │ │ (LLM call) │ │
└────┬──────┘ └─────┬──────┘ │
│ │ │
└────────┬─────────┘──────────────────┘
│
┌──────▼──────┐
│ Merge&Score │ final = α·emb + (1-α)·tree
└──────┬──────┘
│
┌──────▼──────┐
│ 3-state │ CLEAR / AMBIGUOUS / LOW
│ Verdict │
└─────────────┘
pip install intent-fusionfrom intent_fusion import IntentRouter, IntentEntry, FusionVerdict
# Define your intents with separate texts for each channel
entries = [
IntentEntry(
name="query_order_status",
domain="Orders",
embed_text="track order, shipping status, where is my package",
tree_text="user wants to know where their order is or delivery ETA",
),
IntentEntry(
name="cancel_order",
domain="Orders",
embed_text="cancel order, stop my order, don't want it",
tree_text="user wants to cancel an order they placed",
),
# ... more intents
]
# Create a router (loads embedding model once)
router = IntentRouter(entries)
# Any LLM with an async chat() method works
result = await router.fuse("I changed my mind", your_llm)
match result.verdict:
case FusionVerdict.CLEAR:
execute(result.top.name) # "cancel_order"
case FusionVerdict.AMBIGUOUS:
execute(result.top.name)
suggest(result.runner_up.name) # show alternative
case FusionVerdict.LOW:
fallback_to_free_chat()Each intent has independent descriptions optimized for each channel:
IntentEntry(
name="hot_selling",
domain="Reports",
embed_text="best sellers, top selling, popular items, what sells most",
tree_text="user wants to see product sales ranking. Unlike query_revenue (totals), this shows per-product ranking.",
)embed_text: Keyword-dense, synonym-rich. Feed the embedding model what users actually type.tree_text: Descriptive with disambiguation hints. Tell the LLM what this intent is and what it isn't.
Any object with an async chat() method works:
class YourLLM:
async def chat(self, messages: list[dict[str, str]], **kwargs) -> str:
# Call OpenAI, Anthropic, DeepSeek, Ollama, or any provider
...An OpenAI-compatible adapter is included in examples/.
| Verdict | Meaning | Recommended Action |
|---|---|---|
CLEAR |
Top candidate is dominant | Route directly |
AMBIGUOUS |
Top-2 are close | Execute top, suggest runner-up |
LOW |
No confident match | Fall back to free chat / guide |
If the LLM is down, embedding still works (and vice versa):
result = await router.fuse(message, llm)
if result.degraded:
print(f"Running on {result.active_channels} only")Find optimal alpha and delta for your intent set:
python -m intent_fusion.calibration \
--cases your_cases.yaml \
--intents your_intents.yaml
# Output: Best alpha=0.10, delta=0.10 → accuracy=0.971 (101/104)See docs/calibration.md for details.
Two datasets are included:
| Dataset | Intents | Cases | Source |
|---|---|---|---|
ecommerce/ |
10 | 20 | Synthetic |
petstore/ |
15 | 52 | Production (desensitized) |
export OPENAI_API_KEY=sk-...
python benchmarks/petstore/run_benchmark.pySee benchmarks/README.md for methodology and full results.
- Architecture — Dual-channel design, fusion formula, three-state logic
- Pipeline Integration — How to integrate into a production system
- Calibration Guide — Finding optimal parameters for your data
router = IntentRouter(entries, model_name="BAAI/bge-small-en-v1.5")
router.build_index() # optional: pre-warm at startup
result = await router.fuse(message, llm, alpha=0.10, delta=0.10)from intent_fusion import fuse_intent, build_index, invalidate_cache
build_index(entries) # optional pre-warm
result = await fuse_intent(message, entries, llm)
invalidate_cache() # call after updating intentsMIT