OpenRouter embedding models fail due to LiteLLM missing provider route

[jarmen423]

Problem

Agent Zero fails to initialize memory when the embedding model is configured to use OpenRouter. This is not a configuration mistake — it is a LiteLLM limitation.

LiteLLM supports OpenRouter for chat/completions, but does not have a provider route for OpenRouter embeddings. When Agent Zero calls litellm.embedding() with an OpenRouter model, LiteLLM either:

1. Throws LLM Provider NOT provided (if the model string lacks a recognized prefix)
2. Throws Unmapped LLM provider for this endpoint (if provider is explicitly set to openrouter)

This affects any OpenRouter embedding model, including:

• nvidia/llama-nemotron-embed-vl-1b-v2:free
• sentence-transformers/all-MiniLM-L6-v2 (when served through OpenRouter)
• Any other OpenRouter-hosted embedding endpoint

Error Examples

Case 1: Provider = openrouter, model prefixed

litellm.exceptions.LiteLLMUnknownProvider: Unmapped LLM provider for this endpoint.
You passed model=openrouter/nvidia/llama-nemotron-embed-vl-1b-v2:free, custom_llm_provider=openrouter

Case 2: Provider = openai (workaround attempt), no prefix

litellm.exceptions.BadRequestError: LLM Provider NOT provided.
You passed model=nvidia/llama-nemotron-embed-vl-1b-v2:free

Root Cause

In models.py, LiteLLMEmbeddingWrapper.__init__ constructs:

self.model_name = f"{provider}/{model}" if provider != "openai" else model

Then calls:

resp = embedding(model=self.model_name, input=[text], **self.kwargs)

LiteLLM's get_llm_provider() has no mapping for OpenRouter embeddings, so the call fails before reaching the API.

Related Issues

• openrouter embedding models support #797 — requested OpenRouter embedding support (closed as feature request)
• Memory Fail on First Chat #1050 — same symptom, generic title, has PR Fix embeddings init crash for sentence-transformers models #1070 (unclear if it addresses the LiteLLM root cause)
• OpenRouter Free Models #1076 — OpenRouter free models, related provider confusion

Proposed Fix

Bypass LiteLLM for OpenRouter embeddings and call OpenRouter's OpenAI-compatible /v1/embeddings endpoint directly. This is the same approach many projects take when LiteLLM lacks a provider.

Patch for models.py

Add a _openrouter_embed() helper and gate embed_documents / embed_query:

def _openrouter_embed(self, texts: list) -> list:
    import json
    import urllib.request
    api_key = self.kwargs.get("api_key", "")
    if not api_key:
        raise ValueError("OpenRouter API key not configured.")
    url = "https://openrouter.ai/api/v1/embeddings"
    model = self.model_name
    if model.startswith("openrouter/"):
        model = model[11:]
    payload = json.dumps({"model": model, "input": texts}).encode("utf-8")
    req = urllib.request.Request(url, data=payload, method="POST")
    req.add_header("Authorization", f"Bearer {api_key}")
    req.add_header("Content-Type", "application/json")
    req.add_header("HTTP-Referer", "https://agent-zero.ai")
    req.add_header("X-Title", "Agent Zero")
    with urllib.request.urlopen(req, timeout=60) as resp:
        result = json.loads(resp.read().decode("utf-8"))
    return [d["embedding"] for d in result["data"]]

def _is_openrouter(self) -> bool:
    return (
        self.model_name.startswith("openrouter/")
        or "openrouter.ai" in str(self.kwargs.get("api_base", ""))
    )

def embed_documents(self, texts: list) -> list:
    apply_rate_limiter_sync(self.a0_model_conf, " ".join(texts))
    if self._is_openrouter():
        return self._openrouter_embed(texts)
    resp = embedding(model=self.model_name, input=texts, **self.kwargs)
    ...

def embed_query(self, text: str) -> list:
    apply_rate_limiter_sync(self.a0_model_conf, text)
    if self._is_openrouter():
        return self._openrouter_embed([text])[0]
    resp = embedding(model=self.model_name, input=[text], **self.kwargs)
    ...

Environment

• Agent Zero version: v1.10 (Docker image agent0ai/agent-zero:latest)
• LiteLLM version: whatever ships in current image
• Host: Linux, Docker

Workaround for Users (Until Fixed)

Apply the patch above inside the container, or switch to a local embedding model:

• Provider: huggingface
• Model: sentence-transformers/all-MiniLM-L6-v2

Local models avoid the LiteLLM provider problem entirely.

[azalty]

For OpenAI compatible providers with a custom API URL, for models with a /, Agent Zero thinks the first part is the provider, but it might just be the model name. It fails for me because of this. Embedding models without a "/" in their name work though.

[pfeiffee]

I am seeing this on v1.19 where with or without "/" in the model wont work on any OpenAI style endpoint. At one point simply adding "openai/" prefix to whatever the model name was worked, but now I can't get any combination to work.

[asiqur-rahman]

The root cause here is narrower than a missing LiteLLM OpenRouter embeddings route — it is a one-line asymmetry in Agent Zero itself.

LiteLLMEmbeddingWrapper.__init__ drops the provider prefix whenever the resolved provider is openai:

self.model_name = f"{provider}/{model}" if provider != "openai" else model

LiteLLMChatWrapper.__init__ (models.py:383) always prefixes. That asymmetry is why chat models work against these endpoints while embeddings fail from the same provider config and the same api_base.

All five OpenAI-compatible embedding providers resolve to litellm_provider: openai — openai, openrouter, a0_venice, venice, other — so the bare model id reaches litellm.embedding() with no provider, and LiteLLM re-parses its first path segment as one:

provider	configured model	sent to the endpoint today
openrouter	openai/text-embedding-3-small	text-embedding-3-small — the openai/ is silently eaten
openrouter	nvidia/llama-nemotron-embed-vl-1b-v2:free	BadRequestError: LLM Provider NOT provided
other	openrouter/openai/text-embedding-3-small	routed as provider openrouter, sent as openai/text-embedding-3-small

@azalty your observation was the decisive clue — ids without a / work, ids with one do not. That is exactly this mechanism. @pfeiffee you were also right that adding an openai/ prefix used to work; that is the fix, applied inside the wrapper so it holds for every OpenAI-compatible provider.

Worth flagging for the bundled OpenRouter provider specifically: the id is not rejected, it is quietly truncated before OpenRouter ever sees it, which makes this look like an OpenRouter problem when it is not.

Opened #1843 — one line plus a regression test. Verified against a live OpenAI-compatible gateway: a request that previously failed with No credentials for embedding provider: openai now returns a 1536-dimension embedding.

If any of you can try it against your own setup, confirmation from a second gateway would help.