-
Notifications
You must be signed in to change notification settings - Fork 48
docs: improve model configuration guidance #783
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
1538542
58595ba
1c6e91e
796694d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,6 +11,8 @@ | |
|
|
||
| Open `Settings > LLM` to add a reusable LLM profile. Use the **Basic** tab for a provider and model available in the dropdowns. Use the **Advanced** tab when you need to enter a model name and base URL directly. Use the **All** tab to view and customize the full set of model configuration fields. | ||
|
|
||
| If you are deciding between a provider key, local endpoint, LiteLLM proxy, OpenRouter, or ACP agent, start with [Configure a Model](/openhands/usage/agent-canvas/model-configuration). | ||
|
|
||
| <Note> | ||
| ACP agents such as Claude Code, Codex, and Gemini CLI manage their own model access. See [ACP Agents](/openhands/usage/agent-canvas/acp-agents) instead. | ||
| </Note> | ||
|
|
@@ -19,14 +21,14 @@ | |
|
|
||
| | I have | Profile tab | Configure | | ||
| |---|---|---| | ||
| | An API key from Anthropic, OpenAI, Google, or another provider | **Basic** | Select the provider and model, then add its API key. | | ||
| | An OpenHands LLM API key | **Basic** | Select `OpenHands`, choose a model, and add your OpenHands LLM API key. | | ||
| | A local OpenAI-compatible server | **Advanced** | Enter the provider, exact model ID, base URL, and any required API key. | | ||
| | A LiteLLM proxy | **Advanced** | Use the `litellm_proxy/` model prefix, proxy base URL, and proxy API key. | | ||
| | An API key from Anthropic, OpenAI, Google, or another provider | **Basic** | Select the provider and model, then add its API key or reuse a Provider Connection. | | ||
| | An OpenHands LLM API key | **Basic** | Select `OpenHands`, choose a model, then add the key or reuse a Provider Connection. | | ||
| | A local OpenAI-compatible server | **Advanced** | Enter the provider and exact model ID, then add its base URL/key or reuse a Provider Connection. | | ||
| | A LiteLLM proxy | **Advanced** | Use the `litellm_proxy/` model prefix, then add its proxy URL/key or reuse a Provider Connection. | | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We may be happier when this goes back to a single what-to-do, and that is Provider Connection. |
||
|
|
||
| ### Direct Provider | ||
|
|
||
| In the **Basic** tab, select your provider and model, add the API key issued by that provider, and save the profile. Use a new conversation to test the change; an existing conversation continues with the agent and model it started with. | ||
| In the **Basic** tab, select your provider and model. Select a **Provider Connection** to reuse its API key, or add an API key directly when it belongs only to this profile. Save the profile, then use a new conversation to test the change; an existing conversation continues with the agent and model it started with. | ||
|
|
||
| For provider and model recommendations, see [LLM Configuration](/openhands/usage/llms/llms). | ||
|
|
||
|
|
@@ -54,7 +56,7 @@ | |
|
|
||
| ### Local OpenAI-Compatible Endpoint | ||
|
|
||
| A local server can be LM Studio, Ollama, vLLM, SGLang, or another service that exposes an OpenAI-compatible API. In the **Advanced** tab, enter the provider, exact model ID, endpoint base URL, and the required API key or a placeholder value when the server does not require one. | ||
|
Check warning on line 59 in openhands/usage/agent-canvas/llm-profiles.mdx
|
||
|
|
||
| The URL must be reachable from the **backend**, not only from your browser. For example, a backend in Docker cannot use `127.0.0.1` to reach a model server running on the host. Use the host address appropriate for that backend and confirm the endpoint's model inventory before saving. | ||
|
|
||
|
|
@@ -64,7 +66,7 @@ | |
| - **Base URL**: `http://host.docker.internal:1234/v1` | ||
| - **API key**: `local-llm` or another placeholder value when the server does not require authentication | ||
|
|
||
| See [Local LLMs](/openhands/usage/llms/local-llms) for LM Studio, Ollama, and other local-server examples. | ||
|
|
||
| ### LiteLLM Proxy | ||
|
|
||
|
|
@@ -78,7 +80,7 @@ | |
| Provider Connections are available on **local agent-server backends only**. The panel is hidden when using an OpenHands Cloud backend. | ||
| </Note> | ||
|
|
||
| When you want multiple LLM profiles to share the same provider credentials, use **Provider Connections** to store an API key and optional base URL once and reference it across profiles. This avoids pasting the same key into every profile and lets you rotate credentials in one place. | ||
| When you want multiple LLM profiles to share the same provider credentials, use **Provider Connections** to store a provider, API key, and optional base URL once and reference it across profiles. This avoids pasting the same key into every profile and lets you rotate credentials in one place. Linked profiles keep their own model selection while using the shared connection for credentials. | ||
|
|
||
| ### Create a Provider Connection | ||
|
|
||
|
|
@@ -90,13 +92,13 @@ | |
|
|
||
| ### Link a Profile to a Provider Connection | ||
|
|
||
| In the profile editor, use the **provider-connection selector** to link a profile to an existing connection. When a profile is linked, its inline API key and base URL fields are hidden — the profile uses the connection's credentials instead. | ||
| When you add or edit an LLM profile, choose a saved connection in the **Provider Connection** selector. Select **None** to use credentials specific to that profile instead. When a profile is linked, its inline API key and base URL fields are hidden — the profile uses the connection's credentials instead. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is very interesting to me: I have used for ~several months a behavior where you could either
It has been necessary to have that override so that I can use two accounts from the same provider. But fast forward to today in OpenHands, if more than one Provider Connections can be made for a provider, e.g. openai_personal and openai_work, then the old way with a key in the profile can be cleaned out. WDYT?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't quite follow @enyst . Or maybe I do...now that I'm typing this out. Are you suggesting we remove the API key field unless they opt-in by checking a box to fill in an API key? I think thats what I'm visualizing. Separately I think this is in a sort of in-between state of doneness. Provider connection THEN LLM profiles is the intuitive step but its not super clear easy flow yet.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, but that’s just the thing: In my old workflow, it had a checkbox to override for a particular profile, but that’s because I saved one key per provider (that is, a single “litellm provider” had its key saved), but then I needed a second “litellm provider”. But if multiple Provider Connections can be defined and saved for the same provider (“litellm 1 connection”, “litellm 2 connection”) then I think the end state can be: no key directly in profile anymore, no override either. It wouldn’t be needed in the profile at all, right? |
||
|
|
||
| Linked profiles are grouped under their provider connection name in the profile list for readability. | ||
| Linked profiles are grouped under their Provider Connection name in the profile list for readability. To use another model with the same API key, add another LLM profile, select the same connection, choose that model, and save. | ||
|
|
||
| ### Broken Link Badge | ||
| ### Update or Delete a Connection | ||
|
|
||
| If a provider connection is deleted while still referenced by a profile, the profile shows a **Broken link** badge. Re-link the profile to another connection or restore inline credentials to resolve it. | ||
| Edit a Provider Connection to rename it, rotate its API key, or change its base URL. The update applies to every linked profile. Before deleting a connection, re-link or change every profile that uses it; Agent Canvas prevents deleting a connection while profiles still reference it. | ||
|
|
||
| ## Working with LLM Profiles | ||
|
|
||
|
|
@@ -133,7 +135,7 @@ | |
| |---|---|---| | ||
| | Provider is not recognized | Provider selection and model prefix | Use the matching configuration path above. | | ||
| | Model format or identifier error | Exact model ID | Compare it with the provider or proxy model inventory. | | ||
| | Local server cannot be reached | Base URL from the backend | Check host, port, and container or network reachability. | | ||
| | Authentication or permission error | Key type and backend scope | Re-enter the key or follow the provider guide. | | ||
| | Model cannot perform the task | Context and tool support | Choose a compatible model from the provider's recommendations. | | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| --- | ||
| title: Configure a Model | ||
| description: Choose and verify an LLM configuration path in Agent Canvas. | ||
| --- | ||
|
|
||
| Use this guide to choose a model configuration path in Agent Canvas. Start with the credentials or endpoint you have, then save the profile and test it in a new conversation. | ||
|
|
||
| <Note> | ||
| ACP agents such as Claude Code, Codex, and Gemini CLI manage their own model access. Use [ACP Agents](/openhands/usage/agent-canvas/acp-agents) instead of creating an LLM profile for those agents. | ||
| </Note> | ||
|
|
||
| ## Choose a Configuration Path | ||
|
|
||
| | I have | Use | Configure in | | ||
| |---|---|---| | ||
| | An API key from a model provider | A direct provider profile | `Settings > LLM` → `Basic` | | ||
| | An OpenHands LLM API key | An OpenHands provider profile | `Settings > LLM` → `Basic` | | ||
| | A local OpenAI-compatible server | A local endpoint profile | `Settings > LLM` → `Advanced` | | ||
| | A LiteLLM proxy | A proxy profile | `Settings > LLM` → `Advanced` | | ||
| | A signed-in Claude Code, Codex, or Gemini CLI subscription | An ACP agent | `Settings > Agent` | | ||
|
|
||
| ## Provider Connection for Reusable API Credentials | ||
|
|
||
| Create a **Provider Connection** when you expect to use the same provider API key for more than one model or LLM profile. A connection stores the provider, API key, and optional base URL once; each linked profile supplies its own model configuration and uses the connection's credentials. | ||
|
|
||
| 1. Open `Settings > LLM`. | ||
| 2. In **Provider Connections**, select **Add provider connection**. | ||
| 3. Enter a recognizable name, such as `Personal OpenHands API` or `Team OpenAI`. | ||
| 4. Choose the provider from the searchable provider list. | ||
| 5. Enter the API key and, if needed, the provider base URL. | ||
| 6. Save the connection. | ||
| 7. Add or edit an LLM profile, select the connection in **Provider Connection**, then select the model and save the profile. | ||
|
|
||
| When a profile uses a Provider Connection, its API key and base URL come from the connection rather than the profile. Reuse that connection for additional models from the same provider. Update the connection once to rotate its key or change its base URL for every linked profile. | ||
|
|
||
| <Note> | ||
| Provider Connections are available on local agent-server backends. The panel is hidden when using an OpenHands Cloud backend. | ||
| </Note> | ||
|
|
||
| ## Direct Provider or OpenHands Profile | ||
|
|
||
| Use the `Basic` tab when you have an API key from Anthropic, OpenAI, Google, OpenHands, or another provider in the selector. | ||
|
|
||
| 1. If you will reuse the key, create or choose a [Provider Connection](#provider-connection-for-reusable-api-credentials). | ||
| 2. Select the provider and model. | ||
| 3. Select the Provider Connection, or enter the API key directly for a profile-specific credential. | ||
| 4. Save the profile. | ||
| 5. Start a new conversation and send a short message to confirm the model responds. | ||
|
|
||
| For model recommendations and provider references, see [LLM Configuration](/openhands/usage/llms/llms). For the OpenHands provider, see [OpenHands LLM Provider](/openhands/usage/llms/openhands-llms). | ||
|
|
||
| ## Local OpenAI-Compatible Server | ||
|
|
||
| Use the `Advanced` tab for LM Studio, Ollama, vLLM, SGLang, or another server that exposes an OpenAI-compatible API. | ||
|
Check warning on line 54 in openhands/usage/agent-canvas/model-configuration.mdx
|
||
|
|
||
| 1. Find the exact model ID served by your server, usually from its `GET /v1/models` endpoint. | ||
| 2. Enter `openai/<served-model-id>` as the model. | ||
| 3. If the server needs an API key or a reusable base URL, create a [Provider Connection](#provider-connection-for-reusable-api-credentials) with those values and select it for the profile. Otherwise, enter them directly in the profile. | ||
| 4. Make sure the base URL is reachable from the **backend**. | ||
| 5. Save the profile and start a new conversation to verify it. | ||
|
|
||
| If Agent Canvas runs in Docker while the model server runs on the host, `127.0.0.1` points to the container, not the host. Use the host address appropriate for your platform, such as `http://host.docker.internal:<port>/v1` where supported. | ||
|
|
||
| See [Run Local LLMs with OpenHands](/openhands/usage/llms/local-llms) for server-specific examples. | ||
|
|
||
| ## LiteLLM Proxy | ||
|
|
||
| Use the `Advanced` tab when you use a LiteLLM proxy. | ||
|
|
||
| 1. Enter `litellm_proxy/<model-name>` as the model. | ||
| 2. Create or select a [Provider Connection](#provider-connection-for-reusable-api-credentials) for the proxy URL and API key. You can instead enter those values directly when they are specific to one profile. | ||
| 3. Make sure `<model-name>` exactly matches a model configured on the proxy. | ||
| 4. Save the profile and start a new conversation to verify it. | ||
|
|
||
| See [LiteLLM Proxy](/openhands/usage/llms/litellm-proxy) for the complete proxy configuration. | ||
|
|
||
| ## OpenRouter | ||
|
|
||
| Use OpenRouter when you have an OpenRouter API key and want to access a model through its catalog. Create an `OpenRouter` Provider Connection to reuse the key, then in the `Basic` tab select `OpenRouter`, choose a model, select the connection, and save the profile. Use the `Advanced` tab only when you need to enter a model ID that is not available in the selector. | ||
|
|
||
| See [OpenRouter](/openhands/usage/llms/openrouter) for model-ID and recovery guidance. | ||
|
|
||
| ## Fix a Failed Configuration | ||
|
|
||
| | Symptom | Check first | Next step | | ||
| |---|---|---| | ||
| | Provider is not recognized | The profile path and provider/model prefix | Choose the matching path above. | | ||
| | Model ID or format error | The exact model ID from the provider or proxy inventory | Update the model ID or prefix. | | ||
| | Local endpoint cannot be reached | The base URL from the backend | Check host, port, bind address, and container networking. | | ||
| | Authentication or permission error | Key type and provider account access | Re-enter the key and check the provider requirements. | | ||
| | The model cannot complete agent tasks | Context length and tool-use support | Use a more capable model or supported runtime. | | ||
|
|
||
| For additional error-specific guidance, see [Troubleshooting](/openhands/usage/agent-canvas/troubleshooting#model-or-api-key-errors). | ||
|
|
||
| ## Next Steps | ||
|
|
||
| - [Manage LLM Profiles](/openhands/usage/agent-canvas/llm-profiles) | ||
| - [Run Local LLMs with OpenHands](/openhands/usage/llms/local-llms) | ||
| - [LiteLLM Proxy](/openhands/usage/llms/litellm-proxy) | ||
| - [Troubleshooting](/openhands/usage/agent-canvas/troubleshooting) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,14 +1,32 @@ | ||
| --- | ||
| title: OpenRouter | ||
| description: OpenHands uses LiteLLM to make calls to chat models on OpenRouter. You can find their documentation on using OpenRouter as a provider [here](https://docs.litellm.ai/docs/providers/openrouter). | ||
| title: Use OpenRouter with OpenHands | ||
| description: Configure an OpenRouter model and API key in Agent Canvas or OpenHands. | ||
| --- | ||
|
|
||
| ## Configuration | ||
| Use OpenRouter when you have an OpenRouter API key and want to access a model from its [model catalog](https://openrouter.ai/models). | ||
|
|
||
| When running OpenHands, you'll need to set the following in the OpenHands UI through the Settings under the `LLM` tab: | ||
| * `LLM Provider` to `OpenRouter` | ||
| * `LLM Model` to the model you will be using. | ||
| [Visit here to see a full list of OpenRouter models](https://openrouter.ai/models). | ||
| If the model is not in the list, enable `Advanced` options, and enter it in | ||
| `Custom Model` (e.g. openrouter/<model-name> like `openrouter/anthropic/claude-3.5-sonnet`). | ||
| * `API Key` to your OpenRouter API key. | ||
| ## Configure Agent Canvas | ||
|
|
||
| 1. Open `Settings > LLM`. | ||
| 2. In the `Basic` tab, select `OpenRouter` as the provider. | ||
| 3. Select a model, enter your OpenRouter API key, and save the profile. | ||
| 4. Start a new conversation and send a short message to verify the model responds. | ||
|
|
||
| If the model is not in the selector, use the `Advanced` tab and enter its model ID with the `openrouter/` prefix. For example, OpenRouter model ID `anthropic/claude-3.5-sonnet` becomes: | ||
|
|
||
| ```text | ||
| openrouter/anthropic/claude-3.5-sonnet | ||
| ``` | ||
|
|
||
| Copy the current model ID from the [OpenRouter model catalog](https://openrouter.ai/models). Model availability and IDs can change, so do not rely on a previously saved identifier without checking it. | ||
|
|
||
| ## Fix Common Problems | ||
|
|
||
| | Problem | Check | Next step | | ||
| |---|---|---| | ||
| | Model is not found | The exact OpenRouter model ID | Copy the ID from the model catalog and add the `openrouter/` prefix in `Advanced`. | | ||
| | Authentication fails | The API key | Create or copy an active OpenRouter API key, then save the profile again. | | ||
| | The model is unavailable | The selected model in the catalog | Choose an available model or check your OpenRouter account and model access. | | ||
| | The model does not complete agent tasks reliably | Model context and tool-use support | Choose a more capable model that supports the features required for your task. | | ||
|
|
||
| For the broader configuration decision and local/proxy recovery paths, see [Configure a Model](/openhands/usage/agent-canvas/model-configuration). For LiteLLM's provider behavior, see [LiteLLM's OpenRouter documentation](https://docs.litellm.ai/docs/providers/openrouter). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a big deal perhaps, just to note: my impression is that “everyone knows” AgentSkills, and it feels likely a better term than the syntagm “reusable task instructions” - but I’m OK with this too 😅