Repository navigation
docs(use-cua-with): add aimlapi.com provider guide - #1
Merged
Merged
Conversation
The Agent SDK already reaches aimlapi.com: LiteLLM ships an `aiml` provider, so `ComputerAgent(model="aiml/<id>")` resolves with no registration, but nothing said so and readers had no way to discover the prefix, the `AIML_API_KEY` name, or which ids are safe to pass. The loop-selection caveat is the part that costs people a debugging session. Loops are chosen by regex over the entire model string, so `aiml/anthropic/claude-*` matches `.*claude-.*` and lands on the Anthropic hosted-tools loop. That loop sends a `computer_20250124` tool to a chat-completions endpoint, which returns 200 and ignores it, so the run neither errors nor sends a screenshot -- it just narrates. Naming ids that reach a chat-completions loop keeps a first attempt from failing silently. The catalog note exists for the same reason: `GET /v1/models` answers without authentication, so it cannot be used to check a key.
Both lists in this section are hand-ordered by when each provider landed, not alphabetically, so position is editorial rather than mechanical. This moves the aimlapi.com entry to the front of the sidebar group and the "Models and providers" card grid. There is no featured or recommended badge in this docs theme, so nothing is invented; only the order changes. Kept as a separate trailing commit so it can be dropped before the guide is offered upstream.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
aimlprovider, soComputerAgent(model="aiml/<id>")resolves with no registration — but no page said so, and the two things that make a first attempt fail (loop selection by regex, and an unauthenticated model catalog) were undiscoverable.docs/content/docs/use-cua-with/aimlapi.mdx, plus its entry in the sectionmeta.jsonand a card inuse-cua-with/index.mdx. No code changes.Documentation only. The integration itself needs nothing:
aiml/is already a LiteLLM provider pointing athttps://api.aimlapi.com/v1and readingAIML_API_KEY.Related work
Refs — none.
RFC: not required.
rfcs/README.md: "An RFC is not required for small bug fixes, routine documentation changes, internal refactors that preserve public behavior, or urgent private security response." Nothing here touches a public SDK, CLI, MCP, protocol, or file-format contract.Compatibility and risk
/use-cua-with/aimlapi. No behavior change.Validation
Ran from
docs/, on a pristine checkout first and again after the change:pnpm docs:check-hygienepnpm docs:check-linkspnpm buildpython -m pytest libs/python/agent/tests -qExit codes were read from the bare commands, not through a pipe.
Live inference, through
ComputerAgentNot a mock and not a raw curl. Each run constructed a real
ComputerAgentwith a dict-based computer handler (cua_agent.computers.CustomComputerHandler) whosescreenshot()returned a real 1024x768 PNG containing a green button labelledCONTINUEat x 420-620, y 420-480. Prompt: "Look at the screen. Name the button label you see, then click it." Key supplied throughAIML_API_KEY.aiml/z-ai/glm-4.5vGlm4vConfigI can see a Settings window with a green button labeled "CONTINUE". I'll click this button now.then{"type":"click","button":"left","x":526,"y":455}aiml/alibaba/qwen3-vl-plusQwen3VlConfig{"type":"click","button":"left","x":519,"y":452}aiml/alibaba/qwen3-vl-flashQwen3VlConfig{"type":"click","button":"left","x":518,"y":449}aiml/openai/gpt-4o-miniGenericVlmConfig{"type":"click","button":"left","x":512,"y":499}aiml/openai/gpt-5-5GenericVlmConfigCONTINUEThree of the five landed inside the button rectangle and one read its label back verbatim, so the screenshot genuinely reached the model. Tool calling is exercised by the same runs: the computer action is returned as a function call against the loop's
computertool schema.aiml/anthropic/claude-sonnet-4.5was run too and is the reason for the caveat in the page. It selectsAnthropicHostedToolsConfig, which put[{"type":"computer_20250124","function":{"name":"computer","parameters":{"display_height_px":768,"display_width_px":1024,"display_number":1}}}]on a/v1/chat/completionsrequest. The endpoint answered 200 and ignored the tool, so no screenshot was ever sent (23 input tokens) and the model narrated an imaginary "Click Me" button with a pyautogui snippet. It fails silently rather than erroring. This is not aimlapi-specific:find_agent_configmatches.*claude-.*against the whole model string, soopenrouter/anthropic/claude-sonnet-4.5selects the same loop. Left alone here — changing loop selection is a public-behavior change and belongs in its own issue.Request shape on the wire
Every outbound body was captured at the httpx layer:
{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}— thedetailkey is absent, nevernull. That matters, becauseimage_url.detail: nullis rejected with 400 by this endpoint while an omitteddetailand"auto"both return 200. Cua setsdetailnowhere in the tree (grepfor it underlibs/python/agent/cua_agent/returns nothing), so the vision path is safe as written.model,messages, andtoolswhere a loop sends them); no key was serialised asnull. The endpoint rejectsnullontools,temperature,top_p,seed,response_format,streamand others, so this matters and currently holds.Known gaps
litellm.embedding/aembeddingcall anywhere inlibs/python/. Worth knowing anyway: LiteLLM 1.86.2 shipsllms/aiml/chat/andllms/aiml/image_generation/but noembedding/route, so anaiml/<model>embedding call would fail with "Unmapped LLM provider for this endpoint" if one were ever added.pip install cua-agent[qwen]was not sufficient to run the defaultGenericVlmConfigloop in a clean venv:qwen-agentneedsnumpy,soundfileandpython-dateutilat import time andqwen-vl-utilsneedstorch, none of which the extra pulls in. Unrelated to this change, so not touched, but it is the first wall a reader following the page will hit.Contributor and release checks
no-releaselabel is applied — no release-tracked files changed.Commit layout
Two commits, deliberately separated:
docs(use-cua-with): add aimlapi.com provider guide— the page and its registration, in the position a new provider would naturally take (appended afterminimax).chore(aimlapi): fork-only placement — do not send upstream— moves the entry to the front of the sidebar group and the card grid. Drop this commit before offering the guide upstream. No badge was invented; this docs theme has no featured or recommended concept, so only the order changes.