Figure out `chat_openai_compatible()` strategy

[hadley]

From #635 (comment)

My 2¢ is that chat_openai() is too useful of a function for users to deprecate. I think most users will want to just call chat_openai() to start talking to OpenAI models and they won't care at all about the difference in the underlying API.

Personally, I think it's reasonable to push the messiness of the abstractions into the Provider layers, so I really like ProviderOpenAIResponses and an alternate named ProviderOpenAICompletions or ProviderOpenAIApi or ProviderOpenAICompatible (leaning toward the first).

The problem is that there are likely many uses of chat_openai() that currently target the completions endpoint and are taking advantage of an API marketed as "OpenAI-compatible". If it's possible to catch those use-cases at runtime — e.g. by looking for an error message that would occur if chat_openai() tries to use the responses endpoint on a server that doesn't support it — then ellmer could provide migration instructions and point people to a new chat_openai_compatible() or chat_openai_api() that uses the alternate provider.

I suppose that if it's not possible to provide run-time migration advice, my opinion about the chat_openai() name might change. From an educational perspective, I would much rather teach people to use chat_openai() to talk to OpenAI models than to have to teach chat_openai_responses() just to get started. That name is more appropriate for users who are using "off-label" chat_openai(), and those more advanced users are also aware of the fast-moving nature of this space. In general, I think everyone will understand the need for breaking changes in this area, especially if those changes protect the getting started and most common user experiences.

[cpsievert]

I would be in support of moving chat_openai() and chat_azure_openai() to the responses API, and then keeping the completions API as chat_openai_completions() and chat_azure_openai_completions(). I'm pretty sure I'll be going that direction in posit-dev/chatlas#192

Also, I prefer the *_completions() suffix to *_api() for few reasons (some are more relevant for Python):

1. chat_openai_api() doesn't say much about how it's different from chat_openai() -- they're both APIs
2. In OpenAI SDKs, using completions looks something like OpenAI().chat.completions.create()
3. ChatOpenAIAPI() is a bit awkward in CamelCase.

[hadley]

What about chat_openai_compatible()?

[gadenbuie]

IMHO I also prefer chat_openai_completions() because it maps to OpenAI terminology and because, IIUC, part of being an OpenAI-compatible API is that it implements the v1/chat/completions/ endpoint.

[hadley]

The reason I don't like completions is that I don't think anyone uses that in the wild — people don't say "implements the OpenAI completions API", they say "is OpenAI API compatible".

[gadenbuie]

Yeah, I certainly see an argument that as a user-facing function chat_openai_compatible() makes sense, and I'd be fine with the name. The experience with the chat_openai() function gives me pause, though, that at some point the meaning of "OpenAI API compatible" might drift and then we'll face this discussion again. (I'm not sure how much of a risk that is, though.)

[hadley]

Given that OpenAI's preferred API is not "OpenAI API compatible" I suspect it shouldn't be a problem. My gut feeling is that there's sufficiently widespread usage of the OpenAI API it's going to be very hard to change, and "OpenAI compatible" will be a meaningful term for a long time to come.