feat: add native /responses pass-through for eligible providers

[skamenan7]

🚀 Describe the new functionality needed

OGX exposes /v1/responses today, but the built-in Responses provider owns the agentic loop and eventually calls model providers through /v1/chat/completions. That is the right path for OGX-managed tools, guardrails, conversations, and storage, but it means we do not currently take advantage of providers that already expose a native /v1/responses endpoint.

I would like to add a native Responses path for simple requests where the selected provider supports it. This came up in the community call, and I think it is worth tracking as a focused feature.

The rough shape I have in mind:

Client -> OGX /v1/responses
  -> OGX checks request eligibility and provider support
  -> simple eligible request: use provider-native /responses
  -> anything needing OGX orchestration: use the current built-in Responses loop

This should probably be behind explicit config. For example, the built-in Responses provider config could use a mode like this:

providers:
  responses:
    - provider_id: builtin
      provider_type: inline::builtin
      config:
        # disabled: always use the current OGX-managed Responses loop
        # auto: use provider-native /responses for eligible simple requests,
        #       and use the current OGX-managed path for everything else
        # required: use provider-native /responses only; return a clear error
        #           if the provider does not support it or the request is not eligible
        native_responses_passthrough: auto

For the first pass, I would keep the native path narrow. Something like model, input, instructions, stream, temperature, top_p, max_output_tokens, metadata, and provider-supported reasoning fields feels like a reasonable starting point. Anything outside the allowlist can stay on the current OGX path or fail clearly, depending on the selected mode.

I would not include tools, MCP, connectors, guardrails, conversations, background mode, store=true, previous_response_id, prompts, compaction, file search, or vector search in the raw native path initially. Those have OGX-owned semantics today and need a separate contract before we delegate or proxy them.

💡 Why is this needed? What if we don't build it?

Some providers can handle /v1/responses natively. If OGX always translates /responses into /chat/completions, we lose provider-native Responses behavior even for simple requests.

This would let OGX preserve native provider behavior where available, while still keeping the current OGX-managed path for requests that need tools, guardrails, storage, or other orchestration. It also gives us a clean way to support vLLM/Ollama native Responses when a specific deployment actually exposes that capability.

The main thing I want to avoid is turning this into a raw proxy. For example, guardrails=True is enforced inside the current OGX Responses loop: input is checked before inference, streamed text/reasoning is validated before being emitted, and moderation failures fail closed. Guardrailed requests should stay on the OGX-orchestrated path until a native path can preserve those same checkpoints.

Same for tools and MCP. I do not think we should forward MCP URLs, connector credentials, arbitrary headers, or tool auth tokens to a downstream model provider by default. This also matches the concern @mattf raised on the community call around MCP URLs and tool calls. The provider can produce tool-call intent, but OGX should remain responsible for authorization, execution, approval, allowed_tools, max_tool_calls, auditability, and tenant isolation unless we explicitly design a trusted-provider mode. I would treat provider-hosted tools, like web search or code interpreter, with the same caution for the first version.

Other thoughts

For a first pass, I would keep the scope limited to proving the native routing and fallback model rather than trying to cover every Responses feature at once.

Before coding, I would like to agree on a few practical questions:

• Should native /responses pass-through be added inside the current built-in Responses provider, or should it be a separate provider/path?
• How should OGX know that a provider supports native /responses: hardcoded by provider, configured by the admin, or detected somehow?
• If native pass-through is enabled but the request includes tools, guardrails, or other OGX-managed features, should OGX fall back to the existing path or return an error?
• If the provider returns its own response ID, should OGX store or map it? Can a later previous_response_id use it, or should we avoid that in the first version?
• Since not every vLLM/Ollama deployment may support /responses, what test or version evidence do we need before claiming support?

The main thing I want to avoid is silently falling back, silently dropping fields, or claiming provider support without testing the exact deployment.

Some acceptance criteria I would expect:

• Existing /v1/responses behavior is unchanged unless the new config is enabled.
• Eligible requests can call provider /v1/responses instead of provider /v1/chat/completions.
• Ineligible requests either fall back or fail clearly based on config.
• The first native path does not opt into provider-side storage unless we define that contract.
• Tools, MCP, connector auth, guardrails, provider-hosted tools, and local response state do not leak downstream by default.
• guardrails=True does not bypass the current input/output moderation checkpoints.
• Tests cover both non-streaming and streaming native pass-through for the first supported provider.

[skamenan7]

@franciscojavierarceo and @mattf I created this issue to work on what we discussed in our community call. PTAL.

cc: @leseb , @cdoern

[Keesan12]

I like the narrow first pass here. The main thing I would add is an explicit "execution provenance" field in the response so callers can tell whether OGX answered via provider-native /responses or via the managed OGX loop.

That matters for a few reasons:

• debugging behavior differences between the two paths
• making fallback visible instead of silent
• preserving honest audit trails once tools, guardrails, or MCP force the request back onto the managed path

A couple other guardrails feel high value early:

• treat previous_response_id as out of scope unless OGX can prove the ID belongs to the same execution mode and provider family
• surface a machine-readable ineligibility reason when auto mode falls back (requires_tools, requires_guardrails, provider_unsupported, and so on)
• keep streaming event shapes as similar as possible across both modes, even if the backend path differs

The feature seems strongest if operators can always answer two questions from the outside: what path handled this request, and why.

[franciscojavierarceo]

Btw I have #5602