Azure support on URI construction is limited if the service is provided through proxy

[hellocomrade]

Spring AI 2.0 M6 introduced the following change:

"Azure OpenAI functionality is now available through the standard spring-ai-openai module, which provides full Azure OpenAI deployment support."

However, this change does not appear to support a common enterprise deployment scenario where a proxy server sits between the application and Azure OpenAI.

Currently, Azure-specific behavior seems to be inferred from the target domain name. When requests are routed through an enterprise proxy whose hostname does not contain the expected Azure domain pattern, Spring AI does not apply Azure-specific request handling (such as deployment-based paths, query parameters, or headers). As a result, requests sent through the proxy fail, typically with HTTP 404 responses.

Expected Behavior

Azure OpenAI support should not rely solely on hostname/domain detection. It should be possible to explicitly configure Azure mode even when requests are routed through a non-Azure proxy endpoint.

Example Scenario

Application → Enterprise Proxy → Azure OpenAI

Example proxy endpoint:

https://llm-proxy.company.internal

Even though the backend target is Azure OpenAI, Spring AI treats the endpoint as a standard OpenAI-compatible API because the hostname is not an Azure domain.

Impact

This affects enterprise environments where:

outbound traffic must traverse API gateways or proxy servers
Azure OpenAI access is centralized behind internal infrastructure
applications are not allowed to directly access Azure endpoints
Suggested Improvement

Consider adding an explicit configuration option to force Azure OpenAI mode independently of endpoint hostname detection.

For example:

spring.ai.openai.azure.enabled=true

or a configurable API type/provider option similar to previous Azure-specific integrations.

[jewoodev]

Hi, I can take a look at this.

@hellocomrade, thanks for the detailed write-up. I traced the provider selection path, and the proxy topology looks valid, but the current code seems to already have an explicit Microsoft Foundry override before it falls back to hostname-based detection.

OpenAiSetup#detectModelProvider(...) checks the isMicrosoftFoundry flag first, and that flag is exposed as spring.ai.openai.microsoft-foundry. Could you try the proxy endpoint with this configuration?

spring:
  ai:
    openai:
      base-url: https://llm-proxy.company.internal
      api-key: ${PROXY_API_KEY}
      microsoft-foundry: true
      microsoft-foundry-service-version: V2024_10_21
      chat:
        model: <your-azure-deployment-name>

If 404 still happens with this config, that would help confirm whether there is a remaining proxy-specific issue beyond provider detection.

Even if this configuration works, I think there is still a focused documentation gap: the Connection Properties table in openai-chat.adoc does not list the Microsoft Foundry settings, and azure-openai-chat.adoc does not explain the proxy/gateway case. I can work on a small docs PR that adds those properties and a short "Azure / Microsoft Foundry behind a proxy" example, if that sounds useful.

[hellocomrade]

@jewoodev

Thank you for the timely response! I was able to reproduce the 404 issue with the following configuration:

spring:
  ai:
    openai:
      base-url: https://my_enterprise_domain/azure-openai-api
      api-key: ${PROXY_API_KEY}
      microsoft-foundry: true
      microsoft-foundry-service-version: V2024_10_21
      embedding:
        model: <your-azure-deployment-name>

The failure occurs in com.openai.services.blocking.EmbeddingServiceImpl.kt.

The request generated by Spring AI is:

HttpRequest{
  method=POST,
  baseUrl=https://my_enterprise_domain/azure-openai-api,
  pathSegments=[embeddings],
  headers=Headers{map={
    api-key=[my_api_key],
    OpenAI-Organization=[my_org_id],
    User-Agent=[spring-ai-openai],
    X-Stainless-Arch=[x64],
    X-Stainless-Kotlin-Version=[2.2.21],
    X-Stainless-Lang=[java],
    X-Stainless-OS=[Windows],
    X-Stainless-OS-Version=[10.0],
    X-Stainless-Package-Version=[4.34.0],
    X-Stainless-Runtime=[JRE],
    X-Stainless-Runtime-Version=[25.0.1]
  }},
  queryParams=QueryParams{map={}},
  body=com.openai.core.http.HttpRequestBodies$json$1@43c66df5
}

As shown above, the generated URI follows the standard OpenAI format (/embeddings) instead of the Azure OpenAI format:

/openai/deployments/{deployment}/embeddings?api-version=...

This appears to happen because Azure-specific request handling is not activated when the configured base-url uses an enterprise proxy domain instead of an Azure domain.

To verify this, I implemented a custom embedding model that explicitly uses the Azure OpenAI deployment path, and it works correctly through the same proxy setup:

public class AzureRestEmbeddingModel extends AbstractEmbeddingModel {

    private static final ObjectMapper MAPPER = new ObjectMapper();

    private final RestClient restClient;
    private final String apiVersion;
    private final String model;

    public AzureRestEmbeddingModel(RestClient restClient, String model, String apiVersion) {
        this.restClient = restClient;
        this.model = model;
        this.apiVersion = apiVersion;
    }

    @Override
    public EmbeddingResponse call(EmbeddingRequest request) {
        List<String> inputs = request.getInstructions();
        Map<String, Object> body = Map.of("input", inputs);

        String path = "/openai/deployments/" + model + "/embeddings?api-version=" + apiVersion;

        String raw;
        try {
            raw = restClient.post()
                    .uri(path)
                    .body(body)
                    .retrieve()
                    .body(String.class);
        }
        catch (HttpClientErrorException ex) {
            log.error("Embedding call failed: HTTP {} — body: {}",
                    ex.getStatusCode(),
                    ex.getResponseBodyAsString());
            throw ex;
        }

        try {
            JsonNode root = MAPPER.readTree(raw);
            JsonNode data = root.get("data");

            List<Embedding> embeddings = new ArrayList<>();

            for (JsonNode item : data) {
                JsonNode arr = item.get("embedding");

                float[] vector = new float[arr.size()];
                for (int i = 0; i < arr.size(); i++) {
                    vector[i] = (float) arr.get(i).doubleValue();
                }

                embeddings.add(new Embedding(
                        vector,
                        item.get("index").intValue(),
                        EmbeddingResultMetadata.EMPTY));
            }

            return new EmbeddingResponse(
                    embeddings,
                    new EmbeddingResponseMetadata(model, null));
        }
        catch (Exception e) {
            throw new RuntimeException(
                    "Failed to parse embedding response: " + raw, e);
        }
    }

    @Override
    public float[] embed(Document document) {
        EmbeddingResponse response = call(
                new EmbeddingRequest(List.of(document.getText()), null));

        return response.getResult().getOutput();
    }
}

[hellocomrade]

Hi,

I have some extra tokens to burn today... please let me know if this makes sense:

Root Cause

There are two independent layers of Azure URL logic — Spring AI's OpenAiSetup and the Stainless SDK's AzureUrlCategory — and they fail to agree when the base URL is a proxy with a path component.

Layer 1 — Spring AI OpenAiSetup.detectModelProvider()

  // AUTO detection in Spring AI — checks URL suffix
  if (baseUrl.endsWith("openai.azure.com") || baseUrl.endsWith("cognitiveservices.azure.com") ...) {
      return ModelProvider.MICROSOFT_FOUNDRY;
  }
  // fallback: explicit parameters
  if (azureDeploymentName != null || azureOpenAIServiceVersion != null) {
      return ModelProvider.MICROSOFT_FOUNDRY;
  }

With spring.ai.openai.microsoft-foundry=true set, this correctly returns MICROSOFT_FOUNDRY regardless of the URL. Spring AI then calls:

  builder.baseUrl("https://our_entrp_proxy/azure-open-ai")
  builder.credential(AzureApiKeyCredential.create(apiKey))
  builder.azureServiceVersion(azureOpenAiServiceVersion)
  // ← NEVER calls builder.azureUrlPathMode(...)

The azureUrlPathMode stays at its default: AUTO.

Layer 2 — Stainless SDK AzureUrlCategory.categorizeBaseUrl()

  AzureUrlPathMode.AUTO ->
      when {
          host.endsWith(".openai.azure.com", ignoreCase = true) ||
          host.endsWith(".services.ai.azure.com", ignoreCase = true) ||
          host.endsWith(".azure-api.net", ignoreCase = true) ||
          host.endsWith(".cognitiveservices.azure.com", ignoreCase = true) ->
              if (trimmedBaseUrl.endsWith("/openai/v1")) AZURE_UNIFIED else AZURE_LEGACY
          else -> NON_AZURE   // ← proxy host hits this
      }

The host extracted from https://my_enterprise_domain/azure-openai-api is my_enterprise_domain. It matches none of the Azure domain suffixes → NON_AZURE.

The Consequence — HttpRequestBuilderExtensions

  // Only AZURE_LEGACY gets the deployment path injected
  if (urlCategory == AzureUrlCategory.AZURE_LEGACY) {
      builder.addPathSegments("openai", "deployments", deploymentModel, ...)
  }
  // NON_AZURE: path injection is skipped entirely

And in ClientOptions, NON_AZURE also skips the api-version query param injection.

So the actual outgoing request URL becomes:
https://my_enterprise_domain/azure-open-ai/chat/completions ← wrong
instead of:
https://my_enterprise_domain/azure-open-ai/openai/deployments/gpt-5/chat/completions?api-version=2025-01-01-preview

---

Why the workaround Works

AzureChatModelConfig.buildClient() bypasses both broken layers:

.baseUrl(azureEndpoint + "/openai/deployments/" + deployment) // pre-bakes the full path
.putQueryParam("api-version", chatApiVersion) // manually adds api-version

With the deployment path pre-baked, the SDK's skipped path injection doesn't matter. The manually added api-version replaces the skipped query param injection. The SDK ends up appending only
/chat/completions to the already-complete base URL.

---

The Actual Fix (Spring AI side)

Spring AI's OpenAiSetup.setupSyncClient() and OpenAiSetup.setupAsyncClient() should propagate their own MICROSOFT_FOUNDRY decision to the Stainless SDK by adding one line:

  // setupSyncClient()
  if (modelProvider == ModelProvider.MICROSOFT_FOUNDRY) {
      builder.azureUrlPathMode(AzureUrlPathMode.LEGACY);
  }

  // setupAsyncClient() — same fix, different builder type
  if (modelProvider == ModelProvider.MICROSOFT_FOUNDRY) {
      builder.azureUrlPathMode(AzureUrlPathMode.LEGACY);
  }

This would force AZURE_LEGACY regardless of what the host looks like, making the deployment path injection and api-version injection work correctly for any base URL including proxy paths.

[jewoodev]

Thanks for the detailed follow-up. I agree this shows the problem is not only Spring AI's provider detection.

Your analysis of the SDK AzureUrlPathMode.AUTO path matches what I found locally in the openai-java 4.34.0 sources: Spring AI detects MICROSOFT_FOUNDRY, but the SDK still keeps AzureUrlPathMode.AUTO. With a proxy host, the SDK categorizes the URL as NON_AZURE, so it skips both the deployment path injection and the api-version query parameter.

I'll open a small PR for this. One thing I'd like to preserve is the existing unified Azure /openai/v1 behavior, so rather than always forcing LEGACY, I'll propagate an explicit SDK path mode based on the resolved Microsoft Foundry provider and the base URL shape. I'll keep the change focused, cover both sync and async clients, and add a targeted regression test for the proxy-host case. That way the approach is easy to review or adjust if maintainers prefer a different direction.

[hellocomrade]

@jewoodev thank you for the help! Please let me know if you have the PR.

[ilyakastsenevich]

@hellocomrade thanks for report, affected by this too
company-proxy/openai/deployments/${deployment-name}/chat/completions

Also my company's llm proxy requires "Api-Key" header instead of "Authorization".
It worked with azure openai starter in 1.1.0-M3.

[jewoodev]

@ilyakastsenevich Thanks for confirming the same path pattern (company-proxy/openai/deployments/${deployment-name}/chat/completions) — that's exactly what the SDK skips when the host falls outside its Azure-domain whitelist.

I opened #6180 to propagate the resolved Microsoft Foundry provider to the SDK's AzureUrlPathMode. For an enterprise proxy base-url it sets LEGACY, which restores the /openai/deployments/{deployment} path and the ?api-version=... query in both sync and async clients.

Your "1.1.0-M3 worked" note helped narrow the regression to commit 75fa48486 ("Migrate spring-ai-openai to official openai-java SDK"). Before that, Spring AI used its own OpenAI client that did not depend on the SDK's host-whitelist categorization.

The Api-Key vs Authorization header behavior sits in a separate code path from #6180 — please open a follow-up issue with the outgoing headers if Authorization still leaks through after this PR.

If you can build from the branch and try it against your proxy setup before maintainers pick this up, that would be a useful confirmation.

[jewoodev]

@hellocomrade Opened #6180 — propagates the Microsoft Foundry provider decision to the SDK's AzureUrlPathMode, which restores the /openai/deployments/{deployment} path and the ?api-version=... query for proxy base URLs. Covers both sync and async clients in OpenAiSetup with a targeted regression test for the proxy-host case.

[ilyakastsenevich]

@jewoodev i've built your branch and the issue is fixed using this config:

spring:
  ai:
    openai:
      base-url: https://my-company-proxy.com
      api-key: ${API_KEY}
      microsoft-foundry: true
      chat:
        model: anthropic.claude-sonnet-4-6
        temperature: 0.5

the url path is https://my-company-proxy.com/openai/deployments/anthropic.claude-sonnet-4-6/chat/completions as expected and the api-key header is used instead of authorization