Support Microsoft Foundry hosted agents: package GPT-RAG orchestration as a hosted agent (ADR-0001)

[placerda]

Tracking issue for the architecture decision recorded in ADR-0001.

ADR: docs/adr/ADR-0001-hosted-agents.md

Summary

Add support for Microsoft Foundry hosted agents by packaging the GPT-RAG orchestration as a hosted agent (ADR-0001, Option 1). In this mode the orchestration runs inside Foundry, the Chainlit UI talks to it as a thin BFF, and the orchestrator Azure Container Apps becomes optional, surviving only as the administrative panel backend. The mode is opt-in behind a provisioning flag, and the classic Container Apps mode stays as the supported default and fallback while the feature is in preview.

Why will we implement this?

• Problem / opportunity: The field reports that customers want hosted agents as an alternative to Azure Container Apps, that is, one less resource to operate, not one more piece on top of what already exists. Option 2 (consume hosted agents as sub-agents) was discarded because it keeps Container Apps running.
• Business value / outcome: In the no-panel mode the customer no longer operates the orchestrator Container Apps. Runtime, scaling, session state, agent identity, versioning, and observability come from the platform. The project aligns with Foundry's official path for agents.
• Success metrics (how we know it worked): A tenant can run GPT-RAG chat with the orchestrator Container Apps not provisioned; the classic mode keeps working unchanged when the flag is off; native document-level security passes the two-user validation described below.

What does it do? (Functional Overview)

• Core behavior: Introduce a runtime-agnostic adapter boundary. Extract the orchestration core (the strategies) into a library behind a stable contract, with two thin adapters: FastAPI (Container Apps) and Responses (hosted agent). Publish a single hosted agent that loads the eligible strategies and selects by AGENT_STRATEGY at the version level. The strategies are preserved; the runtime change is plumbing, not strategy logic. The UI keeps Chainlit as a thin BFF that speaks Responses with the hosted agent (new path selected by config, for example CHAT_BACKEND=hosted_agent).
• Non-negotiable requirement (document-level security): Option 1 is only acceptable if it preserves, end to end, the native document-level security of Foundry IQ and Azure AI Search. The Foundry gateway drops the Authorization header, so the user token does not reach the container and classic OBO cannot repeat as before. Target path: pass the user identity through the Toolbox (MCP) so Foundry IQ and Search apply the native cut using the user's Entra groups against the document permissions. Fallback: the UI resolves the user's groups via Microsoft Graph and sends a group filter by header (x-client-*) or in the body. Acceptance test: two users from different groups, the one without permission cannot see the restricted document.
• Data collection / storage needs: Basic conversation history moves to Foundry managed Conversations, durable and independent of the compute. Feedback, the administrative panel, and the dashboard stay on the optional smaller Container Apps with Cosmos, provisioned only when the panel is on. Caveat: today conversation and feedback persistence is coupled to the chat flow in orchestration/orchestrator.py; for Container Apps to be truly optional, history and feedback writes need to move into the hosted agent.
• Provisioning flags: DEPLOY_HOSTED_AGENT_ORCHESTRATION (default false) turns on hosted agent mode and makes the orchestrator Container Apps conditional. DEPLOY_ADMINISTRATIVE_PANEL (default false) brings up the smaller Container Apps only for history, feedback, and dashboard. Three modes result: classic (default), hosted agent without panel, hosted agent with panel.
• Nice to have (stretch goals): Optional later consolidation of strategies (fold single_agent_rag and mcp into a lean base, reclassify nl2sql and multimodal as composable tools or connected agents). This is cleanup, not a prerequisite to ship.

Components

• Components (check all that apply):
  • gpt-rag-orchestrator
  • gpt-rag-ingestion
  • gpt-rag-ui
  • gpt-rag-mcp

Also affects infrastructure in this umbrella repo (infra, azure.yaml): the flags, the conditional orchestrator Container Apps module, and the hosted agent deploy via azure.ai.agent. Ingestion is touched mainly to register its MCP source in the Toolbox.

Phased path (from ADR-0001)

1. Orchestrator: extract the agentic core into a library behind adapters, starting from maf_agent_service, and decide persistence ownership.
2. Infra and azd: add the flags, make the orchestrator Container Apps conditional, provision the hosted agent, write the chat backend and agent endpoint into config.
3. UI: add the hosted_agent path in the BFF, adapt streaming and conversation id to Responses.
4. Ingestion and Toolbox: register the MCP source and validate connectivity and identity.
5. Identity and document-level security: close identity passthrough and validate native trimming end to end with the two-user test.

Open decisions (from ADR-0001)

• Persistence ownership: do history and feedback move fully into the hosted agent?
• Document-level security: is the Toolbox identity passthrough enough, or do we keep the group filter as a first-class path too?
• Eligibility under the 2 vCPU / 4 GiB ceiling: where do nl2sql and multimodal run?
• Private ACR build: jumpbox, CI runner in the VNet, or ACR Tasks?
• Target environment: VNet topology, egress, and region (Brazil South is supported).

Constraints and preview notes

• Hosted agents are in preview, no SLA, no native canary or traffic splitting; versions are immutable and rollback means reverting to a previous version.
• Private ACR behind a private endpoint depends on the Foundry project creation date (supported for projects created after 2026-06-25).
• The classic Container Apps mode remains the supported default and fallback.

Delivery model and repository ownership

Program planning, dependency status, and acceptance evidence are consolidated in Azure/GPT-RAG through the sub-issues below. Each implementation issue identifies the repository that owns the corresponding code change.

• Component changes produce pull requests in the owning component repository and reference the canonical issue in Azure/GPT-RAG.
• Canonical issues remain open until their acceptance evidence and component pull requests have been reviewed.
• Blocked work records the unresolved dependency or decision on the canonical issue rather than producing an empty pull request.
• Component tags, umbrella releases, and production publication require explicit maintainer approval.

Ordered sub-issues and precedence

Wave 0 - freeze contracts

• [Hosted agents 01] Freeze implementation contracts and release gates #588

Wave 1 - start in parallel after Wave 0

• [Hosted agents 02] Extract the runtime-neutral orchestration core #590 - implemented in Azure/gpt-rag-orchestrator via feat: extract runtime-neutral orchestration boundary (TurnRequest) gpt-rag-orchestrator#284
• [Hosted agents 04] Expose ingestion retrieval through Foundry Toolbox #596 - implemented in Azure/gpt-rag-ingestion via Secure hosted retrieval through native identity passthrough gpt-rag-ingestion#268

Wave 2 - hosted runtime

• [Hosted agents 03] Add hosted Responses adapters and managed Conversations #598 - implemented in Azure/gpt-rag-orchestrator via feat: add hosted Responses adapters and managed Conversations entrypoint gpt-rag-orchestrator#285

Wave 3 - start in parallel after hosted runtime

• [Hosted agents 05] Add reusable hosted-agent infrastructure primitives #594 - implemented and released in Azure/bicep-ptn-aiml-landing-zone
• [Hosted agents 06] Add the Chainlit hosted-agent BFF path #587 - implemented and released in Azure/gpt-rag-ui

Wave 4 - immutable component release gate

• [Hosted agents 07] Publish the validated component release train #593 - exact released tags and protected commits recorded

Wave 5 - umbrella integration

• [Hosted agents 08] Wire flags, configuration, lifecycle hooks, and exact pins #595 - flags, configuration, lifecycle hooks, and exact pins integrated

Wave 6 - topology validation

• [Hosted agents 09] Validate the three deployment modes end to end #592 - starts after [Hosted agents 08] Wire flags, configuration, lifecycle hooks, and exact pins #595

Wave 7 - security and private-network validation in parallel

• [Hosted agents 10] Prove document-level authorization through the hosted path #591 - starts after [Hosted agents 09] Validate the three deployment modes end to end #592
• [Hosted agents 11] Validate private networking, image build, and version rollback #597 - starts after [Hosted agents 09] Validate the three deployment modes end to end #592 and requires an approved VNet-connected execution path

Wave 8 - documentation and umbrella release

• [Hosted agents 12] Publish documentation and the validated umbrella release #589 - starts after [Hosted agents 10] Prove document-level authorization through the hosted path #591 and [Hosted agents 11] Validate private networking, image build, and version rollback #597; publication requires human approval