Skip to content

Auto collect case from channels - #17060

Open
Kyle Zhang (skywing918) wants to merge 27 commits into
mainfrom
collectionWithTeamConnector
Open

Kyle Zhang (skywing918) wants to merge 27 commits into
mainfrom
collectionWithTeamConnector

Conversation

@skywing918

@skywing918 Kyle Zhang (skywing918) commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

fixes https://github.com/Azure/azure-sdk-pr/issues/2663
This pull request introduces a comprehensive Teams channel backfill capability to the Azure SDK QA Bot Agent, enabling the import of historical Teams messages into the bot's data store. The changes span documentation, configuration, backend models, and deployment pipelines to support this new functionality.

Key changes include:

Teams Channel Backfill Feature

  • Added detailed documentation in README.md describing how to configure, deploy, and operate the Teams channel backfill, including authentication, job polling, and data verification steps.
  • Introduced config/teams_collection_config.json to specify allowed Teams channels, their tenant keys, and paging limits for backfill operations.

Backend Model and API Enhancements

  • Extended models/conversation.py with new models for Teams message attachments and images, and added request/response types for backfill job submission and status tracking. This supports richer message metadata and job lifecycle management.

Logic App and Deployment Pipeline Updates

  • Updated the Logic App ARM template and deployment pipeline (pipelines/logicapp-cd.yml) to support a new teams-collection workflow, including environment-specific parameters and backend server identity configuration. [
  • Added deployment parameter files for dev, test, and prod environments to control allowed channels and resource identities for the Teams backfill Logic App.

Data Model and Storage Changes

  • Modified the Logic App template to store Teams message attachments in the extra_info field of conversation documents, ensuring parity with live message ingestion.

@azure-pipelines

azure-pipelines Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
70 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Kyle Zhang added 8 commits September 29, 2026 15:11
…lient and CosmosThreadStore

- Created a new test file for TeamsCollectionService with comprehensive unit tests covering various scenarios.
- Implemented LogicAppPageClient to handle authenticated requests to Logic Apps for Teams message collection.
- Developed CosmosThreadStore for managing Cosmos DB interactions, including reading, writing, and validating documents.
- Added functionality to collect configured channels and manage collection runs with error handling and logging.
kyleZhang and others added 10 commits September 29, 2026 17:12
- Deleted `teams_backfill_service.py` and `teams_collection_service.py` as part of restructuring.
- Introduced `conversation_service.py` to consolidate backfill and collection logic.
- Updated imports across the codebase to reflect the new service structure.
- Modified tests to align with the new service organization and ensure functionality remains intact.
…nector

# Conflicts:
#	tools/sdk-ai-bots/azure-sdk-qa-bot-agent/scripts/deploy_hosted_agent.py
#	tools/sdk-ai-bots/azure-sdk-qa-bot-agent/server.py
@skywing918
Kyle Zhang (skywing918) marked this pull request as ready for review October 27, 2026 08:25
Copilot AI balanced review requested due to automatic review settings October 27, 2026 08:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An update race can overwrite newer live edits with stale Teams data, while cached configuration can break initial backfills and coarse timestamp checks can retain stale summaries.

3 open issues
What changed in this PR

Adds Teams channel history collection and scheduled Q&A summarization to the QA bot, supporting reusable evaluation content while keeping summaries separate from conversation messages.

Changes:

  • Adds asynchronous backend backfill jobs with pagination, message normalization, and attachment/image preservation.
  • Introduces a hosted summarization agent with validated Q&A output and freshness tracking.
  • Extends deployment pipelines, infrastructure, configuration, documentation, and tests.

Paths below are relative to tools/sdk-ai-bots/azure-sdk-qa-bot-agent/.

File Description
utils/​dashboard_identity.py Uses Pydantic-compatible TypedDict.
utils/​azure_cosmosdb.py Adds summary-container access and initialization cleanup.
tests/​test_teams_collection_service.py Covers backfill, summarization, hosting, and deployment contracts.
services/​teams_thread_processor.py Validates and versions model-generated Q&A.
services/​teams_qa_summary_service.py Discovers threads and persists reusable summaries.
services/​teams_operations.py Defines the summarization request protocol.
services/​conversation_service.py Implements historical imports and background job tracking.
server.py Exposes backfill submission and status endpoints.
scripts/​deploy_hosted_agent.py Adds collection deployment and Routine management.
README.md Documents deployment, identities, backfill, and verification.
pipelines/​teams-collection/​template.json Provisions collection workflow, summary storage, and permissions.
pipelines/​teams-collection/​parameters.azure_sdk.test.json Supplies test deployment settings.
pipelines/​teams-collection/​parameters.azure_sdk.prod.json Supplies production deployment settings.
pipelines/​teams-collection/​parameters.azure_sdk.dev.json Supplies development deployment settings.
pipelines/​logicapp/​template.json Preserves attachments in live conversation requests.
pipelines/​logicapp-cd.yml Adds collection deployment and backend identity resolution.
pipelines/​agent-cd.yml Enables collection-agent deployment.
models/​teams_processing.py Defines validated Q&A and resource-enrichment output.
models/​conversation.py Adds attachment, image, and backfill job models.
config/​teams_collection_config.json Configures channels, processing scopes, and schedule.
agents/​teams_collection_agent/​instructions.md Defines thread selection and summarization rules.
agents/​teams_collection_agent/​init.py Hosts background summarization and configures model tools.
agents/​teams_collection_agent/​Dockerfile Packages the hosted collection agent.
agents/​teams_collection_agent/​agent.yaml Registers the hosted agent name.

Review effort: Balanced

Kyle Zhang added 2 commits October 28, 2026 20:09
…nector

# Conflicts:
#	tools/sdk-ai-bots/azure-sdk-qa-bot-agent/pipelines/logicapp-cd.yml
@skywing918 Kyle Zhang (skywing918) changed the title Auto collect evaluation case from channels Auto collect case from channels Nov 4, 2026
"""Raised when a backfill is requested while another one is still running."""


class TeamsBackfillService:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the logicApp, I'm wondering if it's possible to use the Teams connection and call Graph API directly to get the channel's history messages?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already how the backfill works: the backend requests message and reply pages from the existing Logic App, which uses the Teams connection to call Graph. The backend then normalizes the results and writes them to conversation-messages, using ETags to avoid overwriting newer edits. It does not call Graph directly.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants