Skip to content

feat(journaling): measure catalog traversal and provider retries - #11259

Merged
ReubenBond merged 6 commits into
dotnet:mainfrom
ReubenBond:rb-feat-journaling-provider-metrics
Sep 16, 2026
Merged

ReubenBond merged 6 commits into
dotnet:mainfrom
ReubenBond:rb-feat-journaling-provider-metrics

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Sep 15, 2026 •

Copy link
Copy Markdown
Member

Journaling operators need to distinguish catalog candidates from delivered entries and observe explicit storage recovery retries while keeping provider code straightforward.

Add four counters to the Microsoft.Orleans meter: catalog pages, catalog items, catalog entries, and provider retries. Record them directly in the existing S3, Azure Blob, Azure Table, Redis, and Volatile catalog paths and provider retry loops. Catalog tags contain only provider; retry tags also include a bounded reason. Each provider library defines its own short name (s3, azure_blob, azure_table, redis, or volatile) and supplies it to the provider-agnostic recording helper.

Preserve existing storage latency, outcome, and byte metrics. Client SDK telemetry belongs to the application's hosting integrations, such as Aspire, or its instrumentation libraries and diagnostic configuration. The implementation keeps direct SDK calls and native enumeration, with a recording-only helper rather than storage decorators, delegate-based async execution wrappers, or custom telemetry enumerators.

Include focused counter and behavior regressions for filtering, empty pages, early disposal, cancellation, native disposal failures, explicit retries, and caller-supplied provider identities. Documentation explains the counter boundaries and how to reuse host-provided dependency telemetry.

Split from #11210, on top of the catalog and discovery changes landed in #11258 and #11257. Review the current telemetry changes.

Microsoft Reviewers: Open in CodeFlow

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad multi-provider telemetry, catalog, and API changes require final human review.

Review tier: Lite
Findings: None

What changed in this PR

Adds journaling telemetry that separates logical storage operations from underlying SDK calls across S3, Azure Blob/Table, Redis, and Volatile providers.

Changes:

  • Adds bounded operation, outcome, duration, payload, catalog, item, and retry metrics.
  • Extends catalog ranges, metadata projections, layouts, and provider instrumentation.
  • Updates tests, documentation, and generated API surfaces.
File Description
test/​Orleans.Journaling.Tests/​VolatileJournalStorageProviderTests.cs Volatile catalog and range coverage
test/​Orleans.Journaling.Tests/​S3JournalStorageInstrumentsTests.cs S3 telemetry coverage
test/​Orleans.Journaling.Tests/​JournalStorageTelemetryTests.cs Shared telemetry coverage
test/​Orleans.Journaling.Tests/​JournalCatalogRangeTests.cs Catalog range behavior
test/​Orleans.Journaling.Tests/​AzureTableJournalStorageTests.cs Azure Table storage coverage
test/​Orleans.Journaling.Tests/​AzureTableJournalStorageOptionsTests.cs Table mapping coverage
test/​Orleans.Journaling.Tests/​AzureJournalStorageMetricsFixture.cs Azure telemetry fixture
test/​Orleans.Journaling.Tests/​AzureBlobJournalStorageTests.cs Azure Blob telemetry coverage
test/​Orleans.DurableJobs.Tests/​DurableJobs/​JournaledJobShardManagerTests.cs Durable Jobs catalog coverage
test/​Extensions/​Orleans.Redis.Tests/​Journaling/​RedisJournalStorageTests.cs Redis storage coverage
test/​Extensions/​Orleans.Redis.Tests/​Journaling/​RedisJournalStorageTelemetryTests.cs Redis telemetry coverage
test/​Extensions/​Orleans.Redis.Tests/​Journaling/​RedisJournalStorageCatalogTests.cs Redis catalog coverage
src/​Redis/​Orleans.Journaling.Redis/​RedisJournalStorageProvider.cs Redis catalog and lifecycle telemetry
src/​Redis/​Orleans.Journaling.Redis/​RedisJournalStorageOptions.cs Redis mapping options
src/​Redis/​Orleans.Journaling.Redis/​RedisJournalStorage.cs Redis operation instrumentation
src/​Redis/​Orleans.Journaling.Redis/​README.md Redis documentation
src/​Orleans.Journaling/​VolatileJournalStorage.cs Volatile catalog and instrumentation
src/​Orleans.Journaling/​README.md Journaling documentation
src/​Orleans.Journaling/​Orleans.Journaling.csproj Provider internals access
src/​Orleans.Journaling/​ListOptions.cs Catalog filtering options
src/​Orleans.Journaling/​JournalStorageTelemetry.cs Shared telemetry implementation
src/​Orleans.Journaling/​JournalCatalogRange.cs Catalog range logic
src/​Orleans.Journaling/​JournalCatalogEntry.cs Catalog result type
src/​Orleans.Journaling/​InstrumentedJournalStorage.cs Logical operation instrumentation
src/​Orleans.Journaling/​IJournalStorageCatalog.cs Catalog API contract
src/​Orleans.DurableJobs/​JournaledJobShardManager.cs Catalog API migration
src/​Azure/​Orleans.Journaling.AzureStorage/​README.md Azure provider documentation
src/​Azure/​Orleans.Journaling.AzureStorage/​DefaultBlobContainerFactory.cs Blob initialization instrumentation
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureTableJournalStorageProvider.cs Table catalog and query telemetry
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureTableJournalStorageOptions.cs Table mapping options
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureTableJournalStorageInstruments.cs Table metrics
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureTableJournalStorage.cs Table operation instrumentation
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureBlobJournalStorageProvider.cs Blob catalog and page telemetry
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureBlobJournalStorageOptions.cs Blob naming options
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureBlobJournalStorageLayout.cs Blob object layout
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureBlobJournalStorageInstruments.cs Blob metrics
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureBlobJournalStorage.cs Blob operation instrumentation
src/​AWS/​Orleans.Journaling.S3/​S3JournalStorageProvider.cs S3 catalog and lifecycle telemetry
src/​AWS/​Orleans.Journaling.S3/​S3JournalStorageOptions.cs S3 listing and layout options
src/​AWS/​Orleans.Journaling.S3/​S3JournalStorageInstruments.cs S3 metrics
src/​AWS/​Orleans.Journaling.S3/​S3JournalStorage.cs S3 SDK instrumentation
src/​AWS/​Orleans.Journaling.S3/​README.md S3 documentation
src/​api/​Orleans.Journaling/​Orleans.Journaling.cs Generated journaling API
src/​api/​AWS/​Orleans.Journaling.S3/​Orleans.Journaling.S3.cs Generated S3 API
docs/​site/​src/​content/​docs/​host/​monitoring/​metrics-catalog.md Metrics catalog documentation
docs/​site/​src/​content/​docs/​grains/​journaling/​operations.md Operational telemetry guidance
docs/​site/​src/​content/​docs/​grains/​journaling/​azure-storage.md Azure journaling guidance

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage

Metric Pull request Current main Variance
Lines 81.90% (111,376 / 135,994) 81.87% (111,253 / 135,883) +0.0236 pp
Branches 70.94% (31,880 / 44,938) 70.93% (31,854 / 44,912) +0.0168 pp

Report-only conclusion: improved.

The current-main baseline is commit 8c4887aba9 and uses the same reviewed coverage matrix.

Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.

The comparison remains report-only while normal line and branch variance is calibrated.

Coverage details

Copilot AI review requested due to automatic review settings September 15, 2026 21:59
@ReubenBond
ReubenBond force-pushed the rb-feat-journaling-provider-metrics branch from 8c99b8b to ae553e5 Compare September 15, 2026 21:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Enumeration disposal failures can currently be recorded as successful outcomes.

Get a fresh assessment by requesting another Copilot review.

Review tier: Lite
Findings: 1 Medium severity

Open (1)

Copilot AI review requested due to automatic review settings September 15, 2026 23:51
@ReubenBond ReubenBond changed the title feat(journaling): measure provider operations and SDK calls feat(journaling): measure catalog traversal and provider retries Sep 15, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The only final finding is a non-blocking documentation nit.

Review tier: Lite
Findings: None

Resolved since last review (1)

Copilot AI review requested due to automatic review settings September 16, 2026 00:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

One or more issues must be addressed before approval.

Review tier: Lite
Findings: None

Copilot AI review requested due to automatic review settings September 16, 2026 00:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Restore or link migration guidance for the changed S3 and Redis storage layouts.

Review tier: Lite
Findings: None

Previously missed (1)

In code that hasn't changed since last review

Low severity Retain the S3 layout migration warning

src/​AWS/​Orleans.Journaling.S3/​README.md:57

The previous migration warning was removed while the provider now reads WALs from wal/<base-key> instead of the former <base-key>/wal layout, and there is no fallback for existing objects. Without this warning, an upgrade can make existing journals undiscoverable; restore the drain/recreate guidance before the catalog documentation.

@ReubenBond
ReubenBond merged commit 9222be9 into dotnet:main Sep 16, 2026
73 checks passed
@ReubenBond
ReubenBond deleted the rb-feat-journaling-provider-metrics branch September 16, 2026 03:46
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants