Skip to content

Show multi-step test spans in class timeline, align report ordering with execution, and correlate linked OTel activities#5847

Draft
Copilot wants to merge 9 commits intomainfrom
copilot/feature-multi-step-test-execution
Draft

Show multi-step test spans in class timeline, align report ordering with execution, and correlate linked OTel activities#5847
Copilot wants to merge 9 commits intomainfrom
copilot/feature-multi-step-test-execution

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 7, 2026

Multi-step DependsOn workflows were hard to follow in the HTML report because the class timeline excluded test-case spans, and the test list ordering could diverge from the span ordering. This updates the reporter to surface those per-step spans in the class timeline and present tests in execution order so BDD-style flows are easier to read.

This PR also fixes the missing OpenTelemetry activity correlation path for distributed flows that continue on a linked trace rather than the original test trace. Incoming OTLP spans that carry links to the test trace are now associated back to the owning test so they can appear in the report and correlate logs/output correctly.

  • Class timeline

    • Stop filtering out test-case spans from the class-level waterfall.
    • Keep the timeline readable by collapsing synthetic test body spans and re-parenting their children to the test-case span.
  • Execution ordering

    • Sort tests in each class by StartTime, then by name as a stable tie-breaker.
    • This keeps the visible test list aligned with the order shown in the timeline for chained DependsOn scenarios.
  • OTel linked-trace correlation

    • Register derived OTLP traces back to the owning test when incoming spans include links to the original test trace.
    • Allow the activity collector to ingest external spans that are known via TraceRegistry, even if their trace IDs were not seen by the in-process listener first.
    • This covers async/distributed workflows where downstream processing starts a new trace but preserves causality through OTEL span links.
  • Regression coverage

    • Add focused reporter tests for:
      • test ordering by execution start time
      • generated HTML containing the updated class timeline logic
    • Add focused OpenTelemetry tests for:
      • OTLP receiver ingestion of linked/derived traces
      • trace-to-test correlation for externally reported spans

Example of the kind of flow this makes visible in the class timeline:

[Test]
public Task SetupWiremock() => Task.CompletedTask;

[Test]
[DependsOn(nameof(SetupWiremock))]
public Task SendMessage() => Task.CompletedTask;

[Test]
[DependsOn(nameof(SendMessage))]
public Task VerifyWiremockWasCalled() => Task.CompletedTask;

With this change, those steps are rendered in the class timeline instead of being hidden behind suite-level spans only, and linked downstream OTel activities can now be attributed back to the triggering test.

@codacy-production
Copy link
Copy Markdown

codacy-production Bot commented May 7, 2026

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Copilot AI and others added 2 commits May 7, 2026 11:20
Agent-Logs-Url: https://github.com/thomhurst/TUnit/sessions/48d47cff-3125-44ff-b19b-e1dfe712be9f

Co-authored-by: thomhurst <30480171+thomhurst@users.noreply.github.com>
Agent-Logs-Url: https://github.com/thomhurst/TUnit/sessions/48d47cff-3125-44ff-b19b-e1dfe712be9f

Co-authored-by: thomhurst <30480171+thomhurst@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement multi step test execution with OTEL and Aspire Show multi-step test spans in class timeline and align report ordering with execution May 7, 2026
Copilot AI requested a review from thomhurst May 7, 2026 11:23
Copilot AI temporarily deployed to Pull Requests May 7, 2026 11:27 Inactive
Copilot AI temporarily deployed to Pull Requests May 7, 2026 11:27 Inactive
Copilot AI temporarily deployed to Pull Requests May 7, 2026 11:27 Inactive
@thomhurst
Copy link
Copy Markdown
Owner

@copilot read the latest comments on the issue. You haven't addressed the missing OTel activities.

Agent-Logs-Url: https://github.com/thomhurst/TUnit/sessions/5339a2a6-32c0-4e8e-b65a-72c686b6873e

Co-authored-by: thomhurst <30480171+thomhurst@users.noreply.github.com>
Copilot AI and others added 5 commits May 7, 2026 13:23
Agent-Logs-Url: https://github.com/thomhurst/TUnit/sessions/5339a2a6-32c0-4e8e-b65a-72c686b6873e

Co-authored-by: thomhurst <30480171+thomhurst@users.noreply.github.com>
Agent-Logs-Url: https://github.com/thomhurst/TUnit/sessions/5339a2a6-32c0-4e8e-b65a-72c686b6873e

Co-authored-by: thomhurst <30480171+thomhurst@users.noreply.github.com>
Agent-Logs-Url: https://github.com/thomhurst/TUnit/sessions/5339a2a6-32c0-4e8e-b65a-72c686b6873e

Co-authored-by: thomhurst <30480171+thomhurst@users.noreply.github.com>
Agent-Logs-Url: https://github.com/thomhurst/TUnit/sessions/5339a2a6-32c0-4e8e-b65a-72c686b6873e

Co-authored-by: thomhurst <30480171+thomhurst@users.noreply.github.com>
Agent-Logs-Url: https://github.com/thomhurst/TUnit/sessions/5339a2a6-32c0-4e8e-b65a-72c686b6873e

Co-authored-by: thomhurst <30480171+thomhurst@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 7, 2026

@copilot read the latest comments on the issue. You haven't addressed the missing OTel activities.

Addressed in ec211ae. The OTLP receiver now maps derived/linked traces back to the owning test when incoming spans carry links to the original test trace, and the activity collector now accepts registry-backed external traces so those activities show up in the report. I also added a regression test that posts a linked OTLP trace payload through /v1/traces to cover this path.

Copilot AI changed the title Show multi-step test spans in class timeline and align report ordering with execution Show multi-step test spans in class timeline, align report ordering with execution, and correlate linked OTel activities May 7, 2026
Copilot AI temporarily deployed to Pull Requests May 7, 2026 13:45 Inactive
Copilot AI temporarily deployed to Pull Requests May 7, 2026 13:45 Inactive
Copilot AI temporarily deployed to Pull Requests May 7, 2026 13:45 Inactive
@thomhurst
Copy link
Copy Markdown
Owner

Investigation summary

I dug into this and verified the PR mechanically:

  • The new OtlpReceiverIngestionTests.Receiver_ParsedLinkedTrace_RegistersAgainstOwningTest passes locally (dotnet run -f net10.0 -- --treenode-filter "/*/*/OtlpReceiverIngestionTests/*" → 2/2 passed).
  • The link-registration logic is sound: when an OTLP span arrives carrying a Link to a registered test traceId, TraceRegistry.TryRegisterDerivedTrace correctly maps the derived traceId back to the same test (and copies TraceToContextId so /v1/logs correlation also works).
  • The ActivityCollector.IngestExternalSpan fallback to TraceRegistry.IsRegistered is correct — it lets a newly-derived trace flow through without depending on the in-process listener having seen it first.

However, the link-registration path only matters once OTLP spans actually arrive at TUnit's receiver. Based on the screenshots in #5845, the more likely root cause of "nothing changed" is upstream of this PR:

  1. TUnitOpenTelemetry.Configure(b => b.AddSource("Microsoft.Azure.Functions.Worker")) only configures the in-process TracerProvider running inside the test host. The activities the user is missing are emitted inside the Functions Worker (func.exe) — a separate process. The in-process listener never sees them, regardless of AddSource.
  2. For Worker spans to ever reach TUnit, the Worker itself must:
    • register an OpenTelemetry TracerProvider with AddOtlpExporter(),
    • subscribe to Microsoft.Azure.Functions.Worker and Azure.Messaging.ServiceBus.* sources from inside the Worker,
    • and pick up the OTEL_EXPORTER_OTLP_ENDPOINT env var TUnit injects.
  3. The Wiremock screenshot showing the test's trace-id reaching the downstream HTTP only proves traceparent propagation works (just Activity.Current being set). It does not imply the Worker is exporting spans — Activity emission ≠ OTLP export.

So the linkage code in this PR is correct and worth keeping for messaging scenarios where the consumer creates a new trace with a Span Link (which is what Azure.Messaging.ServiceBus does), but it can't help if no OTLP traffic is reaching the receiver in the first place.

Suggested follow-ups

  • For [Feature]: Multi step test execution with OTEL and Aspire (Gherkin - BDD style) #5845 reporter: confirm the Functions Worker has its own OTel pipeline registered (e.g. via AddServiceDefaults() or an explicit AddOpenTelemetry().WithTracing(t => t.AddOtlpExporter().AddSource(...))) and that the Worker actually picks up OTEL_EXPORTER_OTLP_ENDPOINT. The simplest way to prove it: temporarily add t.AddConsoleExporter() in the Worker and check whether spans are written when running under Aspire.
  • TUnit-side diagnostic improvement (separate PR): have OtlpReceiver track request counts / unique resource names and emit a warning at session end if EnableTelemetryCollection = true but zero /v1/traces requests were received. That would distinguish "SUT didn't export anything" from "SUT exported but TUnit dropped it on the floor".
  • Existing docs note: the Non-HTTP triggers limitation in docs/examples/aspire.md is still the closest thing we document. Worth expanding with a Functions/ServiceBus worked example once we've validated end-to-end on a reproducer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Multi step test execution with OTEL and Aspire (Gherkin - BDD style)

2 participants