Skip to content

Add Microsoft Testing Platform (MTP) integration for device testing#130

Draft
mattleibow wants to merge 50 commits into
mainfrom
mattleibow/microsoft-testing-platform
Draft

Add Microsoft Testing Platform (MTP) integration for device testing#130
mattleibow wants to merge 50 commits into
mainfrom
mattleibow/microsoft-testing-platform

Conversation

@mattleibow
Copy link
Copy Markdown
Owner

Summary

Adds Microsoft Testing Platform (MTP) v2 support to DeviceRunners, enabling dotnet test to work with device-targeted MAUI/Blazor test projects across Android, iOS, macCatalyst, Windows, and WASM.

This is purely additive — existing visual runners are completely untouched.

Architecture

dotnet test → MSBuild → RunCommand (DeviceRunners CLI in MTP host mode)
                              ↓
                    Deploy + Launch app on device
                              ↓
              App starts → MTP discovers/runs tests
                              ↓
              TcpStreamingConsumer → NDJSON over TCP (native)
              ConsoleStreamingConsumer → console.log via CDP (WASM)
                              ↓
              CLI receives events → maps to TestNodeUpdateMessage
                              ↓
              dotnet test shows results in real-time

New Packages

Package Purpose
DeviceRunners.Testing.Platform Core: builder interface, TCP/console streaming consumers, TestNodeMapper
DeviceRunners.Testing.Platform.Maui UseTestingPlatformRunner(MauiAppBuilder) integration
DeviceRunners.Testing.Platform.Blazor UseTestingPlatformRunner(WebAssemblyHostBuilder) integration
DeviceRunners.Testing.Platform.Xunit3 AddXunit3() — registers xunit v3 as test framework
DeviceRunners.Testing.Platform.MSTest AddMSTest() — registers MSTest
DeviceRunners.Testing.Platform.NUnit AddNUnit() — registers NUnit
DeviceRunners.Testing.Platform.Targets MSBuild .props/.targets for dotnet test wiring

Consumer API (what users write)

MAUI app

var builder = MauiApp.CreateBuilder();
builder.UseMauiApp<App>();
builder.UseTestingPlatformRunner(mtp =>
{
    mtp.AddXunit3();
    mtp.AddCliConfiguration();
});
return builder.Build();

Blazor WASM app

var builder = WebAssemblyHostBuilder.CreateDefault(args);
builder.UseTestingPlatformRunner(mtp =>
{
    mtp.AddXunit3();
    mtp.AddCliConfiguration();
});
await builder.Build().RunAsync();

Key Design Decisions

  • Single-assembly model — tests live in the app project (matches MTP's model, enables future AOT)
  • Two transports, same NDJSON format — TCP for native, console.log+CDP for WASM
  • Independent from visual runner — separate UseTestingPlatformRunner path
  • Same CLI binary--server flag branches to MTP host mode
  • context.Complete() always called — host-side calls it in finally block (mandatory or dotnet test hangs)

Commits

  1. Device-side packages (core, MAUI, Blazor, xunit3, MSTest, NUnit)
  2. Host-side MTP bridge in CLI (ITestFramework implementation)
  3. MSBuild targets fix (correct binary name)
  4. MAUI sample app (MtpDeviceTests)
  5. Blazor WASM sample app (MtpBrowserTests)

Build Status

  • ✅ Full solution builds (0 errors, 0 warnings)
  • ✅ All 149 existing tests pass
  • ✅ MTP sample builds for Android, iOS, macCatalyst
  • ✅ MTP Blazor sample builds for WASM

mattleibow and others added 30 commits May 8, 2026 23:50
Add DeviceRunners.VisualRunners.Xunit3 project with in-process test
discovery and execution using xUnit v3's ExtensibilityPointFactory API.

New projects:
- src/DeviceRunners.VisualRunners.Xunit3 - Visual runner for xUnit v3
- test/TestProject.Xunit3Tests - xUnit v3 test data project
- sample/test/DeviceTestingKitApp.MauiLibrary.Xunit3Tests - Sample tests

Key architecture decisions:
- Uses xunit.v3.extensibility.core directly for in-process discovery/execution
- Test class libraries reference xunit.v3.extensibility.core + xunit.v3.assert
  (NOT xunit.v3.core which forces OutputType=Exe and injects Main)
- PrivateAssets=compile prevents v3 types from conflicting with v2 types
- Follows guidance from xunit/xunit#3112 discussion

Changes:
- Directory.Packages.props: Add xunit.v3.* package versions (3.2.2)
- DeviceRunners.slnx: Add new projects
- Sample MauiProgram.cs: Add .AddXunit3() alongside existing runners
- docs/articles/xunit-v3-support.md: Usage guide
- Test base classes: Add virtual TestAssembly property for framework flexibility

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Key fixes:
- Await the ValueTask from ITestFrameworkDiscoverer.Find() which runs
  discovery asynchronously on a ThreadPool thread
- Call TestContext.SetForInitialization() before ExtensibilityPointFactory
  as required by the xUnit v3 framework
- Use ExtensibilityPointFactory.GetTestFramework() directly instead of
  ConsoleRunnerInProcess for simpler in-process discovery/execution
- Make test base classes support overridable ExpectedTestCount

All 148 tests pass (including 12 new xUnit v3 tests).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Critical fixes:
- Await RunTestCases ValueTask (was discarded — exceptions swallowed,
  potential deadlock if execution fails before ITestAssemblyFinished)
- Dispose ITestFramework via await using (implements IAsyncDisposable,
  was leaking on every discovery/execution call)
- Wire Xunit3DiagnosticMessageSink to TestContext.SetForInitialization
  so xUnit framework errors are forwarded to IDiagnosticsManager

Thread safety:
- Add lock to Xunit3ExecutionMessageSink.OnMessage() to protect
  Dictionary mutations from concurrent test execution

Error handling:
- Handle IErrorMessage, ITestAssemblyCleanupFailure,
  ITestCollectionCleanupFailure, ITestClassCleanupFailure,
  ITestCaseCleanupFailure, ITestCleanupFailure in execution sink
- Flush remaining pending results on ITestAssemblyFinished for
  tests that emitted ITestStarting but never ITestFinished
- Improve Xunit3DiagnosticMessageSink to also handle
  IInternalDiagnosticMessage

Cleanup:
- Delete dead Xunit3DiscoveryMessageSink (not needed with
  callback-based ITestFrameworkDiscoverer.Find())
- Update docs to match actual implementation (ExtensibilityPointFactory
  API, not ConsoleRunnerInProcess; note config files not yet supported)
- Add dotnet-reportgenerator-globaltool for code coverage
- Add artifacts/coverage/ to .gitignore
- Fix output capture test to not pass vacuously

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update UITestCase and UITheoryTestCase to use the correct xUnit v3 API:
- Use base constructor instead of non-existent Initialize() method
- Pass resolved values from TestIntrospectionHelper.GetTestCaseDetails
  directly to base class constructors
- Await CreateTests() which returns ValueTask<IReadOnlyCollection>

Update UIFactDiscoverer and UITheoryDiscoverer to call
TestIntrospectionHelper.GetTestCaseDetails with the required
ITestFrameworkDiscoveryOptions parameter and pass decomposed details
to test case constructors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New project:
- DeviceRunners.UITesting.Xunit3 — [UIFact] and [UITheory] attributes
  for xUnit v3 that dispatch test execution to the UI thread via
  ISelfExecutingXunitTestCase

Config loading:
- Xunit3TestDiscoverer now loads xunit.runner.json via
  ConfigReader_Json.LoadFromJson() using FileSystemUtils stream
- TestAssemblyConfiguration is passed through to both discovery and
  execution options
- Xunit3TestAssemblyInfo now holds TestAssemblyConfiguration

Updates:
- DeviceRunners.slnx — added UITesting.Xunit3
- Sample DeviceTests.csproj — added UITesting.Xunit3 reference
- Docs — updated limitations, added UI testing section

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The sample DeviceTests project references both xunit v2 and v3 test
libraries. Fix type conflicts by:
- Remove UITesting.Xunit3 reference from device tests (users choose
  one or the other based on their xunit version)
- Add PrivateAssets=compile to sample Xunit3Tests project's v3 packages
  to prevent v3 types from flowing transitively into the device tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Enable PreEnumerateTheories in both discoverer and runner so each
theory data row becomes its own Xunit3TestCaseInfo (matching xUnit v2).

Without this, all theory rows share one test case ID and the last
row's result overwrites earlier ones — silently hiding failures
(e.g. a theory where row 1 fails but row 3 passes reports Passed).

With pre-enumeration:
- DataTest(1), DataTest(2), DataTest(3) each get a unique test case
- ExpectedTestCount goes back to 8 (matching v2 parity)
- Each row's pass/fail is tracked independently

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
UIFactDiscoverer now derives from FactDiscoverer (inherits parameter
validation, generic method checks, and trait handling).

UITheoryDiscoverer now derives from TheoryDiscoverer (inherits theory
data enumeration, serialization checks, no-data error handling, skip
semantics, and trait propagation).

This matches the v2 pattern where UIFactDiscoverer : FactDiscoverer
and UITheoryDiscoverer : TheoryDiscoverer.

Added TraitsHelper to replicate the internal ToReadWrite() extension
for converting read-only trait dictionaries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# Conflicts:
#	.gitignore
#	DeviceRunners.slnx
#	test/DeviceRunners.VisualRunners.Tests/Testing/TestDiscovererTests.cs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bUnit uses AngleSharp which has TFM compatibility issues (net8.0 vs net10.0).
The bUnit tests (CounterComponentClickTests, LoginComponentTests) use a
fake Razor renderer, not real component rendering. Remove them and the bUnit
package reference. The remaining tests use HtmlRenderer or pure unit testing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use logical assembly name (assembly.GetName().Name + ".dll") instead of
FileSystemUtils.GetAssemblyFileName() which returns empty string on WASM
(Assembly.Location is empty in WebAssembly). This matches the approach
used by the v2 XunitReflectionTestDiscoverer.

The xunit v3 API (ExtensibilityPointFactory.GetTestFramework) is already
fully reflection-based and works on WASM without special handling. The
only issue was our wrapper code using file-path-based assembly matching.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add DeviceTestingKitApp.BlazorLibrary.Xunit3Tests with unit tests for
CounterViewModel, CounterValueFormatter, and BlazorSemanticAnnouncer
using xunit v3 (mirrors the existing v2 XunitTests).

Wire up AddXunit3() in BrowserTests Program.cs alongside AddXunit().
Unlike v2 which needs useReflection: true for WASM, v3 uses
ExtensibilityPointFactory which is already reflection-based.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
xunit v3's TestAssemblyRunner.OnTestAssemblyStarting calls
Path.GetFileNameWithoutExtension(TestAssembly.AssemblyPath) which
crashes on WASM because Assembly.Location returns empty string.

Add WasmXunit3TestAssembly that subclasses XunitTestAssembly and
re-implements IXunitTestAssembly to return a logical assembly path
(assembly.GetName().Name + ".dll") when Assembly.Location is empty.

Add WasmXunit3TestFramework that creates WASM-safe assembly objects
for both discovery and execution.

Update Xunit3TestDiscoverer and Xunit3TestRunner to use the WASM-safe
framework automatically when Assembly.Location is empty.

WASM BrowserTests results: 58 passed, 5 intentional failures, 3 skipped
(previously 24 non-intentional failures from AssemblyName guard).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Create DeviceTestingKitApp.BlazorLibrary.NUnitTests project with sample
  NUnit tests (UnitTests, CounterViewModelTests, CounterValueFormatterTests)
- Wire up NUnit in BrowserTests via AddNUnit() alongside existing xunit
- Add VisualRunners.NUnit project reference to BrowserTests
- Add new project to DeviceRunners.slnx

NUnit's DefaultTestAssemblyBuilder already uses reflection-based discovery
(takes Assembly object directly), so no special WASM-specific discoverer is
needed — unlike xunit v2 which required XunitReflectionTestDiscoverer.
The NUnitTestAssemblyInfoBuilder also short-circuits the runner's Load to
avoid filesystem access.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Root causes of NUnit producing 0 tests on WASM:

1. Thread creation: NUnit's SimpleWorkItemDispatcher calls new Thread().Start()
   which throws PlatformNotSupportedException on single-threaded WASM.
   Fix: Set RunOnMainThread=true to use MainThreadWorkItemDispatcher which
   executes inline without spawning threads.

2. EventPump thread: NUnit creates an EventPump thread for async event delivery.
   Fix: Set SynchronousEvents=true to bypass the EventPump.

3. Assembly path: FileSystemUtils.GetAssemblyFileName used Assembly.Location
   which returns empty string on WASM. Fix: Fall back to AssemblyName + .dll.

4. Silent failures: Added diagnostic messages when NUnit marks assemblies as
   not runnable or discovers 0 tests, making debugging much easier.

5. WorkDirectory: NUnit calls Directory.GetCurrentDirectory() internally.
   Fix: Provide WorkDirectory=. in configuration to short-circuit this.

The RunOnMainThread and SynchronousEvents settings are safe for all platforms
since the NUnit runner is already called from a background thread via
AsyncUtils.RunAsync on MAUI — 'main thread' just means 'calling thread'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add comprehensive documentation for how xUnit v3 works on WASM/Blazor:
- New 'WASM / Blazor Browser Support' section in xunit-v3-support.md
  explaining the automatic WASM detection, WasmXunit3TestAssembly,
  WasmXunit3TestFramework, and the interface dispatch remapping pattern
- Desktop vs WASM comparison table (assembly location, threading, result
  output, configuration, [TestFramework] attribute support)
- Comparison with xUnit v2 WASM approach (reflection-based vs targeted
  assembly path workaround)
- Updated technical-architecture-overview.md with xunit v3 framework
  entries, WASM architecture details, and UITesting.Xunit3
- Updated visual-runner-in-the-ide.md Blazor example to show .AddXunit3()

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ndroid/iOS too

The empty Assembly.Location problem is not WASM-specific. On Android,
assemblies are loaded from APK streams; on iOS, from AOT bundles. Both
return empty Assembly.Location just like WASM.

Renames:
- WasmXunit3TestAssembly → InMemoryXunit3TestAssembly
- WasmXunit3TestFramework → InMemoryXunit3TestFramework

Updated docs to explain the cross-platform Assembly.Location behavior
with a platform comparison table showing which platforms have empty
locations and why. Also documents how xunit v2 handled this differently
per platform (dummy files on Android, reflection discoverer on WASM)
vs xunit v3's unified InMemory approach.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Cache ITestCase in Xunit3TestCaseInfo to avoid re-discovery on every
  run (previously discarded ITestCase during discovery and re-discovered
  each time tests were executed, matching v2 which caches ITestCase)
- Fix ITestNotRun/ITestSkipped results silently dropped when ITestStarting
  hasn't fired (added EnsureTestMapping to populate the lookup directly
  from the message's TestCaseUniqueID)
- Use pattern matching for ITestNotRun case (was using explicit cast
  inconsistent with all other message cases)
- Extract duplicated CreateTestFramework into shared static method
  InMemoryXunit3TestFramework.CreateForAssembly()
- Remove unreachable Assembly.Location check in InMemoryXunit3TestFramework
  (framework is only instantiated when Location is empty)
- Remove unused ILogger parameter from Xunit3TestDiscoverer constructor
- Use collection expressions in UITheoryDiscoverer
- Add upstream issue link for AssemblyPath hack: xunit/xunit#3577
- Run dotnet format on xunit3 code

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes from 4 independent code reviews:

Correctness:
- Use ExceptionUtility.CombineMessages/CombineStackTraces for proper exception
  formatting with types, inner exceptions, and parent indices (matches v2)
- Add EnsureTestMapping for ITestPassed/ITestFailed (not just ITestSkipped/ITestNotRun)
- Fix TraitsHelper: use case-sensitive comparer for trait values (matching xunit v3)
- Add try/catch per assembly in runner so one failure doesn't abort remaining
- Log diagnostic when assembly lookup fails (was silent skip)
- Add TrimmerRootAssembly for Xunit3Tests (prevents trimmed tests on iOS/Android)
- Update _ExcludeTestLibraryRuntimeConfigs for Xunit3Tests runtimeconfig.json

Safety/Performance:
- Move ResultReported/RecordResult callbacks outside lock to prevent deadlocks
- Clean up _testFinishedData/_testUniqueIdToTestCaseId after FlushResult
- Dispose ITestFrameworkDiscoverer and ITestFrameworkExecutor (were leaking)

Cleanup:
- Remove dead Finished ManualResetEventSlim (never awaited, IDisposable leak)
- Remove dead outer try/catch in DiscoverAsync (inner per-assembly catch covers all)
- Remove dead <EmbeddedResource> ItemGroup (no Assets/ folder exists)
- Extract Xunit3DiagnosticMessageSink.TryCreate() shared factory method
- Revert unrelated changes: decompile.csx, .gitignore, reportgenerator tool

Docs:
- Fix architecture doc: remove nonexistent APIs (DeviceTest, IDeviceTestApp, etc.)
- Fix xunit-v3-support.md: selective execution uses cached ITestCase (not re-discover)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
mattleibow and others added 20 commits May 17, 2026 19:12
…ution, conditional PreEnumerateTheories, v3 UIFact/UITheory samples, and docs

Key changes:
- Narrow UI dispatch scope in UITestCase/UITheoryTestCase to only InvokeTest
  (new UIXunitTestRunner and UIXunitTestCaseRunner classes)
- Add parallel assembly execution support to Xunit3TestRunner
- Make PreEnumerateTheories conditional (only force when not configured)
- Move v3 UIFact/UITheory attributes to DeviceRunners.UITesting.Xunit3 namespace
  to avoid ambiguity with v2 attributes in namespace Xunit
- Port v2 sample tests to v3 (CounterView, VisualElement, Converter, Semantic)
- Add v3 UIFact/UITheory sample tests in MauiLibrary.Xunit3Tests
- Document InMemory extensibility limitation in docs
- Document namespace difference for v3 UIFact/UITheory in docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…port

- Multi-target both XunitTests and Xunit3Tests with platform TFMs
  (net10.0, android, ios, maccatalyst, windows)
- Add UseMaui, SingleProject, ApplicationId, SupportedOSPlatformVersion
- Move UI tests from DeviceTests into both test libraries (duplicated)
- Wrap platform-specific tests with #if ANDROID || IOS || MACCATALYST || WINDOWS
- Add dotnet test infrastructure conditionally for host net10.0 TFM:
  - v2: Microsoft.NET.Test.Sdk + xunit.runner.visualstudio
  - v3: xunit.v3 + OutputType=Exe + TestingPlatformDotnetTestSupport
- Use MSBuild GetTargetPlatformIdentifier for reliable TFM conditions
- Add global usings (Usings.cs) to both projects
- Remove UITests from DeviceTests (now in test libraries)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In xUnit v2, InvokeTestMethodAsync encompassed class creation,
IAsyncLifetime, test method invocation, and disposal - dispatching
that single method to the UI thread was sufficient.

In xUnit v3, the pipeline splits these into CreateTestClassInstance,
InvokeTest, and DisposeTestClassInstance. The previous override of
InvokeTest only dispatched the test method, causing IAsyncLifetime
(e.g. Shell.GoToAsync in InitializeAsync) to run on a worker thread
and fail with UIKitThreadAccessException.

Fix: override RunTest instead, which orchestrates the full lifecycle.
Uses clean async (no GetAwaiter().GetResult() blocking needed) since
v3's pipeline properly awaits the returned ValueTask.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The official xUnit v3 [Fact] and [Theory] attributes remain in the
Xunit namespace (same as v2). Our v3 UIFact and UITheory should
follow the same convention so migrating from v2 to v3 is a simple
package swap with no namespace changes needed.

Also updates docs to reflect the corrected UI thread dispatch
behavior (entire lifecycle, not just InvokeTest).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ocument known limitations

Remove SetSynchronousMessageReporting(true) from v3 discoverer and runner
- the message sink is already thread-safe with its own lock
- matches xunit v3 default (false)

Remove forced PreEnumerateTheories=true from v3 discoverer
- xunit v3 default is false (same as v2)
- users can set to true in xunit.runner.json if they want
  individual theory data rows in the visual runner

Update v3 test expectations to use TestCountNoTheoryEnumeration
(6 instead of 8) matching the default behavior.

Add comprehensive Known Limitations section to docs covering:
- Platform workarounds (Assembly.Location, config loading)
- Feature gaps (explicit tests, filter expressions)
- Behavioral defaults
- Verified xUnit v3 features

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tions

Remove false claim that xUnit v3 test libraries cannot use dotnet test.
They work on the host TFM (net10.0) - only the device TFMs use the
in-process visual runner.

Clarify that Known Limitations section applies only to the visual runner,
not to dotnet test on the host TFM.

Add explanatory comment on the lock in Xunit3ExecutionMessageSink
documenting why it exists (IMessageSink threading contract, defensive
against parallel test execution, deadlock prevention pattern).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix critical bug: theory rows with PreEnumerateTheories=false now properly
  aggregate per test case (failing row marks case as failed, not overwritten
  by subsequent passing rows)
- Rewrite Xunit3ExecutionMessageSink to flush on ITestCaseFinished instead
  of ITestFinished, with proper per-test-case aggregation
- Move diagnostic calls outside the lock to prevent potential deadlocks
- Fix stale XML docs on UITestCase, UITheoryTestCase, UIXunitTestCaseRunner
  that still described the old broken dispatch-only-invocation behavior
- Add IAsyncLifetime regression test to TestProject.Xunit3Tests
- Update docs: Quick Start shows multi-target csproj with dotnet test setup,
  verified features split into tested vs expected-to-work categories

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add feedz.io xunit CI feed to nuget.config
- Add UseXunitV3CI MSBuild property to switch to 4.0.0-pre.117
- Add XUNIT_V3_CI define constant for conditional compilation
- Handle API changes in 4.0 (FixtureMappingManager param, non-virtual
  CreateTestCasesForDataRow, obsolete constructors)
- Builds and tests pass on both 3.2.2 and 4.0.0-pre.117

Usage: dotnet build /p:UseXunitV3CI=true

See https://xunit.net/docs/using-ci-builds

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Rename src/DeviceRunners.Testing.Targets/ to src/DeviceRunners.VisualRunners.Targets/
- Update PackageId, build file names, solution reference, and sample imports
- Add Microsoft.Testing.Platform v2.2.1, Microsoft.Testing.Extensions.TrxReport,
  and Mono.Options to Directory.Packages.props for the new MTP integration

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revert the conditional xunit 4.0 prerelease support. The changes needed
for the xunit.v3 package version 4.0.0 are tracked in #129 and will be
applied when the version goes stable.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds initial skeleton for MTP integration:
- DeviceRunners.Testing.Platform.Targets project (MSBuild targets)
- Microsoft.Testing.Platform package reference in CLI project
- Solution file updated

This is a work-in-progress, will be refined after merging xunit-v3-support.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implement the core MTP integration for running tests on devices via
dotnet test. This is additive — existing visual runners are untouched.

New packages:
- DeviceRunners.Testing.Platform: Core package with IDataConsumer
  implementations (TcpStreamingConsumer, ConsoleStreamingConsumer),
  builder interfaces, and configuration extensions
- DeviceRunners.Testing.Platform.Maui: MAUI host integration via
  UseTestingPlatformRunner(MauiAppBuilder) and MtpTestRunnerService
- DeviceRunners.Testing.Platform.Blazor: Blazor/WASM host integration
  via UseTestingPlatformRunner(WebAssemblyHostBuilder)
- DeviceRunners.Testing.Platform.Xunit3: xunit v3 framework via
  TestPlatformTestFramework.RunAsync (desktop) or custom
  DeviceXunitTestFramework (mobile)
- DeviceRunners.Testing.Platform.MSTest: MSTest framework via
  builder.AddMSTest(() => assemblies)
- DeviceRunners.Testing.Platform.NUnit: NUnit framework via
  builder.AddNUnit(() => assemblies)

Wire protocol extended with uid/parentUid fields and new event types
(discovered, inprogress) for MTP compatibility.

All packages build clean, 149 existing tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When dotnet test launches the CLI with --server, the CLI enters MTP host
mode. DeviceTestFramework implements ITestFramework on the host side:
- Starts a TCP listener for NDJSON events from the device app
- Maps TestResultEvent → TestNodeUpdateMessage for MTP protocol
- Handles connection/data timeouts with crash detection
- Calls context.Complete() to signal completion to dotnet test

Files:
- src/DeviceRunners.Cli/Mtp/MtpHost.cs: Entry point for MTP mode
- src/DeviceRunners.Cli/Mtp/DeviceTestFramework.cs: ITestFramework impl
- src/DeviceRunners.Cli/Mtp/DeviceTestFrameworkArgs.cs: CLI arg parser
- src/DeviceRunners.Cli/Mtp/DeviceTestFrameworkCapabilities.cs: Capabilities
- src/DeviceRunners.Cli/Program.cs: --server detection before Spectre

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The targets package was referencing DeviceRunners.Testing.Platform as the
tool binary, but the actual CLI tool is DeviceRunners.Cli. Updated:
- DeviceRunners.Testing.Platform.Targets.csproj: project path
- .targets: binary name (DeviceRunners.Cli / DeviceRunners.Cli.exe)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Demonstrates the UseTestingPlatformRunner API with xunit v3 tests
running on Android, iOS, macCatalyst, and Windows via Microsoft
Testing Platform.

Key setup:
- References xunit.v3.extensibility.core + xunit.v3.assert (avoids
  xunit.v3.core.mtp-v1 build targets that conflict with MAUI)
- Uses GenerateTestingPlatformEntryPoint=false (MAUI owns main)
- Imports Testing.Platform.Targets .props/.targets for dotnet test
- Tests live in the app project (single-assembly model)

Also fixes XML comment syntax in .targets file (-- invalid in XML
comments).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Demonstrates the UseTestingPlatformRunner API with xunit v3 tests
running in a Blazor WebAssembly app via Microsoft Testing Platform.

Uses console streaming (captured via CDP by the host CLI) instead
of TCP for the browser environment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove all xunit v3 visual runner and UITesting.Xunit3 code from
this branch - that will come in a separate PR. This branch now
contains only Microsoft Testing Platform (MTP) integration.

Also adds comprehensive test suites to MTP sample apps matching
the tests in the visual runner samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Microsoft.Testing.Platform sets IsTestingPlatformApplication=true by default
for any project that references it. NuGet pack targets then set IsPackable=false
for such projects. The CLI is a host tool, not a test application, so it needs
to override this. Without this fix, 'dotnet pack' silently produces no nupkg,
causing all CI test jobs that install the tool to fail.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- DeviceRunners.Testing.Platform: set IsTestingPlatformApplication=false
  to prevent NuGet from making it non-packable
- DeviceRunners.Testing.Platform.Targets: set IsPackable=false (expensive
  multi-RID publish, pack explicitly when releasing) and add condition to
  PublishMtpTool target to prevent running when IsPackable=false

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant