Add Microsoft Testing Platform (MTP) integration for device testing#130
Draft
mattleibow wants to merge 50 commits into
Draft
Add Microsoft Testing Platform (MTP) integration for device testing#130mattleibow wants to merge 50 commits into
mattleibow wants to merge 50 commits into
Conversation
Add DeviceRunners.VisualRunners.Xunit3 project with in-process test discovery and execution using xUnit v3's ExtensibilityPointFactory API. New projects: - src/DeviceRunners.VisualRunners.Xunit3 - Visual runner for xUnit v3 - test/TestProject.Xunit3Tests - xUnit v3 test data project - sample/test/DeviceTestingKitApp.MauiLibrary.Xunit3Tests - Sample tests Key architecture decisions: - Uses xunit.v3.extensibility.core directly for in-process discovery/execution - Test class libraries reference xunit.v3.extensibility.core + xunit.v3.assert (NOT xunit.v3.core which forces OutputType=Exe and injects Main) - PrivateAssets=compile prevents v3 types from conflicting with v2 types - Follows guidance from xunit/xunit#3112 discussion Changes: - Directory.Packages.props: Add xunit.v3.* package versions (3.2.2) - DeviceRunners.slnx: Add new projects - Sample MauiProgram.cs: Add .AddXunit3() alongside existing runners - docs/articles/xunit-v3-support.md: Usage guide - Test base classes: Add virtual TestAssembly property for framework flexibility Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Key fixes: - Await the ValueTask from ITestFrameworkDiscoverer.Find() which runs discovery asynchronously on a ThreadPool thread - Call TestContext.SetForInitialization() before ExtensibilityPointFactory as required by the xUnit v3 framework - Use ExtensibilityPointFactory.GetTestFramework() directly instead of ConsoleRunnerInProcess for simpler in-process discovery/execution - Make test base classes support overridable ExpectedTestCount All 148 tests pass (including 12 new xUnit v3 tests). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Critical fixes: - Await RunTestCases ValueTask (was discarded — exceptions swallowed, potential deadlock if execution fails before ITestAssemblyFinished) - Dispose ITestFramework via await using (implements IAsyncDisposable, was leaking on every discovery/execution call) - Wire Xunit3DiagnosticMessageSink to TestContext.SetForInitialization so xUnit framework errors are forwarded to IDiagnosticsManager Thread safety: - Add lock to Xunit3ExecutionMessageSink.OnMessage() to protect Dictionary mutations from concurrent test execution Error handling: - Handle IErrorMessage, ITestAssemblyCleanupFailure, ITestCollectionCleanupFailure, ITestClassCleanupFailure, ITestCaseCleanupFailure, ITestCleanupFailure in execution sink - Flush remaining pending results on ITestAssemblyFinished for tests that emitted ITestStarting but never ITestFinished - Improve Xunit3DiagnosticMessageSink to also handle IInternalDiagnosticMessage Cleanup: - Delete dead Xunit3DiscoveryMessageSink (not needed with callback-based ITestFrameworkDiscoverer.Find()) - Update docs to match actual implementation (ExtensibilityPointFactory API, not ConsoleRunnerInProcess; note config files not yet supported) - Add dotnet-reportgenerator-globaltool for code coverage - Add artifacts/coverage/ to .gitignore - Fix output capture test to not pass vacuously Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update UITestCase and UITheoryTestCase to use the correct xUnit v3 API: - Use base constructor instead of non-existent Initialize() method - Pass resolved values from TestIntrospectionHelper.GetTestCaseDetails directly to base class constructors - Await CreateTests() which returns ValueTask<IReadOnlyCollection> Update UIFactDiscoverer and UITheoryDiscoverer to call TestIntrospectionHelper.GetTestCaseDetails with the required ITestFrameworkDiscoveryOptions parameter and pass decomposed details to test case constructors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New project: - DeviceRunners.UITesting.Xunit3 — [UIFact] and [UITheory] attributes for xUnit v3 that dispatch test execution to the UI thread via ISelfExecutingXunitTestCase Config loading: - Xunit3TestDiscoverer now loads xunit.runner.json via ConfigReader_Json.LoadFromJson() using FileSystemUtils stream - TestAssemblyConfiguration is passed through to both discovery and execution options - Xunit3TestAssemblyInfo now holds TestAssemblyConfiguration Updates: - DeviceRunners.slnx — added UITesting.Xunit3 - Sample DeviceTests.csproj — added UITesting.Xunit3 reference - Docs — updated limitations, added UI testing section Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The sample DeviceTests project references both xunit v2 and v3 test libraries. Fix type conflicts by: - Remove UITesting.Xunit3 reference from device tests (users choose one or the other based on their xunit version) - Add PrivateAssets=compile to sample Xunit3Tests project's v3 packages to prevent v3 types from flowing transitively into the device tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Enable PreEnumerateTheories in both discoverer and runner so each theory data row becomes its own Xunit3TestCaseInfo (matching xUnit v2). Without this, all theory rows share one test case ID and the last row's result overwrites earlier ones — silently hiding failures (e.g. a theory where row 1 fails but row 3 passes reports Passed). With pre-enumeration: - DataTest(1), DataTest(2), DataTest(3) each get a unique test case - ExpectedTestCount goes back to 8 (matching v2 parity) - Each row's pass/fail is tracked independently Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
UIFactDiscoverer now derives from FactDiscoverer (inherits parameter validation, generic method checks, and trait handling). UITheoryDiscoverer now derives from TheoryDiscoverer (inherits theory data enumeration, serialization checks, no-data error handling, skip semantics, and trait propagation). This matches the v2 pattern where UIFactDiscoverer : FactDiscoverer and UITheoryDiscoverer : TheoryDiscoverer. Added TraitsHelper to replicate the internal ToReadWrite() extension for converting read-only trait dictionaries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# Conflicts: # .gitignore # DeviceRunners.slnx # test/DeviceRunners.VisualRunners.Tests/Testing/TestDiscovererTests.cs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bUnit uses AngleSharp which has TFM compatibility issues (net8.0 vs net10.0). The bUnit tests (CounterComponentClickTests, LoginComponentTests) use a fake Razor renderer, not real component rendering. Remove them and the bUnit package reference. The remaining tests use HtmlRenderer or pure unit testing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use logical assembly name (assembly.GetName().Name + ".dll") instead of FileSystemUtils.GetAssemblyFileName() which returns empty string on WASM (Assembly.Location is empty in WebAssembly). This matches the approach used by the v2 XunitReflectionTestDiscoverer. The xunit v3 API (ExtensibilityPointFactory.GetTestFramework) is already fully reflection-based and works on WASM without special handling. The only issue was our wrapper code using file-path-based assembly matching. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add DeviceTestingKitApp.BlazorLibrary.Xunit3Tests with unit tests for CounterViewModel, CounterValueFormatter, and BlazorSemanticAnnouncer using xunit v3 (mirrors the existing v2 XunitTests). Wire up AddXunit3() in BrowserTests Program.cs alongside AddXunit(). Unlike v2 which needs useReflection: true for WASM, v3 uses ExtensibilityPointFactory which is already reflection-based. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
xunit v3's TestAssemblyRunner.OnTestAssemblyStarting calls Path.GetFileNameWithoutExtension(TestAssembly.AssemblyPath) which crashes on WASM because Assembly.Location returns empty string. Add WasmXunit3TestAssembly that subclasses XunitTestAssembly and re-implements IXunitTestAssembly to return a logical assembly path (assembly.GetName().Name + ".dll") when Assembly.Location is empty. Add WasmXunit3TestFramework that creates WASM-safe assembly objects for both discovery and execution. Update Xunit3TestDiscoverer and Xunit3TestRunner to use the WASM-safe framework automatically when Assembly.Location is empty. WASM BrowserTests results: 58 passed, 5 intentional failures, 3 skipped (previously 24 non-intentional failures from AssemblyName guard). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Create DeviceTestingKitApp.BlazorLibrary.NUnitTests project with sample NUnit tests (UnitTests, CounterViewModelTests, CounterValueFormatterTests) - Wire up NUnit in BrowserTests via AddNUnit() alongside existing xunit - Add VisualRunners.NUnit project reference to BrowserTests - Add new project to DeviceRunners.slnx NUnit's DefaultTestAssemblyBuilder already uses reflection-based discovery (takes Assembly object directly), so no special WASM-specific discoverer is needed — unlike xunit v2 which required XunitReflectionTestDiscoverer. The NUnitTestAssemblyInfoBuilder also short-circuits the runner's Load to avoid filesystem access. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Root causes of NUnit producing 0 tests on WASM: 1. Thread creation: NUnit's SimpleWorkItemDispatcher calls new Thread().Start() which throws PlatformNotSupportedException on single-threaded WASM. Fix: Set RunOnMainThread=true to use MainThreadWorkItemDispatcher which executes inline without spawning threads. 2. EventPump thread: NUnit creates an EventPump thread for async event delivery. Fix: Set SynchronousEvents=true to bypass the EventPump. 3. Assembly path: FileSystemUtils.GetAssemblyFileName used Assembly.Location which returns empty string on WASM. Fix: Fall back to AssemblyName + .dll. 4. Silent failures: Added diagnostic messages when NUnit marks assemblies as not runnable or discovers 0 tests, making debugging much easier. 5. WorkDirectory: NUnit calls Directory.GetCurrentDirectory() internally. Fix: Provide WorkDirectory=. in configuration to short-circuit this. The RunOnMainThread and SynchronousEvents settings are safe for all platforms since the NUnit runner is already called from a background thread via AsyncUtils.RunAsync on MAUI — 'main thread' just means 'calling thread'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This reverts commit 88f0b03.
This reverts commit f65cf50.
Add comprehensive documentation for how xUnit v3 works on WASM/Blazor: - New 'WASM / Blazor Browser Support' section in xunit-v3-support.md explaining the automatic WASM detection, WasmXunit3TestAssembly, WasmXunit3TestFramework, and the interface dispatch remapping pattern - Desktop vs WASM comparison table (assembly location, threading, result output, configuration, [TestFramework] attribute support) - Comparison with xUnit v2 WASM approach (reflection-based vs targeted assembly path workaround) - Updated technical-architecture-overview.md with xunit v3 framework entries, WASM architecture details, and UITesting.Xunit3 - Updated visual-runner-in-the-ide.md Blazor example to show .AddXunit3() Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ndroid/iOS too The empty Assembly.Location problem is not WASM-specific. On Android, assemblies are loaded from APK streams; on iOS, from AOT bundles. Both return empty Assembly.Location just like WASM. Renames: - WasmXunit3TestAssembly → InMemoryXunit3TestAssembly - WasmXunit3TestFramework → InMemoryXunit3TestFramework Updated docs to explain the cross-platform Assembly.Location behavior with a platform comparison table showing which platforms have empty locations and why. Also documents how xunit v2 handled this differently per platform (dummy files on Android, reflection discoverer on WASM) vs xunit v3's unified InMemory approach. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Cache ITestCase in Xunit3TestCaseInfo to avoid re-discovery on every run (previously discarded ITestCase during discovery and re-discovered each time tests were executed, matching v2 which caches ITestCase) - Fix ITestNotRun/ITestSkipped results silently dropped when ITestStarting hasn't fired (added EnsureTestMapping to populate the lookup directly from the message's TestCaseUniqueID) - Use pattern matching for ITestNotRun case (was using explicit cast inconsistent with all other message cases) - Extract duplicated CreateTestFramework into shared static method InMemoryXunit3TestFramework.CreateForAssembly() - Remove unreachable Assembly.Location check in InMemoryXunit3TestFramework (framework is only instantiated when Location is empty) - Remove unused ILogger parameter from Xunit3TestDiscoverer constructor - Use collection expressions in UITheoryDiscoverer - Add upstream issue link for AssemblyPath hack: xunit/xunit#3577 - Run dotnet format on xunit3 code Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes from 4 independent code reviews: Correctness: - Use ExceptionUtility.CombineMessages/CombineStackTraces for proper exception formatting with types, inner exceptions, and parent indices (matches v2) - Add EnsureTestMapping for ITestPassed/ITestFailed (not just ITestSkipped/ITestNotRun) - Fix TraitsHelper: use case-sensitive comparer for trait values (matching xunit v3) - Add try/catch per assembly in runner so one failure doesn't abort remaining - Log diagnostic when assembly lookup fails (was silent skip) - Add TrimmerRootAssembly for Xunit3Tests (prevents trimmed tests on iOS/Android) - Update _ExcludeTestLibraryRuntimeConfigs for Xunit3Tests runtimeconfig.json Safety/Performance: - Move ResultReported/RecordResult callbacks outside lock to prevent deadlocks - Clean up _testFinishedData/_testUniqueIdToTestCaseId after FlushResult - Dispose ITestFrameworkDiscoverer and ITestFrameworkExecutor (were leaking) Cleanup: - Remove dead Finished ManualResetEventSlim (never awaited, IDisposable leak) - Remove dead outer try/catch in DiscoverAsync (inner per-assembly catch covers all) - Remove dead <EmbeddedResource> ItemGroup (no Assets/ folder exists) - Extract Xunit3DiagnosticMessageSink.TryCreate() shared factory method - Revert unrelated changes: decompile.csx, .gitignore, reportgenerator tool Docs: - Fix architecture doc: remove nonexistent APIs (DeviceTest, IDeviceTestApp, etc.) - Fix xunit-v3-support.md: selective execution uses cached ITestCase (not re-discover) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ution, conditional PreEnumerateTheories, v3 UIFact/UITheory samples, and docs Key changes: - Narrow UI dispatch scope in UITestCase/UITheoryTestCase to only InvokeTest (new UIXunitTestRunner and UIXunitTestCaseRunner classes) - Add parallel assembly execution support to Xunit3TestRunner - Make PreEnumerateTheories conditional (only force when not configured) - Move v3 UIFact/UITheory attributes to DeviceRunners.UITesting.Xunit3 namespace to avoid ambiguity with v2 attributes in namespace Xunit - Port v2 sample tests to v3 (CounterView, VisualElement, Converter, Semantic) - Add v3 UIFact/UITheory sample tests in MauiLibrary.Xunit3Tests - Document InMemory extensibility limitation in docs - Document namespace difference for v3 UIFact/UITheory in docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…port - Multi-target both XunitTests and Xunit3Tests with platform TFMs (net10.0, android, ios, maccatalyst, windows) - Add UseMaui, SingleProject, ApplicationId, SupportedOSPlatformVersion - Move UI tests from DeviceTests into both test libraries (duplicated) - Wrap platform-specific tests with #if ANDROID || IOS || MACCATALYST || WINDOWS - Add dotnet test infrastructure conditionally for host net10.0 TFM: - v2: Microsoft.NET.Test.Sdk + xunit.runner.visualstudio - v3: xunit.v3 + OutputType=Exe + TestingPlatformDotnetTestSupport - Use MSBuild GetTargetPlatformIdentifier for reliable TFM conditions - Add global usings (Usings.cs) to both projects - Remove UITests from DeviceTests (now in test libraries) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In xUnit v2, InvokeTestMethodAsync encompassed class creation, IAsyncLifetime, test method invocation, and disposal - dispatching that single method to the UI thread was sufficient. In xUnit v3, the pipeline splits these into CreateTestClassInstance, InvokeTest, and DisposeTestClassInstance. The previous override of InvokeTest only dispatched the test method, causing IAsyncLifetime (e.g. Shell.GoToAsync in InitializeAsync) to run on a worker thread and fail with UIKitThreadAccessException. Fix: override RunTest instead, which orchestrates the full lifecycle. Uses clean async (no GetAwaiter().GetResult() blocking needed) since v3's pipeline properly awaits the returned ValueTask. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The official xUnit v3 [Fact] and [Theory] attributes remain in the Xunit namespace (same as v2). Our v3 UIFact and UITheory should follow the same convention so migrating from v2 to v3 is a simple package swap with no namespace changes needed. Also updates docs to reflect the corrected UI thread dispatch behavior (entire lifecycle, not just InvokeTest). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ocument known limitations Remove SetSynchronousMessageReporting(true) from v3 discoverer and runner - the message sink is already thread-safe with its own lock - matches xunit v3 default (false) Remove forced PreEnumerateTheories=true from v3 discoverer - xunit v3 default is false (same as v2) - users can set to true in xunit.runner.json if they want individual theory data rows in the visual runner Update v3 test expectations to use TestCountNoTheoryEnumeration (6 instead of 8) matching the default behavior. Add comprehensive Known Limitations section to docs covering: - Platform workarounds (Assembly.Location, config loading) - Feature gaps (explicit tests, filter expressions) - Behavioral defaults - Verified xUnit v3 features Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tions Remove false claim that xUnit v3 test libraries cannot use dotnet test. They work on the host TFM (net10.0) - only the device TFMs use the in-process visual runner. Clarify that Known Limitations section applies only to the visual runner, not to dotnet test on the host TFM. Add explanatory comment on the lock in Xunit3ExecutionMessageSink documenting why it exists (IMessageSink threading contract, defensive against parallel test execution, deadlock prevention pattern). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix critical bug: theory rows with PreEnumerateTheories=false now properly aggregate per test case (failing row marks case as failed, not overwritten by subsequent passing rows) - Rewrite Xunit3ExecutionMessageSink to flush on ITestCaseFinished instead of ITestFinished, with proper per-test-case aggregation - Move diagnostic calls outside the lock to prevent potential deadlocks - Fix stale XML docs on UITestCase, UITheoryTestCase, UIXunitTestCaseRunner that still described the old broken dispatch-only-invocation behavior - Add IAsyncLifetime regression test to TestProject.Xunit3Tests - Update docs: Quick Start shows multi-target csproj with dotnet test setup, verified features split into tested vs expected-to-work categories Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add feedz.io xunit CI feed to nuget.config - Add UseXunitV3CI MSBuild property to switch to 4.0.0-pre.117 - Add XUNIT_V3_CI define constant for conditional compilation - Handle API changes in 4.0 (FixtureMappingManager param, non-virtual CreateTestCasesForDataRow, obsolete constructors) - Builds and tests pass on both 3.2.2 and 4.0.0-pre.117 Usage: dotnet build /p:UseXunitV3CI=true See https://xunit.net/docs/using-ci-builds Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Rename src/DeviceRunners.Testing.Targets/ to src/DeviceRunners.VisualRunners.Targets/ - Update PackageId, build file names, solution reference, and sample imports - Add Microsoft.Testing.Platform v2.2.1, Microsoft.Testing.Extensions.TrxReport, and Mono.Options to Directory.Packages.props for the new MTP integration Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revert the conditional xunit 4.0 prerelease support. The changes needed for the xunit.v3 package version 4.0.0 are tracked in #129 and will be applied when the version goes stable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds initial skeleton for MTP integration: - DeviceRunners.Testing.Platform.Targets project (MSBuild targets) - Microsoft.Testing.Platform package reference in CLI project - Solution file updated This is a work-in-progress, will be refined after merging xunit-v3-support. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…o mattleibow/microsoft-testing-platform
Implement the core MTP integration for running tests on devices via dotnet test. This is additive — existing visual runners are untouched. New packages: - DeviceRunners.Testing.Platform: Core package with IDataConsumer implementations (TcpStreamingConsumer, ConsoleStreamingConsumer), builder interfaces, and configuration extensions - DeviceRunners.Testing.Platform.Maui: MAUI host integration via UseTestingPlatformRunner(MauiAppBuilder) and MtpTestRunnerService - DeviceRunners.Testing.Platform.Blazor: Blazor/WASM host integration via UseTestingPlatformRunner(WebAssemblyHostBuilder) - DeviceRunners.Testing.Platform.Xunit3: xunit v3 framework via TestPlatformTestFramework.RunAsync (desktop) or custom DeviceXunitTestFramework (mobile) - DeviceRunners.Testing.Platform.MSTest: MSTest framework via builder.AddMSTest(() => assemblies) - DeviceRunners.Testing.Platform.NUnit: NUnit framework via builder.AddNUnit(() => assemblies) Wire protocol extended with uid/parentUid fields and new event types (discovered, inprogress) for MTP compatibility. All packages build clean, 149 existing tests pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When dotnet test launches the CLI with --server, the CLI enters MTP host mode. DeviceTestFramework implements ITestFramework on the host side: - Starts a TCP listener for NDJSON events from the device app - Maps TestResultEvent → TestNodeUpdateMessage for MTP protocol - Handles connection/data timeouts with crash detection - Calls context.Complete() to signal completion to dotnet test Files: - src/DeviceRunners.Cli/Mtp/MtpHost.cs: Entry point for MTP mode - src/DeviceRunners.Cli/Mtp/DeviceTestFramework.cs: ITestFramework impl - src/DeviceRunners.Cli/Mtp/DeviceTestFrameworkArgs.cs: CLI arg parser - src/DeviceRunners.Cli/Mtp/DeviceTestFrameworkCapabilities.cs: Capabilities - src/DeviceRunners.Cli/Program.cs: --server detection before Spectre Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The targets package was referencing DeviceRunners.Testing.Platform as the tool binary, but the actual CLI tool is DeviceRunners.Cli. Updated: - DeviceRunners.Testing.Platform.Targets.csproj: project path - .targets: binary name (DeviceRunners.Cli / DeviceRunners.Cli.exe) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Demonstrates the UseTestingPlatformRunner API with xunit v3 tests running on Android, iOS, macCatalyst, and Windows via Microsoft Testing Platform. Key setup: - References xunit.v3.extensibility.core + xunit.v3.assert (avoids xunit.v3.core.mtp-v1 build targets that conflict with MAUI) - Uses GenerateTestingPlatformEntryPoint=false (MAUI owns main) - Imports Testing.Platform.Targets .props/.targets for dotnet test - Tests live in the app project (single-assembly model) Also fixes XML comment syntax in .targets file (-- invalid in XML comments). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Demonstrates the UseTestingPlatformRunner API with xunit v3 tests running in a Blazor WebAssembly app via Microsoft Testing Platform. Uses console streaming (captured via CDP by the host CLI) instead of TCP for the browser environment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove all xunit v3 visual runner and UITesting.Xunit3 code from this branch - that will come in a separate PR. This branch now contains only Microsoft Testing Platform (MTP) integration. Also adds comprehensive test suites to MTP sample apps matching the tests in the visual runner samples. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Microsoft.Testing.Platform sets IsTestingPlatformApplication=true by default for any project that references it. NuGet pack targets then set IsPackable=false for such projects. The CLI is a host tool, not a test application, so it needs to override this. Without this fix, 'dotnet pack' silently produces no nupkg, causing all CI test jobs that install the tool to fail. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- DeviceRunners.Testing.Platform: set IsTestingPlatformApplication=false to prevent NuGet from making it non-packable - DeviceRunners.Testing.Platform.Targets: set IsPackable=false (expensive multi-RID publish, pack explicitly when releasing) and add condition to PublishMtpTool target to prevent running when IsPackable=false Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Microsoft Testing Platform (MTP) v2 support to DeviceRunners, enabling
dotnet testto work with device-targeted MAUI/Blazor test projects across Android, iOS, macCatalyst, Windows, and WASM.This is purely additive — existing visual runners are completely untouched.
Architecture
New Packages
DeviceRunners.Testing.PlatformDeviceRunners.Testing.Platform.MauiUseTestingPlatformRunner(MauiAppBuilder)integrationDeviceRunners.Testing.Platform.BlazorUseTestingPlatformRunner(WebAssemblyHostBuilder)integrationDeviceRunners.Testing.Platform.Xunit3AddXunit3()— registers xunit v3 as test frameworkDeviceRunners.Testing.Platform.MSTestAddMSTest()— registers MSTestDeviceRunners.Testing.Platform.NUnitAddNUnit()— registers NUnitDeviceRunners.Testing.Platform.Targetsdotnet testwiringConsumer API (what users write)
MAUI app
Blazor WASM app
Key Design Decisions
UseTestingPlatformRunnerpath--serverflag branches to MTP host modecontext.Complete()always called — host-side calls it in finally block (mandatory or dotnet test hangs)Commits
Build Status