Microsoft.Testing.Platform overview - .NET (original) (raw)

Microsoft.Testing.Platform is a lightweight and portable alternative to VSTest for running tests in all contexts, including continuous integration (CI) pipelines, CLI, Visual Studio Test Explorer, and VS Code Test Explorer. The Microsoft.Testing.Platform is embedded directly in your test projects, and there's no other app dependencies, such as vstest.console or dotnet test needed to run your tests.

Microsoft.Testing.Platform is open source. You can find Microsoft.Testing.Platform code in microsoft/testfx GitHub repository.

Microsoft.Testing.Platform pillars

This new testing platform is built on the .NET Developer Experience Testing team's experience and aims to address the challenges encountered since the release of .NET Core in 2016. While there's a high level of compatibility between the .NET Framework and the .NET Core/.NET, some key features like the plugin-system and the new possible form factors of .NET compilations have made it complex to evolve or fully support the new runtime feature with the current VSTest platform architecture.

The main driving factors for the evolution of the new testing platform are detailed in the following:

Supported test frameworks

Run and debug tests

Microsoft.Testing.Platform test projects are built as executables that can be run (or debugged) directly. There's no extra test running console or command. The app exits with a nonzero exit code if there's an error, which is typical for most executables. For more information on the known exit codes, see Microsoft.Testing.Platform exit codes.

Tip

You can ignore a specific exit code using the --ignore-exit-code command line option.

You can also set command line options that apply to a specific test project in the project file using the TestingPlatformCommandLineArguments MSBuild property. One common use case is for test projects that have all the tests ignored, which will normally exit with exit code 8 (the test session ran zero tests). In this scenario, you can add the following under a PropertyGroup in your project file:

<TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>

Publishing the test project using dotnet publish and running the app directly is another way to run your tests. For example, executing the ./Contoso.MyTests.exe. In some scenarios it's also viable to use dotnet build to produce the executable, but there can be edge cases to consider, such Native AOT.

Use dotnet run

The dotnet run command can be used to build and run your test project. This is the easiest, although sometimes slowest, way to run your tests. Using dotnet run is practical when you're editing and running tests locally, because it ensures that the test project is rebuilt when needed. dotnet run will also automatically find the project in the current folder.

dotnet run --project Contoso.MyTests

For more information on dotnet run, see dotnet run.

Use dotnet exec

The dotnet exec or dotnet command is used to execute (or run) an already built test project, this is an alternative to running the application directly. dotnet exec requires path to the built test project dll.

dotnet exec Contoso.MyTests.dll

or

dotnet Contoso.MyTests.dll

Note

Providing the path to the test project executable (*.exe) results in an error:

Error:
  An assembly specified in the application dependencies manifest
  (Contoso.MyTests.deps.json) has already been found but with a different
  file extension:
    package: 'Contoso.MyTests', version: '1.0.0'
    path: 'Contoso.MyTests.dll'
    previously found assembly: 'S:\t\Contoso.MyTests\bin\Debug\net8.0\Contoso.MyTests.exe'

For more information on dotnet exec, see dotnet exec.

Use dotnet test

Microsoft.Testing.Platform offers a compatibility layer with vstest.console.exe and dotnet test ensuring you can run your tests as before while enabling new execution scenario.

dotnet test Contoso.MyTests.dll

Options

The list below described only the platform options. To see the specific options brought by each extension, either refer to the extension documentation page or use the --help option.

./TestExecutable.exe @"filter.rsp" --timeout 10s

where filter.rsp can have the following contents: 

--filter "A very long filter"

Or a single rsp file can be used to specify both timeout and filter as follows: 

./TestExecutable.exe @"arguments.rsp"  
--filter "A very long filter"  
--timeout 10s

MSBuild integration

The NuGet package Microsoft.Testing.Platform.MSBuild provides various integrations for Microsoft.Testing.Platform with MSBuild:

Note

This integration works in a transitive way (a project that references another project referencing this package will behave as if it references the package) and can be disabled through the IsTestingPlatformApplication MSBuild property.

See also