matlab.unittest.plugins.TestReportPlugin - Plugin that generates a test report - MATLAB (original) (raw)

Namespace: matlab.unittest.plugins
Superclasses: matlab.unittest.plugins.TestRunnerPlugin, matlab.unittest.plugins.Parallelizable

Plugin that generates a test report

Description

The matlab.unittest.plugins.TestReportPlugin class provides a plugin that generates a test report. You can use this plugin to produce readable and archivable test reports.

The matlab.unittest.plugins.TestReportPlugin class is a handle class.

Creation

Create a TestReportPlugin instance using one of its static methods:

Properties

expand all

Title of the test report, returned as a string scalar. You can specify the value of this property during creation of the plugin. By default, the plugin uses"MATLAB® Test Report" as the title.

Attributes:

GetAccess public
SetAccess immutable

Whether to include the text output from the Command Window, returned as a0 or 1 of data type logical. You can specify the value of this property during creation of the plugin. By default, the plugin does not include the text output from the Command Window in the test report.

Attributes:

GetAccess public
SetAccess immutable

Whether to include the diagnostics for passing events, returned as a0 or 1 of data type logical. You can specify the value of this property during creation of the plugin. By default, the plugin does not include the diagnostics for passing events in the test report.

Attributes:

GetAccess public
SetAccess immutable

Maximum verbosity level of logged diagnostics to include in the test report, returned as a matlab.automation.Verbosity enumeration object. You can specify the value of this property during creation of the plugin. By default, the plugin includes diagnostics logged at thematlab.automation.Verbosity.Terse level.

Logged diagnostics are diagnostics that you supply to the testing framework with the log (TestCase) and log (Fixture) methods.

Attributes:

GetAccess public
SetAccess private

Methods

Examples

collapse all

Create a test suite from two test files, run the suite, and generate a .docx report of the results.

Create a new file in your working folder named ScriptBasedTest.m containing the following test script. The script includes two failing and incomplete tests.

%% Test double class expSolution = 'double'; actSolution = ones; assert(isa(actSolution,expSolution))

%% Test single class expSolution = 'single'; actSolution = ones('single'); assert(isa(actSolution,expSolution))

%% Test uint16 class expSolution = 'uint16'; actSolution = ones('uint16'); assert(isa(actSolution,expSolution))

%% Test that fails assert(false==true);

%% Another test that fails assert(strcmp('correlation','causation'))

Create a file named ClassBasedTest.m containing the following test class.

classdef ClassBasedTest < matlab.unittest.TestCase properties (ClassSetupParameter) generator = {'twister','combRecursive','multFibonacci'}; end properties (MethodSetupParameter) seed = {0,123,4294967295}; end properties (TestParameter) dim1 = struct('small',1,'medium',2,'large',3); dim2 = struct('small',2,'medium',3,'large',4); dim3 = struct('small',3,'medium',4,'large',5); type = {'single','double'}; end methods (TestClassSetup) function ClassSetup(testCase,generator) orig = rng; testCase.addTeardown(@rng,orig) rng(0, generator) end end methods (TestMethodSetup) function MethodSetup(testCase,seed) orig = rng; testCase.addTeardown(@rng,orig) rng(seed) end end methods (Test, ParameterCombination='sequential') function testSize(testCase,dim1,dim2,dim3) testCase.verifySize(rand(dim1,dim2,dim3),[dim1 dim2 dim3]) end end methods (Test, ParameterCombination='pairwise') function testRepeatable(testCase,dim1,dim2,dim3) state = rng; firstRun = rand(dim1,dim2,dim3); rng(state) secondRun = rand(dim1,dim2,dim3); testCase.verifyEqual(firstRun,secondRun); end end methods (Test) function testClass(testCase,dim1,dim2,type) testCase.verifyClass(rand(dim1,dim2,type),type) end end end

At the command prompt, create a test suite from both test files.

import matlab.unittest.TestRunner; import matlab.unittest.TestSuite; import matlab.unittest.plugins.TestReportPlugin;

suite = testsuite({'ScriptBasedTest','ClassBasedTest'})

suite =

1×284 Test array with properties:

Name
ProcedureName
TestClass
BaseFolder
Parameterization
SharedTestFixtures
Tags

Tests Include: 17 Unique Parameterizations, 0 Shared Test Fixture Classes, 0 Tags.

Create a silent test runner, so that there is no information output to the command window. Create a TestReportPlugin that sends output to the file MyTestReport.docx.

runner = TestRunner.withNoPlugins; docxFile = 'MyTestReport.docx'; plugin = TestReportPlugin.producingDOCX(docxFile);

Add the plugin to the TestRunner and run the suite.

runner.addPlugin(plugin); result = runner.run(suite)

Generating report. Please wait. Preparing content for the report. Adding content to the report. Writing report to file. Report has been saved to: C:\work\MyTestReport.docx

result =

1×284 TestResult array with properties:

Name
Passed
Failed
Incomplete
Duration
Details

Totals: 282 Passed, 2 Failed, 2 Incomplete. 0.73477 seconds testing time.

Open the test report.

Create a test suite from a function-based test, run the suite, and generate a report of the results. Include passing diagnostics and the text output to the Command Window.

Create a new file in your working folder named FunctionBasedTest.m containing the following function-based test. The test file includes two failing tests.

%% Main function to generate tests function tests = FunctionBasedTest tests = functiontests(localfunctions); end

%% Test Functions function passingTest(testCase) actSolution = 133+75; expSolution = 74; verifyEqual(testCase,actSolution,expSolution) end

function failingTest(testCase) actSolution = single(1); verifyTrue(testCase,actSolution) end

function anotherPassingTest(testCase) verifyClass(testCase,string('some text'),'string') end

function anotherFailingTest(testCase) verifyTrue(testCase,strcmp('42','everything')) end

At the command prompt, create a test suite from FunctionBasedTest.m. Create a test runner that displays output to the command window using the default plugin.

import matlab.unittest.TestRunner; import matlab.unittest.TestSuite; import matlab.unittest.plugins.TestReportPlugin;

suite = testsuite('FunctionBasedTest'); runner = TestRunner.withTextOutput;

Create a TestReportPlugin that sends output to the file MyTestReport2.pdf. Include passing diagnostics and text output from the Command Window in the report.

pdfFile = 'MyTestReport2.pdf'; plugin = TestReportPlugin.producingPDF(pdfFile,... 'IncludingPassingDiagnostics',true,'IncludingCommandWindowText',true);

Add the plugin to the TestRunner and run the suite.

runner.addPlugin(plugin); result = runner.run(suite);

Running FunctionBasedTest .

Verification failed in FunctionBasedTest/failingTest.

---------------------
Framework Diagnostic:
---------------------
verifyTrue failed.
--> The value must be logical. It is of type "single".

Actual single:
         1

------------------
Stack Information:
------------------
In C:\Work\FunctionBasedTest.m (failingTest) at 15

================================================================================ ..

Verification failed in FunctionBasedTest/anotherFailingTest.

---------------------
Framework Diagnostic:
---------------------
verifyTrue failed.
--> The value must evaluate to "true".

Actual logical:
       0

------------------
Stack Information:
------------------
In C:\Work\FunctionBasedTest.m (anotherFailingTest) at 23

================================================================================ . Done FunctionBasedTest

Failure Summary:

 Name                                  Failed  Incomplete  Reason(s)
===================================================================================
 FunctionBasedTest/failingTest           X                 Failed by verification.
-----------------------------------------------------------------------------------
 FunctionBasedTest/anotherFailingTest    X                 Failed by verification.

Generating report. Please wait. Preparing content for the report. Adding content to the report. Writing report to file. Report has been saved to: C:\Work\MyTestReport2.pdf

Open the test report.

In a file in your current working folder, create the FigurePropTest test class. If the failingTest test method fails (it always does in this example), it uses a FigureDiagnostic to save the figure so you can examine it later.

classdef FigurePropTest < matlab.unittest.TestCase properties TestFigure end methods (TestMethodSetup) function createFigure(testCase) testCase.TestFigure = figure; end end methods (TestMethodTeardown) function closeFigure(testCase) close(testCase.TestFigure) end end methods (Test) function defaultCurrentPoint(testCase) cp = testCase.TestFigure.CurrentPoint; testCase.verifyEqual(cp,[0 0], ... 'Default current point is incorrect') end function defaultCurrentObject(testCase) import matlab.unittest.constraints.IsEmpty co = testCase.TestFigure.CurrentObject; testCase.verifyThat(co,IsEmpty, ... 'Default current object should be empty') end function failingTest(testCase) import matlab.unittest.diagnostics.FigureDiagnostic fig = testCase.TestFigure; ax = axes(fig); surf(ax,peaks) testCase.verifyEmpty(testCase.TestFigure.Children, ... FigureDiagnostic(testCase.TestFigure)) end end end

At the command prompt, create a test suite.

suite = testsuite('FigurePropTest');

Create a silent test runner that records diagnostics and generates a PDF report.

import matlab.unittest.plugins.DiagnosticsRecordingPlugin import matlab.unittest.plugins.TestReportPlugin runner = matlab.unittest.TestRunner.withNoPlugins; runner.addPlugin(DiagnosticsRecordingPlugin) runner.addPlugin(TestReportPlugin.producingPDF('MyTestReport.pdf'))

Change the default artifact root to your current working folder.

runner.ArtifactsRootFolder = pwd;

Run the tests. The third test fails.

results = runner.run(suite)

Generating test report. Please wait. Preparing content for the test report. Adding content to the test report. Writing test report to file. Test report has been saved to: C:\wok\MyTestReport.pdf

results =

1×3 TestResult array with properties:

Name
Passed
Failed
Incomplete
Duration
Details

Totals: 2 Passed, 1 Failed (rerun), 0 Incomplete. 8.3355 seconds testing time.

Display the test diagnostic results for the third test. The testing framework saved two artifacts related to the third test. By default, a FigureDiagnostic object saves a figure as both a PNG file and a FIG file.

results(3).Details.DiagnosticRecord.TestDiagnosticResults

ans =

DiagnosticResult with properties:

     Artifacts: [1×2 matlab.automation.diagnostics.FileArtifact]
DiagnosticText: 'Figure saved to:↵--> C:\work\530aa31c-86c7-4712-a064-d9f00ce041fb\Figure_dfd8c611-8387-4579-804f-6384642ba4ff.fig↵--> C:\work\530aa31c-86c7-4712-a064-d9f00ce041fb\Figure_dfd8c611-8387-4579-804f-6384642ba4ff.png'

Display the stored location of the first artifact.

results(3).Details.DiagnosticRecord.TestDiagnosticResults.Artifacts(1)

ans =

FileArtifact with properties:

    Name: "Figure_dfd8c611-8387-4579-804f-6384642ba4ff.fig"
Location: "C:\work\530aa31c-86c7-4712-a064-d9f00ce041fb"
FullPath: "C:\work\530aa31c-86c7-4712-a064-d9f00ce041fb\Figure_dfd8c611-8387-4579-804f-6384642ba4ff.fig"

To inspect the image related to the failed test, open the file at the location shown in the FullPath field. Additionally, since you generated a PDF test report, the image is captured in MyTestReport.pdf. The test report also contains the path to the artifacts.

Version History

Introduced in R2016b

expand all

To support modifying the test report title, the TestReportPlugin class has a new property named Title.

Test reports generated using the TestReportPlugin class display the test tags for tagged test suite elements. You can generate tagged test reports in DOCX, HTML, and PDF formats.

See Also

Classes

Functions