Justify Missing Coverage for MATLAB Source Code - MATLAB & Simulink (original) (raw)

You can improve your coverage metrics by using justifications to filter blocks of MATLAB® source code covered by methods other than automated testing, for instance, by interactive testing. Justifying missing coverage for a block of code is useful if it is not feasible to satisfy all coverage outcomes for that block in automated tests or if you do not intend for your automated tests to exercise the unsatisfied outcomes. Justifications allow you to focus on parts of source code that can and should be tested through automated tests.

This example shows how to justify missing coverage for a block of code in a MATLAB project. First, you run the tests in the project and generate an interactive code coverage report. Then, you justify missing coverage for a block of code by selecting its entry point in the report and creating a justification. When you save the justification, the testing framework treats the entire block of code as covered and updates the coverage metrics as if all coverage outcomes for that block were satisfied.

Generate Code Coverage Report

To justify the missing coverage for your code, first generate an interactive HTML code coverage report that includes information at the decision-coverage level or above. For more information about coverage types, see Types of Code Coverage for MATLAB Source Code.

Open the ShortestPath project, which includes a function namedshortest_path in its src folder and tests for the function in its tests folder. Theshortest_path function returns the length of the shortest path between two nodes in a graph.

openExample("matlabtest/ShortestPathExample")

Using a test runner that provides programmatic access to information on all possible coverage types, run the tests in the project and collect code coverage information for the source code in thesrc folder. The coverage result indicates that the tests in the project achieve full function coverage. However, the tests do not achieve full coverage for the remaining coverage types.

import matlab.unittest.plugins.CodeCoveragePlugin import matlab.unittest.plugins.codecoverage.CoverageResult

runner = testrunner("minimal"); format = CoverageResult; plugin = CodeCoveragePlugin.forFolder("src", ... Producing=format,MetricLevel="mcdc"); addPlugin(runner,plugin)

suite = testsuite; testResults = run(runner,suite);

coverageResult = format.Result

coverageResult =

Result with properties:

    Filename: "C:\work\Examples\matlabtest\ShortestPathExample\MATLABShortestPath\src\shortest_path.m"
CreationDate: 25-Oct-2024 14:52:15
      Filter: [0×0 matlabtest.coverage.Justification]

Coverage summary (HTML report): Function: 3/3 (100%) Statement: 44/50 (88%) Decision: 29/36 (80.55%) Condition: 23/30 (76.66%) MC/DC: 8/15 (53.33%)

Use coverageSummary to retrieve information from the coverage results.

Generate an interactive code coverage report from the coverage result. The report displays the collected metrics and uses different colors to highlight the covered, missed, or partially covered executables.

generateHTMLReport(coverageResult)

Code coverage report for the source code in the src folder

Create Justification to Filter Code Block

You can filter blocks of code that did not receive full coverage by creating justifications in an interactive code coverage report. The framework automatically creates a matlabtest.coverage.Justification object for each justification that you create in the report. The framework also saves information about theJustification objects to an XML file for future use. For example, suppose that you create an interactive code coverage report in a project folder, justify the missing coverage, and then close the report. If you reopen the report, the report includes the justifications that you previously created.

Creating justifications in a code coverage report is subject to these requirements:

This example creates a justification in the generated code coverage report for illustrative purposes. To create a justification, select Decision from the Currently viewing list on the report. Then, in theSource Details section, click the highlighted codeif(isscalar(node) && isnumeric(node) && ~isinf(node) && floor(node) == node), which is the entry point to anif-else code block. The framework displays the Create Coverage Justification dialog box that prompts you to specify the reason for coverage filtering. Enter a reason for the justification and then click OK. The framework creates a Justification object and saves information about it to an XML file under the project root folder. If you create additional justifications, the framework saves them to the same XML file.

Note

The framework creates the XML file at a predetermined location only in MATLAB projects. If you are not using a project, you must specify a file path using your system file browser.

Create Coverage Justification dialog box, with the reason entered as "For illustrative purposes"

Once you justify missing coverage for a block of code, the framework updates the code coverage report as if all the coverage outcomes for that block were satisfied. The Overall Coverage Summary andBreakdown by Source sections of the report include the updated coverage metrics and represent the percentages of justified outcomes using gray bars. In the Source Details section, code highlighting for the affected executables depends on the executable type. The framework highlights the missed executables as justified (in gray) and the partially covered executables as covered (in green).

Code coverage report after justification. In the Overall Coverage Summary and Breakdown by Source sections, the justified outcomes are represented in gray.

Interact with Justification in Report

You can interact with justifications in the Source Details section of the code coverage report:

For example, with MC/DC selected from the Currently viewing list on the code coverage report, click Justification Summary in the Statement column for the justified outcome out = false; of the filtered code block. The framework displays a summary of the justification in the Justification Applied dialog box. The summary includes the name of the file containing the code block, the entry point to the code block, and the justification reason. The dialog box contains a button for deleting the justification and another button for closing the dialog box. For this example, close the dialog box by clicking the Cancel button.

Justification Applied dialog box, showing the filename, code block entry point, and justification reason

Apply Justification to Coverage Result

After justifying the missing coverage for your code, you can create new coverage results by using the applyFilter and resetFilter methods. For example, create a coverage result that incorporates the justified outcomes by applying the justification to the original coverage result. In this example, which is based on a MATLAB project, the XML file containing the justification information has a path predetermined by the framework. If you are not using a project to organize your work, specify the path to the XML file you used when creating your justifications.

newResult = coverageResult.applyFilter( ... fullfile(pwd,"resources","CodeCoverageFilter.xml"))

newResult =

Result with properties:

    Filename: "C:\work\Examples\matlabtest\ShortestPathExample\MATLABShortestPath\src\shortest_path.m"
CreationDate: 25-Oct-2024 14:52:15
      Filter: [1×1 matlabtest.coverage.Justification]

Coverage summary (HTML report): Function: 3/3 (100%) Statement: (44+1)/50 (90%) Decision: (29+1)/36 (83.33%) Condition: (23+4)/30 (90%) MC/DC: (8+4)/15 (80%)

Use coverageSummary to retrieve information from the coverage results.

Generate a standalone code coverage report from newResult. The report includes the justified outcomes and the justification reason.

generateStandaloneReport(newResult)

Generating standalone report. Please wait. Preparing content for the standalone report. Adding content to the standalone report. Writing standalone report to file.

Standalone code coverage report including the justified outcomes in the Overall Summary and Table of Contents sections and the Justification column in the source file section

Note

If you generate code coverage results in Cobertura XML format, the resulting XML file represents the justified outcomes as covered.

Reuse Justification in Fresh Analysis

If you rerun your tests to collect fresh code coverage information, you can reuse the justifications previously created for the source code under test. For instance, run the tests in the project to create a new interactive code coverage report in thereport subfolder of the project root folder. The framework automatically applies the existing justification in the project to the generated report.

import matlab.unittest.plugins.codecoverage.CoverageReport

r = testrunner("minimal"); f = CoverageReport("report"); p = CodeCoveragePlugin.forFolder("src",Producing=f,MetricLevel="mcdc"); addPlugin(r,p) results = run(r,suite);

MATLAB code coverage report has been saved to: C:\work\Examples\matlabtest\ShortestPathExample\MATLABShortestPath\report\index.html

Note

If you are not using a MATLAB project, reusing justifications requires you to specify theFilter name-value argument when creating a plugin using one of the static methods of the matlab.unittest.plugins.CodeCoveragePlugin class.

See Also

Functions

Classes

Apps

Topics