matlab.unittest.plugins.StopOnFailuresPlugin - Plugin to debug test failures - MATLAB (original) (raw)
Namespace: matlab.unittest.plugins
Superclasses: matlab.unittest.plugins.TestRunnerPlugin
Plugin to debug test failures
Description
The matlab.unittest.plugins.StopOnFailuresPlugin class defines a plugin for debugging test failures. When a test runner with a StopOnFailuresPlugin encounters a qualification failure or uncaught error, it pauses test execution and puts MATLAB® into debug mode. You can then use MATLAB debugging commands, such as dbstep,dbcont, and dbquit, to investigate the cause of the test failure or error.
The matlab.unittest.plugins.StopOnFailuresPlugin class is a handle class.
Creation
Description
p = matlab.unittest.plugins.StopOnFailuresPlugin constructs a plugin to debug test failures.
p = matlab.unittest.plugins.StopOnFailuresPlugin("IncludingAssumptionFailures",tf) sets the [IncludeAssumptionFailures](matlab.unittest.plugins.stoponfailuresplugin-class.html#mw%5F1a67cd9f-9810-4ce6-b2c0-34f068aa5b2a) property to tf. Use this syntax to indicate whether to react to assumption failures. By default,StopOnFailuresPlugin reacts only to uncaught errors, verification failures, assertion failures, and fatal assertion failures. However, whenIncludingAssumptionFailures is specified as true, the plugin reacts also to assumption failures.
Properties
Whether to react to assumption failures, specified as a numeric or logical0 (false) or 1 (true) and stored as a logical value. If the value istrue, then the plugin reacts to assumption failures. If the value is false, then the plugin ignores assumption failures.
Attributes:
| GetAccess | public |
|---|---|
| SetAccess | private |
Examples
Investigate the cause of test failures by adding aStopOnFailuresPlugin instance to the test runner.
In your current folder, create the ExampleTest test class.
classdef ExampleTest < matlab.unittest.TestCase methods (Test) function testOne(testCase) % Test fails act = 3.1416; exp = pi; testCase.verifyEqual(act,exp) end function testTwo(testCase) % Test does not complete testCase.assumeEqual(5,4) end end end
At the command prompt, create a test suite from ExampleTest and run the tests. As a result of the qualifications in the test class, the first test fails, and the second test does not complete.
import matlab.unittest.plugins.StopOnFailuresPlugin
suite = testsuite("ExampleTest");
runner = testrunner("textoutput");
results = runner.run(suite);
Running ExampleTest
================================================================================
Verification failed in ExampleTest/testOne.
---------------------
Framework Diagnostic:
---------------------
verifyEqual failed.
--> The numeric values are not equal using "isequaln".
--> Failure table:
Actual Expected Error RelativeError
______ ________________ ____________________ ____________________
3.1416 3.14159265358979 7.34641020683213e-06 2.33843499679617e-06
Actual Value:
3.141600000000000
Expected Value:
3.141592653589793
------------------
Stack Information:
------------------
In C:\work\ExampleTest.m (ExampleTest.testOne) at 6
.
ExampleTest/testTwo was filtered.
. Done ExampleTest
Failure Summary:
Name Failed Incomplete Reason(s)
==================================================================
ExampleTest/testOne X Failed by verification.
------------------------------------------------------------------
ExampleTest/testTwo X Filtered by assumption.Add a StopOnFailuresPlugin instance to the test runner and rerun the tests. During the test run, when the failure occurs, MATLAB enters debug mode at the source of the failure.
runner.addPlugin(StopOnFailuresPlugin) result = runner.run(suite);
Running ExampleTest
================================================================================
Verification failed in ExampleTest/testOne.
---------------------
Framework Diagnostic:
---------------------
verifyEqual failed.
--> The numeric values are not equal using "isequaln".
--> Failure table:
Actual Expected Error RelativeError
______ ________________ ____________________ ____________________
3.1416 3.14159265358979 7.34641020683213e-06 2.33843499679617e-06
Actual Value:
3.141600000000000
Expected Value:
3.141592653589793
------------------
Stack Information:
------------------
In C:\work\ExampleTest.m (ExampleTest.testOne) at 6
Test execution paused due to failure. To terminate the test run, use dbquit. To continue, use dbcont.
Investigate the cause of the test failure by using the whos function to examine the variables in the workspace.
Name Size Bytes Class Attributes
act 1x1 8 double
exp 1x1 8 double
testCase 1x1 8 ExampleTest
Try rerunning the failed test with a relative tolerance of100*eps. The test fails even with the specified tolerance.
testCase.verifyEqual(act,exp,"RelTol",100*eps)
================================================================================
Verification failed in ExampleTest/testOne.
---------------------
Framework Diagnostic:
---------------------
verifyEqual failed.
--> The numeric values are not equal using "isequaln".
--> The error was not within relative tolerance.
--> Failure table:
Actual Expected Error RelativeError RelativeTolerance
______ ________________ ____________________ ____________________ ____________________
3.1416 3.14159265358979 7.34641020683213e-06 2.33843499679617e-06 2.22044604925031e-14
Actual Value:
3.141600000000000
Expected Value:
3.141592653589793
------------------
Stack Information:
------------------
In C:\work\ExampleTest.m (ExampleTest.testOne) at 6
Try rerunning the failed test with an absolute tolerance of0.001. The failed test passes with the absolute tolerance.
testCase.verifyEqual(act,exp,"AbsTol",0.001)
Use dbquit to end the test run. You also can usedbcont to quit debug mode and run the rest of the tests.
To enter debug mode for tests that fail by assumption, such astestTwo in the ExampleTest class, specifyIncludingAssumptionFailures as true when you create the plugin. (Make sure to add only one StopOnFailuresPlugin instance to the test runner. Adding more than one StopOnFailuresPlugin instance can result in unexpected behavior.)
When you run the tests, MATLAB enters debug mode in both testOne andtestTwo.
runner.addPlugin(StopOnFailuresPlugin( ... "IncludingAssumptionFailures",true)) result = runner.run(suite);
Debug an error in your tests using aStopOnFailuresPlugin instance.
In your current folder, create the ExampleTest test class. Inject an error into your tests by including an invalid function call in thetestTwo method.
classdef ExampleTest < matlab.unittest.TestCase methods (Test) function testOne(testCase) % Test passes act = round(pi); exp = 3; testCase.verifyEqual(act,exp) end function testTwo(testCase) % Test throws an error act = cosine(0); % Invalid function call exp = 1; testCase.verifyEqual(act,exp) end end end
At the command prompt, create a test suite from ExampleTest and run the tests. As a result of the error, the second test fails and does not complete.
import matlab.unittest.plugins.StopOnFailuresPlugin
suite = testsuite("ExampleTest");
runner = testrunner("textoutput");
results = runner.run(suite);
Running ExampleTest .
Error occurred in ExampleTest/testTwo and it did not run to completion.
---------
Error ID:
---------
'MATLAB:UndefinedFunction'
--------------
Error Details:
--------------
Undefined function 'cosine' for input arguments of type 'double'.
Error in ExampleTest/testTwo (line 9)
act = cosine(0); % Invalid function call
. Done ExampleTest
Failure Summary:
Name Failed Incomplete Reason(s)
====================================================
ExampleTest/testTwo X X Errored.Add a StopOnFailuresPlugin instance to the runner and rerun the tests. During the test run, when the error is thrown, MATLAB enters debug mode at the source of the error.
runner.addPlugin(StopOnFailuresPlugin) result = runner.run(suite);
Running ExampleTest .9 act = cosine(0); % Invalid function call K>>
Fix the error by using the correct function name (that is,cos). When you rerun the tests, both of them pass.
Version History
Introduced in R2013b
When a test runner with a StopOnFailuresPlugin instance encounters an uncaught error, MATLAB enters debug mode at the source of the error and lets you use debugging commands to investigate the cause of the error.
In previous releases, while the plugin stops the test run to report the error, debugging capabilities are limited because the error disrupts the stack.