Write Plugin to Save Diagnostic Details - MATLAB & Simulink (original) (raw)

This example shows how to create a custom plugin to save diagnostic details. The plugin listens for test failures and saves diagnostic information so you can access it after the framework completes the tests.

Create Plugin

In a file in your working folder, create a class, myPlugin, that inherits from the matlab.unittest.plugins.TestRunnerPlugin class. In the plugin class:

The plugin saves information contained in the PluginData and QualificationEventData objects. It also saves the type of failure and timestamp.

classdef DiagnosticRecorderPlugin < matlab.unittest.plugins.TestRunnerPlugin

properties
    FailedTestData
end

methods (Access = protected)
    function runTestSuite(plugin, pluginData)
        plugin.FailedTestData = [];
        runTestSuite@...
            matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
    end
    
    function testCase = createTestMethodInstance(plugin, pluginData)
        testCase = createTestMethodInstance@...
            matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
        
        testName = pluginData.Name;
        testCase.addlistener('AssertionFailed', ...
            @(~,event)plugin.recordData(event,testName, 'Assertion'));
        testCase.addlistener('FatalAssertionFailed', ...
            @(~,event)plugin.recordData(event,testName, 'Fatal Assertion'));
        testCase.addlistener('VerificationFailed', ...
            @(~,event)plugin.recordData(event,testName, 'Verification'));
    end
end

methods (Access = private)
    function recordData(plugin,eventData,name,failureType)
        s.Name = {name};
        s.Type = {failureType};
        if isempty(eventData.TestDiagnosticResult)
            s.TestDiagnostics = 'TestDiagnostics not provided';
        else
            s.TestDiagnostics = eventData.TestDiagnosticResult;
        end
        s.FrameworkDiagnostics = eventData.FrameworkDiagnosticResult;
        s.Stack = eventData.Stack;
        s.Timestamp = datetime;
        
        plugin.FailedTestData = [plugin.FailedTestData; struct2table(s)];
    end
end

end

Create Test Class

In your working folder, create the file ExampleTest.m containing the following test class.

classdef ExampleTest < matlab.unittest.TestCase methods(Test) function testOne(testCase) testCase.assertGreaterThan(5,10) end function testTwo(testCase) wrongAnswer = 'wrong'; testCase.verifyEmpty(wrongAnswer,'Not Empty') testCase.verifyClass(wrongAnswer,'double','Not double') end

    function testThree(testCase)
        testCase.assertEqual(7*2,13,'Values not equal')
    end
    function testFour(testCase)
        testCase.fatalAssertEqual(3+2,6);
    end
end

end

The fatal assertion failure in testFour causes the framework to halt and throw an error. In this example, there are no subsequent tests. If there was a subsequent test, the framework would not run it.

Add Plugin to Test Runner and Run Tests

At the command prompt, create a test suite from the ExampleTest class, and create a test runner.

import matlab.unittest.TestSuite import matlab.unittest.TestRunner

suite = TestSuite.fromClass(?ExampleTest); runner = TestRunner.withNoPlugins;

Create an instance of myPlugin and add it to the test runner. Run the tests.

p = DiagnosticRecorderPlugin; runner.addPlugin(p) result = runner.run(suite);

Error using ExampleTest/testFour (line 16) Fatal assertion failed.

With the failed fatal assertion, the framework throws an error, and the test runner does not return a TestResult object. However, the DiagnosticRecorderPlugin stores information about the tests preceding and including the test with the failed assertion.

Inspect Diagnostic Information

At the command prompt, view information about the failed tests. The information is saved in the FailedTestData property of the plugin.

T =

5×6 table

         Name                    Type                  TestDiagnostics                                                                                                                                                                FrameworkDiagnostics                                                                                                                                                           Stack             Timestamp      
_______________________    _________________    ______________________________    ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________    ____________    ____________________

'ExampleTest/testOne'      'Assertion'          'TestDiagnostics not provided'    'assertGreaterThan failed.↵--> The value must be greater than the minimum value.↵↵Actual Value:↵     5↵Minimum Value (Exclusive):↵    10'                                                                                                                                                                                       [1x1 struct]    17-Jul-2017 12:41:18
'ExampleTest/testTwo'      'Verification'       'Not Empty'                       'verifyEmpty failed.↵--> The value must be empty.↵--> The value has a size of [1  5].↵↵Actual char:↵    wrong'                                                                                                                                                                                                                  [1x1 struct]    17-Jul-2017 12:41:18
'ExampleTest/testTwo'      'Verification'       'Not double'                      'verifyClass failed.↵--> The value's class is incorrect.↵    ↵    Actual Class:↵        char↵    Expected Class:↵        double↵↵Actual char:↵    wrong'                                                                                                                                                                        [1x1 struct]    17-Jul-2017 12:41:18
'ExampleTest/testThree'    'Assertion'          'Values not equal'                'assertEqual failed.↵--> The values are not equal using "isequaln".↵--> Failure table:↵        Actual    Expected    Error      RelativeError   ↵        ______    ________    _____    __________________↵    ↵        14        13          1        0.0769230769230769↵↵Actual Value:↵    14↵Expected Value:↵    13'         [1x1 struct]    17-Jul-2017 12:41:18
'ExampleTest/testFour'     'Fatal Assertion'    'TestDiagnostics not provided'    'fatalAssertEqual failed.↵--> The values are not equal using "isequaln".↵--> Failure table:↵        Actual    Expected    Error      RelativeError   ↵        ______    ________    _____    __________________↵    ↵        5         6           -1       -0.166666666666667↵↵Actual Value:↵     5↵Expected Value:↵     6'    [1x1 struct]    17-Jul-2017 12:41:18

There are many options to archive or post-process this information. For example, you can save the variable as a MAT-file or use writetable to write the table to various file types, such as .txt, .csv, or .xls.

View the stack information for the third test failure

ans =

struct with fields:

file: 'C:\Work\ExampleTest.m'
name: 'ExampleTest.testTwo'
line: 9

Display the diagnostics that the framework displayed for the fifth test failure.

celldisp(T.FrameworkDiagnostics(5))

ans{1} =

fatalAssertEqual failed. --> The values are not equal using "isequaln". --> Failure table: Actual Expected Error RelativeError
______ ________ _____ __________________

    5         6           -1       -0.166666666666667

Actual Value: 5 Expected Value: 6

See Also

matlab.unittest.plugins.TestRunnerPlugin | matlab.unittest.TestCase | matlab.unittest.TestRunner | addlistener

Topics