testsuite - Create suite of tests - MATLAB (original) (raw)

Syntax

Description

suite = testsuite creates a suite of tests from your current folder, and returns the suite as a TestSuite array.

To run a test suite created with testsuite, use the run method of matlab.unittest.TestSuite, matlab.unittest.TestRunner, or matlab.perftest.TimeExperiment.

example

suite = testsuite([tests](#bu4b4az-tests)) creates a suite from a set of specified tests.

suite = testsuite(___,[Name,Value](#namevaluepairarguments)) creates a suite of tests with additional options specified by one or more name-value arguments.

example

Examples

collapse all

Create a folder myExample in your current working folder, make it your current working folder, and create a couple of tests.

In the myExample folder, create a script-based test, onesTest.m.

%% Test double class expClass = 'double'; act = ones; assert(isa(act,expClass))

%% Test single class expClass = 'single'; act = ones('single'); assert(isa(act,expClass))

%% Test uint16 class expClass = 'uint16'; act = ones('uint16'); assert(isa(act,expClass))

%% Test size expSize = [7 13]; act = ones([7 13]); assert(isequal(size(act),expSize))

%% Test values act = ones(42); assert(unique(act) == 1)

In the myExample folder, create a function-based test, eyeTest.m.

function tests = eyeTest tests = functiontests(localfunctions);

function doubleClassTest(testCase) actValue = eye; verifyClass(testCase,actValue,'double')

function singleClassTest(testCase) actValue = eye('single'); verifyClass(testCase,actValue,'single')

function uint16ClassTest(testCase) actValue = eye('uint16'); verifyClass(testCase,actValue,'uint16')

function sizeTest(testCase) expSize = [7 13]; actValue = eye(expSize); verifySize(testCase,actValue,expSize);

function valueTest(testCase) actValue = eye(42); verifyEqual(testCase,unique(diag(actValue)),1) % diagonal are 1s verifyEqual(testCase,unique(triu(actValue,1)),0) % upper tri vals are 0 verifyEqual(testCase,unique(tril(actValue,-1)),0) % lower tri vals are 0

Create a test suite from all tests in the current folder.

suite =

1×10 Test array with properties:

Name
BaseFolder
ProcedureName
SharedTestFixtures
Parameterization
Tags

Tests Include: 0 Parameterizations, 0 Shared Test Fixture Classes, 0 Tags.

If onesTest and eyesTest are the only tests in your folder, MATLAB® creates a suite of 10 tests.

View the names of the tests in suite.

ans =

'eyeTest/doubleClassTest'
'eyeTest/singleClassTest'
'eyeTest/uint16ClassTest'
'eyeTest/sizeTest'
'eyeTest/valueTest'
'onesTest/TestDoubleClass'
'onesTest/TestSingleClass'
'onesTest/TestUint16Class'
'onesTest/TestSize'
'onesTest/TestValues'

Create a test suite from all tests in eyeTest.

suite2 = testsuite('eyeTest')

suite2 =

1×5 Test array with properties:

Name
BaseFolder
ProcedureName
SharedTestFixtures
Parameterization
Tags

Tests Include: 0 Parameterizations, 0 Shared Test Fixture Classes, 0 Tags.

In your working folder, create a class-based test, testZeros.m. This class contains five test methods.

classdef testZeros < matlab.unittest.TestCase properties (TestParameter) type = {'single','double','uint16'} outSize = struct('s2d',[3 3],'s3d',[2 5 4]) end

methods (Test)
    function testClass(testCase,type,outSize)
        testCase.verifyClass(zeros(outSize,type),type)
    end
    
    function testSize(testCase,outSize)
        testCase.verifySize(zeros(outSize),outSize)
    end
    
    function testDefaultClass(testCase)
        testCase.verifyClass(zeros,'double')
    end
    
    function testDefaultSize(testCase)
        testCase.verifySize(zeros,[1 1])
    end
    
    function testDefaultValue(testCase)
        testCase.verifyEqual(zeros,0)
    end
end

end

The full test suite has 11 test elements: 6 from the testClass method, 2 from the testSize method, and 1 each from the testDefaultClass, testDefaultSize, and testDefaultValue methods.

Create a test suite from the test elements with test names that contain 'Default'.

suite = testsuite('testZeros','Name','Default')

suite =

1×3 Test array with properties:

Name
ProcedureName
TestClass
BaseFolder
Parameterization
SharedTestFixtures
Tags

Tests Include: 0 Parameterizations, 0 Shared Test Fixture Classes, 0 Tags.

Create a test suite from the test elements that use the outSize parameter property.

suite = testsuite('testZeros','ParameterProperty','outSize')

suite =

1×8 Test array with properties:

Name
ProcedureName
TestClass
BaseFolder
Parameterization
SharedTestFixtures
Tags

Tests Include: 5 Unique Parameterizations, 0 Shared Test Fixture Classes, 0 Tags.

The test suite contains eight tests that use the outSize parameter property: six from the testClass method and two from the testSize method.

Input Arguments

collapse all

Tests, specified as a string array, character vector, or cell array of character vectors. Use this argument to specify your test content. For example, you can specify a test file, a test class, a folder that contains test files, a namespace that contains test classes, or a project folder that contains test files.

Example: testsuite("myTestFile.m")

Example: testsuite(["myTestFile/test1" "myTestFile/test3"])

Example: testsuite("myNamespace.MyTestClass")

Example: testsuite(pwd)

Example: testsuite({'myNamespace.MyTestClass','myTestFile.m',pwd,'myNamespace.innerNamespace'})

Example: testsuite("C:\projects\project1")

Name-Value Arguments

expand all

Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: suite = testsuite(tests,Name="productA_*") creates a test suite from tests that have names starting with"productA_".

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: suite = testsuite(tests,"Name","productA_*") creates a test suite from tests that have names starting with"productA_".

Test Identification

expand all

Option to include tests in subfolders in the suite, specified as a numeric or logical 0 (false) or1 (true). By default, the framework creates a suite from tests in the specified folders and not in their subfolders.

Option to include tests in inner namespaces in the suite, specified as a numeric or logical 0 (false) or1 (true). By default, the framework creates a suite from tests in the specified namespaces, but not in their inner namespaces.

Option to include tests from referenced projects, specified as a numeric or logical0 (false) or 1 (true). For more information on referenced projects, see Componentize Large Projects.

Action to take against an invalid test file in a folder or namespace that the function is processing, specified as one of these values:

• "warn" — The function issues a warning for each invalid test file in a folder or namespace and creates a test suite from the valid files.
• "error" — The function throws an error if it finds an invalid test file in a folder or namespace.

An invalid test file is a test file from which the framework cannot generate a test suite. Examples include a test file that contains syntax errors, a function-based test file that is missing local functions, and a file with a Test method that is passed an undefined parameterization property.

Test Filtering

expand all

Name of the base folder that contains the test file, specified as a string array, character vector, or cell array of character vectors. This argument filters the test suite. For the testing framework to include a test in the filtered suite, the Test element must be contained in one of the base folders specified byBaseFolder. If none of theTest elements match a base folder, an empty test suite is returned. Use the wildcard character (*) to match any number of characters. Use the question mark character (?) to match a single character.

For test files defined in namespaces, the base folder is the parent of the top-level namespace folder.

Names of the files and folders that contain source code, specified as a string vector, character vector, or cell vector of character vectors. This argument filters the test suite by including only the tests that depend on the specified source code. If none of the tests depend on the source code, an empty test suite is returned.

The specified value must represent at least one existing file. If you specify a folder, the framework extracts the paths to the files within the folder.

You must have a MATLAB Test™ license to use DependsOn. For more information about selecting tests by source code dependency, see matlabtest.selectors.DependsOn (MATLAB Test).

Example: DependsOn=["myFile.m" "myFolder"]

Example: DependsOn=["folderA" "C:\work\folderB"]

Name of the test, specified as a string array, character vector, or cell array of character vectors. This argument filters the test suite. For the testing framework to include a test in the filtered suite, the Name property of theTest element must match one of the names specified byName. If none of the Test elements have a matching name, an empty test suite is returned. Use the wildcard character (*) to match any number of characters. Use the question mark character (?) to match a single character.

For a given test file, the name of a test uniquely identifies the smallest runnable portion of the test content. The test name includes the namespace name, filename (excluding the extension), procedure name, and information about parameterization.

Name of a test class property that defines a parameter used by the test, specified as a string array, character vector, or cell array of character vectors. This argument filters the test suite. For the testing framework to include a test in the filtered suite, theParameterization property of the Test element must contain at least one of the property names specified byParameterProperty. If none of the Test elements have a matching property name, an empty test suite is returned. Use the wildcard character (*) to match any number of characters. Use the question mark character (?) to match a single character.

Name of a parameter used by the test, specified as a string array, character vector, or cell array of character vectors. MATLAB generates parameter names based on the test class property that defines the parameters. For example:

• If the property value is a cell array, MATLAB generates parameter names from the elements of the cell array by taking into account their values, types, and dimensions.
• If the property value is a structure, MATLAB generates parameter names from the structure fields.

The ParameterName argument filters the test suite. For the testing framework to include a test in the filtered suite, theParameterization property of theTest element must contain at least one of the parameter names specified by ParameterName. If none of the Test elements have a matching parameter name, an empty test suite is returned. Use the wildcard character (*) to match any number of characters. Use the question mark character (?) to match a single character.

Name of the class that the test class derives from, specified as a string array, character vector, or cell array of character vectors. This argument filters the test suite. For the testing framework to include a test in the filtered suite, theTestClass property of the Test element must point to a test class that derives from one of the classes specified bySuperclass. If none of the Test elements match a class, an empty test suite is returned.

Name of a tag used by the test, specified as a string array, character vector, or cell array of character vectors. This argument filters the test suite. For the testing framework to include a test in the filtered suite, the Tags property of the Test element must contain at least one of the tag names specified by Tag. If none of the Test elements have a matching tag name, an empty test suite is returned. Use the wildcard character (*) to match any number of characters. Use the question mark character (?) to match a single character.

More About

collapse all

You can use the testsuite function to create a test suite from an MLDATX file (requires Simulink® Test). For example, suite = testsuite('myTestFile.mldatx') creates a suite from the tests specified in the file myTestFile.mldatx.

When you specify an MLDATX file, testsuite creates a suite including all of the tests in the file. You cannot instructtestsuite to create a suite from specific tests in an MLDATX file.

Tips

• If you do not need to create a test suite explicitly, use runtests or runperf to create the suite implicitly before running the tests.
• An alternative way to create an explicit test suite is to use the matlab.unittest.TestSuite methods.
• When you specify the input to the testsuite function as a string array or cell array of character vectors (for example, suite = testsuite(["Test1","Test2"])), the testing framework sorts the array to reduce shared test fixture setup and teardown operations. As a result, the tests might run in an order that is different from the order of elements in the input array.
  To enforce the order of the test run, create the suite by using several calls to testsuite. For example, to ensure that the tests specified by Test1 run before the tests specified byTest2, use this syntax:
  suite = [testsuite("Test1") testsuite("Test2")]

Version History

Introduced in R2016a

expand all

When you select function-based or class-based tests using theDependsOn name-value argument (requires MATLAB Test), the function more accurately selects tests that depend on the specified source code. If the function can determine which individual tests in the test file depend on the source code, then it selects only the dependent tests and excludes the rest. Otherwise, the function includes all the tests in the test file.

In previous releases, the function includes all the tests in a test file if the file depends on the specified source code, without attempting to exclude tests that are not dependent on the source code.

The IncludeSubpackages name-value argument is now named IncludeInnerNamespaces. The behavior remains the same, and existing instances of IncludeSubpackages in your code continue to work as expected. There are no plans to remove support for existing references to IncludeSubpackages.

If you have a MATLAB Test license, you can specify any type of source file using theDependsOn name-value argument. In previous releases, you can specify files only with a .m, .p,.mlx, .mlapp, .mat, or.slx extension.

You can filter a test suite by test file dependency on specified source code. Use theDependsOn name-value argument (requires MATLAB Test) to specify the source files and folders.

If you have Requirements Toolbox™ and MATLAB Test installed, you can use the testsuite function to create a test suite from tests that verify requirement sets. To create a suite, specify one or more requirement set files as a string scalar or string vector. For example, suite = testsuite("myRequirementSet.slreqx") creates a suite of tests that verify the specified requirement set.

To specify whether the testing framework issues a warning or throws an error when it encounters an invalid test file in a folder or namespace, use theInvalidFileFoundAction name-value argument.

When you assign a nonempty cell array to a parameterization property, the testing framework generates parameter names from the elements of the cell array by taking into account their values, types, and dimensions. In previous releases, if the property value is a cell array of character vectors, the framework generates parameter names from the values in the cell array. Otherwise, the framework specifies parameter names as value1, value2, …, valueN.

If your code uses parameter names to create or filter test suites, replace the old parameter names with the descriptive parameter names. For example, update suite = testsuite(pwd,"ParameterName","value1") by replacing value1 with a descriptive parameter name.

The IncludeSubfolders name-value argument treats folders and namespaces the same way. For example, suite = testsuite(pwd,IncludeSubfolders=true) creates a suite from all the test files in the current folder and any of its subfolders, including namespace folders. In previous releases, IncludeSubfolders ignores namespace folders.

The testsuite function ignores any files in a MATLAB project that do not define test procedures. For example, if an abstract TestCase class definition file is labeled with theTest classification, the function ignores it. In previous releases, MATLAB produces an error if testsuite is called on a project that uses the Test classification for any files other than concrete test files.

If you start MATLAB without the Java® Virtual Machine (JVM®) software and create a suite from the test files in a project usingtestsuite, the function uses the matlab.unittest.TestSuite.fromProject method to create the suite. If you then try to run the test suite without the JVM software, MATLAB produces an error because the project cannot be opened without the JVM software. In previous releases, when MATLAB runs without the JVM software, testsuite uses matlab.unittest.TestSuite.fromFolder to create a suite from the test files in the project, and the testing framework runs the resulting test suite.

This behavior change also applies to the runtests and runperf functions when they operate on code organized into files and folders within a project.

When your current folder is a project root folder or when you pass the path to a project root folder to the testsuite function, the function creates a test suite from all test files contained in the specified project that are labeled with the Test classification.

To include the tests from referenced projects in the test suite, use theIncludeReferencedProjects name-value argument.