Test Performance Using Scripts or Functions - MATLAB & Simulink (original) (raw)

Main Content

This example shows how to create and run a script-based or function-based performance test that times the preallocation of a vector using four different approaches.

Write Performance Test

Create a performance test in a file namedpreallocationTest.m in your current folder. In this example, you can choose to use either the following script-based test or the function-based test. The output in this example is for the function-based test. If you use the script-based test, then your test names will be different.

Script-Based Performance Test Function-Based Performance Test
vectorSize = 1e7; %% Ones Function x = ones(1,vectorSize); %% Indexing With Variable id = 1:vectorSize; x(id) = 1; %% Indexing On LHS x(1:vectorSize) = 1; %% For Loop for i=1:vectorSize x(i) = 1; end function tests = preallocationTest tests = functiontests(localfunctions); end function testOnes(testCase) vectorSize = getSize(); x = ones(1,vectorSize()); end function testIndexingWithVariable(testCase) vectorSize = getSize(); id = 1:vectorSize; x(id) = 1; end function testIndexingOnLHS(testCase) vectorSize = getSize(); x(1:vectorSize) = 1; end function testForLoop(testCase) vectorSize = getSize(); for i=1:vectorSize x(i) = 1; end end function vectorSize = getSize() vectorSize = 1e7; end

Run Performance Test

Run the performance test using the runperf function.

results = runperf("preallocationTest.m")

Running preallocationTest .......... .......... .......... .......... ....... Done preallocationTest

results =

1×4 TimeResult array with properties:

Name
Valid
Samples
TestActivity

Totals: 4 Valid, 0 Invalid. 8.7168 seconds testing time.

The results variable is a 1-by-4 TimeResult array. Each element in the array corresponds to one of the tests defined in preallocationTest.m.

Display Test Results

Display the measurement results for the second test. Your results might vary.

ans =

TimeResult with properties:

        Name: 'preallocationTest/testIndexingWithVariable'
       Valid: 1
     Samples: [4×7 table]
TestActivity: [9×12 table]

Totals: 1 Valid, 0 Invalid. 0.87973 seconds testing time.

As indicated by the size of the TestActivity property, the performance testing framework collected nine measurements. This number of measurements includes five measurements to warm up the code. TheSamples property excludes warm-up measurements.

Display the sample measurements for the second test.

ans =

4×7 table

                   Name                       MeasuredTime         Timestamp             Host        Platform                 Version                             RunIdentifier            
__________________________________________    ____________    ____________________    ___________    ________    __________________________________    ____________________________________

preallocationTest/testIndexingWithVariable      0.096513      14-Oct-2022 14:04:00    MY-HOSTNAME     win64      9.14.0.2081372 (R2023a) Prerelease    7ff84b09-8a58-4961-bfcb-75c86f4a3ae1
preallocationTest/testIndexingWithVariable      0.097008      14-Oct-2022 14:04:00    MY-HOSTNAME     win64      9.14.0.2081372 (R2023a) Prerelease    7ff84b09-8a58-4961-bfcb-75c86f4a3ae1
preallocationTest/testIndexingWithVariable      0.096777      14-Oct-2022 14:04:00    MY-HOSTNAME     win64      9.14.0.2081372 (R2023a) Prerelease    7ff84b09-8a58-4961-bfcb-75c86f4a3ae1
preallocationTest/testIndexingWithVariable      0.097157      14-Oct-2022 14:04:00    MY-HOSTNAME     win64      9.14.0.2081372 (R2023a) Prerelease    7ff84b09-8a58-4961-bfcb-75c86f4a3ae1

Compute Statistics for Single Test Element

Display the mean measured time for the second test. To exclude data collected in the warm-up runs, use the values in the Samples property.

sampleTimes = results(2).Samples.MeasuredTime; meanTest2 = mean(sampleTimes)

Compute Statistics for All Test Elements

To compare the different preallocation methods, create a table of summary statistics from results. In this example, theones function was the fastest way to initialize the vector to ones. The performance testing framework made four measurement runs for this test.

T = sampleSummary(results)

T =

4×7 table

                   Name                       SampleSize      Mean      StandardDeviation      Min        Median       Max   
__________________________________________    __________    ________    _________________    ________    ________    ________

preallocationTest/testOnes                         4        0.016716       0.00018455        0.016515    0.016699    0.016952
preallocationTest/testIndexingWithVariable         4        0.096864        0.0002817        0.096513    0.096892    0.097157
preallocationTest/testIndexingOnLHS               15        0.024099        0.0025168        0.022721      0.0232    0.031685
preallocationTest/testForLoop                      4         0.79044         0.016054         0.78203     0.78261     0.81452

Change Statistical Objectives and Rerun Tests

Change the statistical objectives defined by the runperf function by constructing and running a time experiment. Construct a time experiment that collects two warm-up measurements and runs the test a variable number of times to reach a sample mean with a 4% relative margin of error within a 98% confidence level.

Create a test suite.

suite = testsuite("preallocationTest");

Construct a time experiment with the specified requirements, and run the test suite.

import matlab.perftest.TimeExperiment experiment = TimeExperiment.limitingSamplingError("NumWarmups",2, ... "RelativeMarginOfError",0.04,"ConfidenceLevel",0.98); resultsTE = run(experiment,suite);

Running preallocationTest .......... .......... .......... .......... Done preallocationTest

Compute the summary statistics for all the test elements.

T1 = sampleSummary(resultsTE)

T1 =

4×7 table

                   Name                       SampleSize      Mean      StandardDeviation      Min        Median       Max   
__________________________________________    __________    ________    _________________    ________    ________    ________

preallocationTest/testOnes                        16        0.017424         0.001223        0.016911    0.017056    0.021938
preallocationTest/testIndexingWithVariable         8        0.099153        0.0039523        0.096619    0.097218     0.10736
preallocationTest/testIndexingOnLHS                4        0.022985       0.00018664        0.022843    0.022921    0.023257
preallocationTest/testForLoop                      4         0.80613         0.005993         0.79979     0.80591      0.8129

See Also

runperf | testsuite | matlab.perftest.TimeExperiment | matlab.perftest.TimeResult

Topics