Model Coverage for Variant Blocks - MATLAB & Simulink (original) (raw)

Simulink® Coverage™ can analyze and report coverage data for models containing variant blocks, such asVariant Source or Variant Subsystem blocks. When you simulate a model containing one or more variant blocks with coverage enabled, Simulink Coverage reports the coverage results depending on the type of variant block and the**Variant activation time** parameter.

For Variant Source and Variant Sink blocks, Simulink Coverage analyzes the upstream and downstream blocks of the variant choices, respectively. The Variant Source and Variant Sink block itself does not receive coverage. For Variant Subsystem and Variant Model blocks,Simulink Coverage analyzes the contents of the subsystem or model blocks for the selected metrics.

Update-Time and Compile-Time Variants

If you set the Variant activation time parameter of a variant block to update diagram, update diagram analyze all choices, or code compile Simulink Coverage reports coverage only for the variant choice that is active during the simulation.

Tip

You cannot aggregate coverage for simulations with different active variant choices for update-time or compile-time variants. If you want to test inactive variant choices, setVariant activation time to startup orruntime.

For example, suppose you have a model that contains a Variant Subsystem block with two variant choices. The Variant activation time parameter of the Variant Subsystem block is set to update diagram.

Simulink model containing a variant subsystem. A graphic shows that the variant subsystem expands into two variant choices.

If Subsystem1 is active for the simulation, you receive coverage results for Subsystem1 and not Subsystem2.

In the coverage results, Subsystem1 is red, which indicates that it received incomplete coverage, and Subsystem2 is grey which indicates that it was not analyzed. Additionally, the coverage report summary omits the variant choices that were not active.

Coverage report summary showing Subsystem 1 has 50% decision and 100% block execution coverage. Subsystem 2 is omitted from the summary.

When you set the Variant activation time parameter of a variant block to update diagram, update diagram analyze all choices, or code compile, changing the active variant is a structural change to the model. Because the coverage data for a model is only valid as long as the model is not changed or closed, you cannot aggregate coverage for the same model simulated with different active variants. If you want to test multiple active variants and aggregate coverage for them, change theVariant activation time parameter to startup or runtime.

Startup and Run-Time Variants

If you set the Variant activation time parameter of a variant block to startup or runtime, Simulink Coverage reports coverage for all variant choices, even if they are not active during the simulation. You can also aggregate coverage for simulations with different active variant choices.

For example, if you modify the model in the previous example and set the Variant activation time to startup andSubsystem1 is active for the simulation, you get the same coverage forSubsystem1 as the case with the update-time variant. However, Simulink Coverage also reports on the inactive variant, Subsystem2.

In the coverage results, Subsystem1 and Subsystem2 are both red which indicates incomplete coverage. The coverage report summary includes both variant choices.

Coverage report summary showing the top model has 30% decision coverage, the variant subsystem has 25% decision coverage, the variant child Subsystem 1 has 50% decision coverage, and the variant child Subsystem 2 has 0% decision coverage.

The inactive variants receive 0% coverage because they were not executed during the simulation. To increase the reported coverage for Variant_Subsystem, you can simulate the model again with Subsystem2 as the active variant and then aggregate the coverage results.

Aggregated Tests and Summary sections of the coverage report. Coverage results from two simulations aggregated together show the top model has 60% decision coverage, the variant subsystem has 50% decision coverage, the variant child Subsystem 1 has 50% decision coverage, and the variant child Subsystem 2 has 50% decision coverage.

Customizing the Coverage Report for Models that Contain Variants

When creating a coverage report for a model that uses variant blocks, you can aggregate coverage data from multiple simulations. For startup variants, you can specify whether to include or exclude inactive variants.

Exclude Inactive Variants from the Coverage Report

Inactive configurations of update-time and compile-time Simulink variants are excluded from the coverage report.

For Simulink variant blocks that have the Variant activation time parameter set to startup or runtime and variant configurations in Stateflow® charts, the coverage report includes inactive variant choices by default. For startup variants, you can choose to exclude the inactive variant choices in the coverage report by selecting the Exclude inactive choices of variants parameter:

In the Configuration Parameters window, in the left pane, clickCoverage.
Expand Advanced parameters and then select Exclude inactive choices of variants.

You can also set this parameter programmatically by entering:

set_param(modelName,'CovExcludeInactiveVariants',1)

If you are using the Test Manager in Simulink Test™, you can select Exclude inactive variants in the coverage settings at the test file level. For more information, see Coverage Settings (Simulink Test).

Note

Runtime variants ignore the value of the Exclude inactive choices of variants parameter and report coverage for inactive variant configurations.

You can also change the reporting behavior of startup Simulink variants and variant configurations in Stateflow charts after your simulation completes using one of these methods:

In the Coverage Results Explorer, select or clear the option Exclude inactive choices of variants.
Change the value of the [excludeInactiveVariants](../ref/cvdata.html#mw%5Fb89a73e7-6c73-410f-82bd-8f9a69066377) property in the cvdata object before generating the coverage report.
If you are using the Test Manager in Simulink Test, in the Test Browser, at the results summary level, underAggregated coverage results, select Exclude inactive variants.

Aggregate Coverage Data for Models that Contain Variants

There are two ways to aggregate coverage data for models that contain Simulink variants and variant configurations in Stateflow charts:

In the Configuration Parameters dialog box, click Coverage, then expand Advanced parameters and select the Include cumulative data in coverage report parameter. When you run two or more simulations with this parameter selected, the Coverage Details shows the aggregated coverage results of those simulations.
Run coverage analysis programmatically and aggregate the cvdata objects by using the + operator.

If you aggregate two cvdata objects that have different values for the excludeInactiveVariants property, either by using the+ operator or by changing the Exclude inactive choices of variants parameter, the resulting aggregated cvdata object has this property set to a value of 0. The result is that the aggregated coverage report shows inactive variants as if the Exclude inactive choices of variants parameter is not selected, even if the parameter was selected for one simulation.