Evaluate Differences Between Model and Code Coverage

This example shows how to use Simulink® Test™ Manager to analyze model coverage and generated code coverage, and investigate differences in the results. For information about how to analyze code coverage without using Simulink Test, see Collect Code Coverage Metrics with Simulink Coverage and Software-in-the-Loop Code Coverage.

When you analyze code coverage, that the code coverage results may not exactly match the model coverage results.

Although the goal of Embedded Coder is generating code that behaves the same as the model, Simulink® models and C/C++ code use fundamentally different levels of abstraction. Simulink models are high-level representations of the desired behavior, but C and C++ are low-level programming languages. Due to this difference, the way that the behavior is implemented is different, and could result in a different structure.

The model and code coverage results may differ because:

• Statement coverage is not the same as block execution coverage. For example, Stateflow® charts and transitions do not receive block execution coverage, but the code generated from them does receive statement coverage. Additionally, because the implementation for a block often contains multiple code statements, one block execution outcome often translates to multiple statements in the generated code.

• The saturate-on-integer overflow metric in model coverage becomes a decision outcome in code coverage.

• A function call inside a MATLAB® Function block reports a decision for model coverage, but reports function call coverage for code coverage.

• Embedded Coder tries to optimize the generated code to eliminate execution branches that are unnecessary. If your model contains dead logic, Embedded Coder may remove that logic from the generated code.

• By default, Embedded Coder performs expression folding optimizations, in which multiple decisions can be folded into one, more complex decision. This optimization can cause differences in decision, condition, and MCDC coverage. For more information, see Minimize Computations and Storage for Intermediate Results at Block Outputs (Embedded Coder).

Set Up Coverage

This example model CruseControl.slx is a Stateflow chart that implements simple cruise control logic.

This example contains a Simulink® Test™ test file that contains a suite of test cases that achieve 100% model coverage, and then uses those test cases to analyze code coverage using software-in-the-loop (SIL) simulation mode.

Open the Simulink Test Manager and load the test cases.

sltestmgr
testFile = sltest.testmanager.load("CruiseControl_Tests.mldatx");

Run Test Suite

Run the test suite by using sltest.testmanager.run. Providing no inputs runs all the loaded tests.

testFile.run();

View Coverage Results in Test Manager

In the Test Manager, click Results and Artifacts. Expand the results, then select CruiseControl_Tests > CruiseControl_TestSuite > Model-in-the-loop Tests. Then, under Aggregated Coverage Results, create the model coverage report by clicking the create aggregated coverage report icon . In this set of tests, model coverage reports 100% coverage for decision coverage, condition coverage, and MCDC.

In the left pane, select CruiseControl_Tests > CruiseControl_TestSuite > Software-in-the-loop Tests. Under Aggregated Coverage Results, create the code coverage report by clicking the create aggregated coverage report icon . These tests use the same test cases and test inputs, but code coverage reports 97% decision coverage, 96% condition coverage, and 91% MCDC. To understand why this happened, you need to investigate the coverage report.

Review Code Coverage Report

Look at the model with coverage results highlighting enabled. In the Test Manager, in the results for Software-in-the-loop Tests, under Aggregated Coverage Results, click the name of the analyzed model, CruiseControl to open the model, the Coverage Details pane, and highlight the model with the coverage results. Enter the Compute target speed chart by double-clicking the chart.

In the Coverage Details pane, scroll to the Details section. The first section of the Details section shows the coverage results for the entire system. In the model coverage report, it is for the model CruiseControl, and in the code coverage report, which is the report that the Coverage Details pane currently displays, it is for the file CruiseControl.c. The model coverage and code coverage reports display these coverage results, respectively:

Coverage Metric	Model Coverage Results	Code Coverage Results
Decision	100% (38/38 outcomes)	97% (38/39 outcomes)
Condition	100% (32/32 outcomes)	96% (69/72 outcomes)
MCDC	100% (16/16 outcomes)	91% (20/22 outcomes)
Execution	N/A	N/A
Statement	N/A	100% (89/89 statements)
Function	N/A	100% (5/5 functions)
Function Call	N/A	100% (3/3 function calls)

By comparing the results, you can see that decision, condition, and MCDC coverage have more outcomes in the code coverage report.

Review Specific Differences

In the Simulink canvas, there are two transitions that are missing coverage. Click the lower transition, [hasChangedTo(CoastSetSw,true)]. In the model coverage report, you would see one analyzed decision with two possible outcomes, which is the overall decision that determines whether to take this transition. However, in the code coverage report, you see one analyzed decision with two possible outcomes, two analyzed conditions each with two possible outcomes, and two MCDC outcomes.

Clicking on the transition line when the model is highlighted with code coverage results brings you to the code section of the code coverage report. If you scroll up to the Details section, you can see the coverage tables. In this case, section 3.18 is the part relevant to this transition.

The Conditions analyzed and MC/DC analysis tables illustrate why there are new outcomes in the code coverage report. The hasChangedTo operator reports a decision in model coverage, but Embedded Coder® translates the transition into a complex expression that contains multiple conditions, so it also receives condition and MCDC coverage:

else if ((CruiseControl_DW.CoastSetSw_prev != CruiseControl_DW.CoastSetSw_start) && CruiseControl_DW.CoastSetSw_start) {...}

Resolve Missing Code Coverage

In this example, the code coverage analysis does not have 100% coverage because generating code created additional coverage outcomes that need to be satisfied. To achieve 100% coverage, you can:

• Add new test cases to reach the logic that current test cases do not exercise. You could use Simulink® Design Verifier™ to generate test cases to satisfy missing coverage.

• Justify coverage outcomes if one or more outcomes are unreachable, or if the test case requires excessive machine resources.

As a best practice, satisfy missing model coverage before analyzing code coverage. For example, if your model is missing decision coverage in the model coverage report, then the code coverage report for the generated code also contains missing statement coverage. Additionally, you can reuse test cases that you use to analyze model coverage for code coverage analysis, which enables you to verify their behavioral equivalence.