Define Requirements for Transmission System Fault Detection

R2026b

Since R2026b

This example shows how to define requirements for an end-to-end workflow for fault detection in an automotive transmission system. Requirements are organized into data and model requirements, with each requirement addressing an important aspect of deploying machine learning models in safety-critical automotive applications.

Importance of Requirements in Safety-Critical Applications

In automotive systems, an undetected transmission fault can lead to vehicle breakdown, loss of control, or cascading damage to the drivetrain. Missed faults can lead to unsafe operation. Meanwhile, false alarms can erode operator trust and trigger unnecessary maintenance shutdowns. Machine learning models introduce unique challenges for safety assurance. For example, a model that works in MATLAB^® might behave differently after code generation or integration into Simulink^® due to numerical precision, data type conversion, or architectural translation. Formal requirements mitigate safety issues by providing the following:

Clear pass/fail criteria that are agreed upon before model training begins
Traceability from system-level safety goals down to specific test assertions, enabling auditors and certification bodies to verify that every safety concern is addressed
A mechanism for catching regressions automatically when a model is retrained with new data or the deployment pipeline changes

Fault and Failure Scenarios

Faults are the conditions that a system is designed to detect. For each fault type, a trained classifier returns a binary output that indicates whether a fault is present. Failures are degraded operating conditions that can occur along with faults but are not detection targets. They affect the signal environment, and classifiers must remain robust in their presence. However, no classifier is trained to detect failures directly.

In this example, the transmission system has two possible target faults and one possible failure:

Sensor drift fault — Progressive offset in the vibration sensor reading
Shaft wear fault — Mechanical degradation of the output shaft (increased damping and friction)
Gear tooth failure — Failure mode that changes the vibration and tacho signal characteristics, used to ensure the fault detectors work under realistic drivetrain degradation

These faults and failures combine to form eight simulation scenarios:

Scenario 1: Healthy (no faults, no failure)
Scenario 2: Sensor drift fault
Scenario 3: Shaft wear fault
Scenario 4: Sensor drift and shaft wear faults
Scenario 5: Gear tooth failure
Scenario 6: Sensor drift fault and gear tooth failure
Scenario 7: Shaft wear fault and gear tooth failure
Scenario 8: Sensor drift fault, shaft wear fault, and gear tooth failure

This design ensures that the sensor drift and shaft wear detectors are validated under both normal and degraded drivetrain conditions.

Signal Data Requirements

The signal data requirements pertain to the raw vibration and tacho signals generated by the transmission system Simulink model.

Requirement Type	Requirement	Description
Completeness	Data completeness	Each expected operating scenario (1-8) must have at least 200 observations
Validity	Signal validity	Vibration signals must range between [-3, 3]

Completeness — To be trustworthy, a fault detection classifier must be trained and evaluated on data that covers all expected operating scenarios. Completeness requirements ensure that sufficient observations exist for every simulation scenario.
Validity — Signals must be physically plausible. Training on invalid data can embed artifacts into a model, and evaluating on invalid data can produce misleading accuracy metrics.

Extracted Tabular Data Requirements

The extracted tabular data requirements pertain to the data returned after processing the raw signals and extracting features. The raw signals are converted into 19 signal-derived features (statistical, spectral, and rotational) and a variable indicating from which simulation scenario each observation is extracted.

Requirement Type	Requirement	Description
Completeness	Training data completeness	Training set contains sufficient observations from each simulation scenario (at least 140)
Completeness	Test data completeness	Test set contains sufficient observations from each simulation scenario (at least 60)
Feature count	Training data feature count	Training set contains exactly 20 extracted features
Feature count	Test data feature count	Test set contains exactly 20 extracted features

Completeness — Data completeness must be verified again because the feature extraction process can result in the loss of segments of data and data partitioning can inadvertently concentrate a rare scenario entirely in the test set. If any scenario is missing from the training data, classifiers cannot learn to detect faults under those conditions. Meanwhile, accuracy metrics are only meaningful if the test set includes all operating scenarios.
Feature count — Both the training and test sets must contain the full set of features so that feature selection and model evaluation are performed on consistent inputs. The variable containing the scenario ID is excluded from model training but retained for requirements verification.

Model Fit Requirements

The model fit requirements define the acceptance criteria for the two trained classifiers (sensor drift detector and shaft wear detector) evaluated in MATLAB on the test set.

Requirement Type	Requirement	Description
Accuracy	Sensor drift accuracy	Sensor drift detector accuracy > 92%
Accuracy	Shaft wear accuracy	Shaft wear detector accuracy > 92%
True Positive Rate (TPR)	Sensor drift TPR	Sensor drift detector TPR > 90%
True Positive Rate (TPR)	Shaft wear TPR	Shaft wear detector TPR > 90%
False Positive Rate (FPR)	Sensor drift FPR	Sensor drift detector FPR < 5%
False Positive Rate (FPR)	Shaft wear FPR	Shaft wear detector FPR < 5%
Memory footprint	Memory footprint	Estimated model memory < 500 KB

Accuracy — Accuracy provides an overall measure of correct classification. The 92% threshold ensures the models perform much better than chance (50% in case of binary classification).
TPR or sensitivity — In a safety-critical application, missing a real fault is more dangerous than raising a false alarm. Setting a high true positive rate threshold prevents high accuracy from being achieved at the cost of classifying borderline scenarios as healthy (scenario 1).
FPR — While missed faults are the primary safety concern, excessive false positives have operational costs. The 5% threshold balances safety with operational usability.
Memory footprint — You cannot deploy a model that is accurate but exceeds the available memory budget. To prevent investing in integration work for a model that will not fit on the target hardware, check the estimated model memory during the development stage.

Simulink Integration Requirements

The integration model requirements verify that the models behave correctly after they are integrated into a Simulink environment using blocks like ClassificationTree Predict (Statistics and Machine Learning Toolbox) or MATLAB Function (Simulink).

Requirement Type	Requirement	Description
Model equivalence	Sensor drift model equivalence	Sensor drift predictions in Simulink match MATLAB
Model equivalence	Shaft wear model equivalence	Shaft wear predictions in Simulink match MATLAB
Inference speed	Sensor drift inference speed	Sensor drift inference for test set < 10 seconds
Inference speed	Shaft wear inference speed	Shaft wear inference for test set < 10 seconds

Model equivalence — For each classifier, the same test data is run through both the original MATLAB model and its Simulink integration, to check that predictions match exactly. Any mismatch indicates a translation error that must be resolved before you can proceed to code generation.
Inference speed — The Simulink models must process predictions within real-time constraints. If the models cannot meet timing requirements in simulation, they are unlikely to do so on resource-constrained targets.

Code Generation Inference Requirements

The inference model requirements apply after code generation, when you need to verify the compiled C/C++ implementation in software-in-the-loop (SIL) mode.

Requirement Type	Requirement	Description
Model equivalence	Sensor drift model equivalence	Sensor drift predictions in SIL match MATLAB
Model equivalence	Shaft wear model equivalence	Shaft wear predictions in SIL match MATLAB
Inference speed	Sensor drift inference speed	Sensor drift inference for test set < 1 second
Inference speed	Shaft wear inference speed	Shaft wear inference for test set < 1 second

Model equivalence — For each classifier, SIL testing executes the generated C/C++ code on the host machine and compares the output against the original MATLAB model. The deployed model must behave like the validated MATLAB model.
Inference speed — The timing measurements provided by SIL are better at estimating actual deployment performance than those provided through Simulink simulation. If the generated code meets the 1-second inference threshold for the test set in SIL, then it is efficient enough for the target sampling rate.

Summary

The requirements defined above establish a complete verification framework that spans the machine learning development lifecycle. For each step in the Transmission System Fault Detection Using Tree-Based Machine Learning Models example, you must verify that the outputs adhere to the relevant requirements, creating a traceable chain from raw data to deployed code.