Monitor Drift Using Detect Drift Block

Since R2024b

This example shows how to use the Detect Drift block for monitoring drift in a data stream in Simulink®. The block detects drift in an input datastream using a configured incremental concept drift detector (incrementalConceptDriftDetector) and outputs the drift status. The sample reset logic provided with the example enables the block to reset the internal statistics of the drift detector upon detecting a drift in the data.

Create Data Stream

Create a random data stream where the first 1000 observations come from a normal distribution with mean 2 and standard deviation 0.75, and the next 2000 come from a normal distribution with mean 4 and standard deviation 1. In a typical incremental drift detection application, accessing a data stream and updating the model happen consecutively. You usually do not collect the data first and then feed it into the model. However, for the purpose of clarification, this example demonstrates the simulation of data separately.

rng(1234) % For reproducibility
numObservations = 3000;
switchPeriod = 1000;
X = zeros([numObservations 1]);

for i = 1:numObservations
   if i <= switchPeriod
      X(i) = normrnd(2,0.75);
   else
      X(i) = normrnd(4,1);
   end
end

Initiate an incremental concept drift detector using the Hoeffding's bound method with exponentially weighted moving average method (EWMA). Specify the input type as continuous, a warm-up of 50 observations, and an estimation period of 50 observations. For more information, see incrementalConceptDriftDetector.

IncCDDetector = incrementalConceptDriftDetector("hddmw",InputType="continuous", ...
          WarmupPeriod=50,EstimationPeriod=50)

IncCDDetector = 
  HoeffdingDriftDetectionMethod

        PreviousDriftStatus: 'Stable'
                DriftStatus: 'Stable'
                     IsWarm: 0
    NumTrainingObservations: 0
                Alternative: 'greater'
                  InputType: 'continuous'
                 TestMethod: 'ewma'

IncDDetector is a HoeffdingDriftDetectionMethod object. When you first create the object, properties such as DriftStatus, IsWarm, and NumTrainingObservations are set to their initial states.

Open Provided Simulink Model

This example provides the Simulink model slexDetectDriftExample.slx, which includes the Detect Drift block. It also contains a sample reset logic that enables the block to reset the drift detector. You can open the Simulink model or create a new model as described later.

Open the Simulink model slexDetectDriftExample.slx.

SimMdlName = "slexDetectDriftExample"; 
open_system(SimMdlName)

When you open the Simulink model, the software runs the code in the PreLoadFcn callback function before loading the Simulink model. The PreLoadFcn callback function of slexDetectDriftExample includes code to check if your workspace contains the IncCDDetector variable for the incremental concept drift detector. If the workspace does not contain the variable, PreLoadFcn loads the sample data, initializes the incremental concept drift detector, and creates an input signal for the Simulink model. To view the callback function, in the Setup section on the Modeling tab, click Model Settings and select Model Properties. Then, on the Callbacks tab, select the PreLoadFcn callback function in the Model callbacks pane.

Create Simulink Model

To create a new Simulink model, open the Blank Model template and add the Detect Drift block from the Drift Detection section of the Statistics and Machine Learning Toolbox™ library. Add the Inport and Outport blocks and connect them to the Detect Drift block. Add your custom reset logic for the reset port.

Double-click the Detect Drift block to open the Block Parameters dialog box. Specify the name of the workspace variable that contains the incremental concept drift detector. The default variable name is IncCDDetector. Click the Refresh button. The dialog box displays the options used to initialize the incremental concept drift detector IncCDDetector under Concept Drift Detector.

At the command line, create an input signal in the form of a structure array for the Simulink model. The structure array must contain these fields:

• time — The points in time at which the observations enter the detector. In this example, the duration includes the integers from 0 through numObservations - 1, where numObservations is the number of samples in the input data. The orientation must correspond to the observations in the data. So, in this case, time must be a column vector.

• signals — A 1-by-1 structure array describing the input data and containing the fields values and dimensions, where values is a matrix of the data, and dimensions is the data dimension of X.

[numObservations,p] = size(X);
inputStruct.time = (1:numObservations)'- 1;
inputStruct.signals(1).values = X;
inputStruct.signals(1).dimensions = p;

Import the signal data from the workspace:

• Open the Configuration Parameters dialog box in Simulink. In the Setup section of the Modeling tab, click the top half of the Model Settings button.

• In the Data Import/Export pane, select the Input check box and enter inputStruct in the adjacent text box.

• In the Solver pane, under Simulation time, set Stop time to inputStruct.time(end). Under Solver selection, set Type to Fixed-step, and set Solver to discrete (no continuous states). Under Solver details, set Fixed-step size to 1. These settings enable the model to run the simulation for each sample in inputStruct.

For more details, see Load Signal Data for Simulation (Simulink) (Simulink).

The Detect Drift block resets the drift detector when the status is 2, which means drift is detected by the Reset Logic subsystem. The sample reset logic provided with the example accepts the status signal as input. This signal is converted to a double, and the memory block delays the signal by one time-step. The Relational Operator block equates the status signal to 2. If the status is 2, the Switch block outputs a true signal (1). You can modify the reset logic to suit the needs of your application.

Simulate Model

Next, simulate the Simulink model. When the Inport block detects an observation, it directs the observation into the Detect Drift block. You can use the Simulation Data Inspector (Simulink) (Simulink) to view the logged data of the Outport blocks. Export the simulation outputs to the workspace in a variable simOut.

simOut = sim(SimMdlName);
isWarm = simOut.yout{1}.Values.Data;
status = simOut.yout{2}.Values.Data;

Plot the simulation outputs IsWarm and Status.

figure
tiledlayout(2,1);
nexttile
plot(isWarm,".-")
ylabel("IsWarm")
xlabel("Observations")
xlim([0 numObservations])
nexttile
plot(status,".-")
ylabel("Status")
xlabel("Observations")
xlim([0 numObservations])

For the first 100 observations, isWarm is false. During this period, the detector estimates the input bounds for 50 observations, and then it warms up for the next 50 observations. In the warm-up period, the detector uses the observations to update the internal statistics, but does not check the drift status. After 100 observations, the detector is warmed up and isWarm is true. The status is 0 (stable) for 1018 observations, and then changes to 1 (warning). After 1022 observations, the status is 2 (drift detected). At this point, the drift detector is reset and the detector warms up again. The status for the remaining observations is 0, showing no further drift detection.