Online and Recursive System Identification | System Identification, Part 4 - MATLAB
Video Player is loading.
Current Time 0:00
Duration 18:47
Loaded: 0%
Stream Type LIVE
Remaining Time 18:47
 
1x
  • Chapters
  • descriptions off, selected
  • en (Main), selected
    Video length is 18:47

    Online and Recursive System Identification | System Identification, Part 4

    From the series: System Identification

    Online system identification algorithms estimate the parameters and states of a model as new data is measured and available in real-time or near real-time. Brian Douglas covers what online system identification is, why it’s a good option for real-time situations, and the general ideas behind how it works. Due to the real-time nature of these algorithms, computational time and resources play a significant role. Therefore, all of the fancy nonlinear regressors, and nonlinear output functions, and optimization algorithms shown in the previous videos in this series might produce a good answer, but not necessarily one that can be completed during real-time operation.

    Published: 3 Jan 2022

    So far in this series, we've talked about so-called offline system identification methods. That is identifying a model using data that was previously collected from the system, and then used all at once to estimate the model. And with these methods, the amount of time it takes to get to a solution isn't critically important.

    You could take your input and output data, kick off the learning algorithm, and then go have some tea and relax a little bit while it runs. And whenever it finishes, you'll have an answer.

    Online system identification basically means that the algorithm is running during the operation of the physical system. And these algorithms estimate the parameters in states of a model as new data is measured and available in real-time or near real-time. And the sort of relaxed approach that we did with the offline methods doesn't really work in these situations, because usually computational time and resources play a much larger part. So all of that fancy nonlinear regressors and nonlinear output functions and optimization algorithms that we did in the last video might produce a good answer, but not necessarily one that can be completed during real-time operation.

    So in this video, we're going to talk about what online system identification is; why it's a good option for real time situations; and the general idea behind how it works. So I hope you stick around for it. I'm Brian, and welcome to a MATLAB Tech Talk.

    Let's describe online system identification using the example of a thermostat that uses a thermodynamic model of a house in order to best control the temperature. And if the thermostat isn't provided a model, it has to build one based on the available information. For example, it could use the current inputs into the system, which are the control inputs like command and fan speed and compressor working fluid temperature, as well as other inputs like the outside temperature. And it can use the current output of the system, which is the measured temperature inside the house. And it can use all of that to develop a thermodynamic model over time.

    It does that by updating the model parameters each time step. And here's one way to do that. The algorithm could take the measured house temperature, y, and compare it to the predicted house temperature, y hat, to produce a prediction error. The predicted temperature comes from its internal model, which we'll say is a linear model in the following form.

    Where if you remember from the second video, psi is a vector of the linear aggressors, and theta hat contains the estimated model parameters. And at the beginning of operation, theta is just initialized to some default values. But if there's a difference between the measured and predicted output, then the model parameters aren't correct and they need to be updated. And an online algorithm would adjust the model parameters in some way to reduce that error.

    And one way is simply to use a gradient method where an adaptation gain K scales the error and then adds it to the old parameters to get a new set of estimated parameters. And then in the next time step, a new set of measurements come in, which are then compared to the new prediction from the model. And that error is then used to update the parameters again. And it keeps doing this until it converges on the model.

    This is a recursive infinite history algorithm, which we're going to get into in a little bit what that means. But for now, all I want to illustrate is how, with online methods, the model is developed over time using the data as it's available.

    All right so, why would we want to solve system identification online one data point at a time rather than offline and just all at once? Well, as you might imagine, there's lots of reasons. And I'm not going to get into most of them in this video.

    But the one reason that I do want to go into detail on is that online system identification is useful if the system changes over time, or the parameters of the model are time varying, and you want the model to constantly adapt to it or to recognize those changes. With the temperature control example, the house thermodynamics can change from weekday to weekends based on whether people are home and opening windows and doors, or throughout the year as the seasons change, or when more energy-efficient insulation is installed, and any number of things that would cause a change in the way heat flows into and out of a house.

    So for control purposes, by maintaining an accurate model of the house as the parameters vary, a temperature control system can be developed that uses that model to more effectively maintain the desired temperature. This is a form of adaptive control. And adaptive control and other methods that use system ID, like model predictive control, are absolutely awesome. And I've left some links to some MATLAB examples that you can play around with in the description of this video.

    But perhaps my favorite use case for online system identification is for fault detection, because it just makes so much sense to me. With a model that is adapting to changes in the system, we can create fault tests that look at the parameters of the model rather than the measurements of the system to determine if a fault has occurred. And it can work like this.

    Imagine we have a DC motor where we can apply a voltage to the motor which changes the angle of the motor shaft. And we can wrap a control system around it to control the angle and speed. Now, over time, the lubricants and the bearings might degrade, causing more friction in the motor, which would then require more current or a higher voltage to achieve the same spin rate. The controller will compensate for this increased friction, and the motor might still spin as commanded.

    However, this might not be ideal, and so we want to detect this high-friction condition and trigger a fault response. Now, instead of setting up a fault test that looks at current draw and motor speed and motor acceleration and then tries to come up with a threshold that indicates a high-friction scenario, we could approach this with online system identification. We could estimate the parameters of the motor model itself, which, if we set it up to do so, could have a parameter for the motor viscous friction constant.

    Now, instead of tracking current and trying to figure out some complex fault condition, we can just check if the friction constant is larger than some maximum threshold. And no matter what speed we're running the motor or how it's accelerating, we have a direct estimate of the parameter that we care about. And I think that's pretty awesome because estimating model parameters rather than looking only at the measured data could give you a lot more insight into the properties of the system that you care most about.

    And we can't do this kind of real-time adaptive control or real-time fault detection with offline system identification. Well, at least not for systems that change quickly relative to how long it takes the offline algorithms to run. If your system is slow enough, like some chemical processes, then collecting a bunch of data once a day and then just running offline identification is probably sufficient. But for fast-changing systems, we need online algorithms.

    All right, so with that behind us, let's talk a little bit about what these online algorithms are. And I already mentioned that we can't really approach them exactly the same way that we did with the offline methods because of timing concerns. And a lot of this comes down to the fact that those offline models, like the nonlinear ARX, are often non-convex and therefore require slower optimization solvers that iterate towards the solution over time.

    So to get around this, we're going to forego those more complex models and algorithms and limit ourselves to models that can be solved with just a few matrix operations without having to iterate on the solution. And we're going to look at two different ways that we can use the online data to update these linear models-- finite-history algorithms and infinite-history algorithms.

    Finite-history means that we're only using part of the past data. And, to me, these types of algorithms are the most intuitive because they just seem to make so much sense. When a real-time system is running, it's generating a continuous stream of data. And it doesn't make sense to try to store all of that data and use all of it to develop a model; that would get overwhelming quickly.

    So instead, we just store a fixed number of past data points in a buffer. And we would store both the input and output data and anything that's needed to identify a model. And when a new data point comes in, it just pushes out the old one so that the window is always the same length.

    Now, at each time step, we use this entire buffer of data to estimate a linear model. And in this way, this is very similar to linear offline methods because we're fitting a linear model to a complete set of data all at once by minimizing a least squares problem. And it just so happens that this data is collected real-time from the system; and we're limiting ourselves to just the last in samples.

    And one thing that makes finite-history algorithms a good choice for rapidly-changing systems is that we only ever look at and consider the latest data when fitting a model. We move that window of buffer data forward in time and therefore only based the updated model on the most recent data. So if the system is in one state in the past, and we estimate a model of the system, then, later in the future, when the system has transitioned to a new state, none of that past data will influence the new model.

    Now, a downside of finite-history algorithms is that we need to store all of that data. And when we perform the identification step, we have to compute a model using all of that data. And this can be potentially time-consuming and resource-intensive if the window needs to be really large to capture the essential dynamics of the system.

    On the other hand, recursive infinite-history algorithms don't typically have that problem. Infinite-history algorithms use, in some small part, all of the past data to estimate the model. And this gets us back to the equation that we created for the smart thermostat from earlier, where the new model parameters are a function of the old parameters plus only the latest set of measurements.

    These methods are typically much faster to run and require much less stored data because only the most recent data point is used to update the result and not an entire buffer's worth of data. In this way every, past data point has impacted the solution in some way, since this is a recursive algorithm. And therefore, the result is always a combination of all of the past values-- hence, infinite-history.

    Oh, and by the way, just as a side note, this is very much analogous to finite impulse response filters and infinite impulse response filters in how they each use past data points. So if you're familiar with them, then this concept shouldn't be a surprise.

    OK. So we have this general form of the infinite-history recursive algorithm, but how do we set it up? That is, how do we choose the adaptation gain so that the parameters are updated appropriately based on the prediction error each time step? And one way is to just set the gain equation to a fixed value, Q, multiplied by the regression vector.

    With this, each time step-- the error between the prediction and the measurement-- is scaled by a fixed amount proportional to the gradient of the linear system. And then that is added to the old parameters. And over time, the parameters will converge on the solution since it's descending the gradient, assuming the gain isn't so large as to cause an instability, and not too small as to not be able to keep up with the time-varying parameters. In this-- the linear regression case-- this method is also known as the least mean squares method.

    And there are actually multiple flavors of this approach. I just described the unnormalized gradient method, but there's also a normalized method that scales the adaptation gained by the square of the 2 norm of the gradient vector. And if you're interested in learning more, check out this document that I reference down below.

    OK, if picking a fixed gain seems arbitrary and not optimal enough for you, well, then we can approach this in a more optimal way. Consider the measurements y generated by the actual system, which we're assuming is linear, plus some measurement noise assumed to be Gaussian white noise. In this way, we can't trust the measurements 100% because the real output of the system is corrupted with some measurement noise.

    In addition to that, the true parameters of the system aren't static. They vary over time. And let's say that the parameters of the system at time t is a function of the parameters at time t minus 1 plus some random value. Again, this random value is assumed to be Gaussian, which means that the parameters are kind of on a random walk over time.

    All right, so now here's where it gets interesting. When a new measurement comes in, we want to know how much to update the old parameters. What should this gain be?

    Well, we have the measurement itself, which might be corrupted with noise. And we have a sense, at least statistically, of how much the parameters could move in one time step. So if the measured y is drastically different than the predicted y, we might be inclined to believe the measurement and want a large Q value to correct the parameters.

    However, if we know that there is a lot of measurement noise, then maybe we don't want to believe the measurement as much, and only move the parameters a little bit, despite the error. So a small Q value. Yet if we know that the parameters in the system can move quite a bit each sample time, then maybe that large difference between the measurement and the prediction can be trusted more. And we want a Q value somewhere in the middle.

    And this problem of trying to balance two uncertain elements is starting to look a lot like what a Kalman filter does, right? In fact, we can actually just use a Kalman filter to solve this problem. A Kalman filter estimates the state of the following standard format linear system, which if we look closely, we can see that they exactly match our measurement and parameter models if F of t is set to the identity matrix and H of t is the regression vector.

    So now to find the gain Q of t, we can use the Kalman filter to solve for the optimal gain, where P is the prediction covariance, R2 is the measurement covariance, and R1 is the process covariance. And note, that even though this is significantly more complex than a fixed gain approach, it's still something we can solve for using a relatively small number of matrix operations, which is blazing fast compared to the more iterative optimization methods that we used in offline system identification.

    This is why recursive approaches like this are so well-suited for real-time operations. And over time, one sample at a time, this will converge on the optimal model parameters. And by the way, if you want to learn more about the Kalman filter, the assumptions under which it was derived and all of that, check out the MATLAB Tech Talk series on them that I've linked to in the description of this video.

    All right, you might notice that this video was heavy on the general concepts and light on the specific mathematics or applications. Plus, I didn't even cover many other popular online methods, like adding a forgetting factor into the infinite history algorithms so that old measurements are discounted exponentially, leaving just the newest measurements, influencing the optimal result more heavily. Well, this is because there's always too much to cover in short videos like this.

    The goal here is to just hopefully give you some baseline knowledge and motivation to go and learn the details of the mathematics and to go and try out an application yourself. And to do that, I recommend checking out all of the references below, especially chapter 11 of Dr. Lennart Ljung's book that covers recursive estimation methods and the MATLAB and Simulink examples that demonstrate each of these concepts.

    Specifically, check out this Simulink example that shows how you can use online recursive least squares estimation to detect a change in an engine's inertia. And if we jump over to Simulink, I can show you some of the highlights of this example. The engine is nonlinear. And this portion of the model here represents the true dynamics of the system.

    And at about 300 seconds, the engine inertia changes from a little over 7 to just over 2 kilograms meter squared. And this is what we want to detect. This second portion is where the motor model is identified. And this is done using just the throttle position and the motor speed's input, which are used to make the progression vector, which is then used to estimate the parameters of a second order polynomial model.

    And inside this block, you can see that I'm using the Kalman filter to estimate the model parameters. The process noise covariance and the parameter covariance matrix are both specified. Now, this model is set up such that the first parameter, A1, is proportional to the motor inertia.

    And if we look at this scope, you can see that it takes 100 seconds or so for the filter to converge. And then at 300 seconds, it becomes very obvious that the inertia has changed. And what's cool about this whole thing is, like I said, the entire system identification algorithm is just packaging the regressors, some really basic matrix operations, and then some function that determines if a parameter has surpassed a threshold. So we could test these algorithms in simulation like this and then generate code that can run on relatively low-power microcontrollers.

    And by the way, I used a Kalman filter here, since that's what we talked about in the video. But if you run the default example, you'll see that it uses the forgetting factor algorithm, which converges much faster-- in about 50 seconds or so initially and then just about 15 seconds when the inertia changes. And this is because it's discounting past measurements and only looking at about the last 15 seconds or so in this particular example. So this is a good example to check out if you'd like to see what we talked about in this video in action.

    All right, so that's where I'm going to leave this video. And if you don't want to miss any future Tech Talk videos, don't forget to subscribe to this channel. And if you want to check out my channel, Control System Lectures, I cover more control theory topics there as well. Thanks for watching, and I'll see you next time.

    View more related videos