idGaussianProcess

Gaussian process regression mapping function for nonlinear ARX and Hammerstein-Wiener models (requires Statistics and Machine Learning Toolbox)

Since R2021b

Description

An idGaussianProcess object implements a Gaussian process (GP) regression model, and is a nonlinear mapping function for estimating nonlinear ARX and Hammerstein-Wiener models. This mapping object, which is also referred to as a nonlinearity, incorporates RegressionGP (Statistics and Machine Learning Toolbox) objects that the mapping function creates using Statistics and Machine Learning Toolbox™. The mapping object contains three components: an offset, a nonlinear component, which, in this case, is the GP kernel, and a linear component that uses a combination of linear weights.

The input to the mapping object can be an internal signal of a nonlinear black-box model, such as one of the following signals:

• Vector of the regressors of a nonlinear ARX model

• Output of the linear block of a Hammerstein-Wiener model

Mathematically, idGaussianProcess is a function that maps m inputs X(t) = [x(t₁),x₂(t),…,x_m(t)]^T to a scalar output y(t) using the following relationship:

$$y(t)=y_0+{\rm X}{(t)}^{T}PL+G({\rm X}(t),\theta ))$$

Here,

• X(t) is an m-by-1 vector of inputs, or regressors.

• y₀ is the output offset, a scalar.

• P is an m-by-p projection matrix, where m is the number of regressors and p is the number of linear weights. m must be greater than or equal to p.

• L is a p-by-1 vector of weights.

• G(X,θ) is the regressive Gaussian process that constitutes the kernel of the idGaussianProcess object. G has a mean of zero and a covariance that the user specifies by choosing a kernel, and can be expressed generally as

  $$G(X)=GP(0,K(X_{test},X_{train},\theta ))$$

A zero-mean Gaussian process G predicts the output Y_test for a given input X_test using the following relationship:

$$G(X_{test})=K(X_{test},X_{train}){[K(X_{train},X_{train})+{\sigma }_n{}^{2}I]}^{-1}{Y}_{train}$$

Here:

• K(X_test,X_train) is the covariance kernel function.

• X_train is a matrix representing the set of training inputs.

• X_test is a matrix representing the set of test inputs.

• Y_train is the vector of outputs from the training set.

• σ_n is the standard deviation of the additive measurement noise.

Gaussian process modeling is especially useful when you have only limited measurement data. For more information about creating GP regression models, see fitrgp (Statistics and Machine Learning Toolbox).

Use idGaussianProcess as the value of the OutputFcn property of an idnlarx model or the OutputNonlinearity property (but not the InputNonlinearity property) of an idnlhw object. For example, specify idGaussianProcess when you estimate an idnlarx model with the following command.

sys = nlarx(data,regressors,idGaussianProcess)

When nlarx estimates the model sys, it essentially estimates the parameters of the idGaussianProcess function.

You can use a similar approach when you specify input or output linearities using the nlhw command. For example, specify idGaussianProcess as an output nonlinearity with the following command.

sys = nlhw(data,orders,idSaturation,idGaussianProcess)

You can configure the idGaussianProcess function to disable components and fix parameters. To omit the linear component, set LinearFcn.Use to false. To omit the offset, set Offset.Use to false. To specify known values for the linear function and the offset, set their Value attributes directly and set the corresponding Free attributes to False. To modify the estimation options, set the option property in EstimationOptions. For example, to change the fit method to 'exact', use G.EstimationOptions.FitMethod = 'exact'. Use evaluate to compute the output of the function for a given vector of inputs.

Creation

Syntax

G = idGaussianProcess

G = idGaussianProcess(kernelFunction)

G = idGaussianProcess(kernelFunction,kernelParameters)

G = idGaussianProcess(kernelFunction,kernelParameters,UseLinearFcn)

G = idGaussianProcess(kernelFunction,kernelParameters,UseLinearFcn,UseOffset)

Description

G = idGaussianProcess creates an idGaussianProcess object G with the kernel function 'SquaredExponential' and default kernel parameters. The number of inputs is determined during model estimation and the number of outputs is 1.

G = idGaussianProcess(kernelFunction) specifies a specific kernel.

G = idGaussianProcess(kernelFunction,kernelParameters) initializes the parameters of the specified kernel to the values in kernelParameters.

G = idGaussianProcess(kernelFunction,kernelParameters,UseLinearFcn) specifies whether the function uses a linear function as a subcomponent.

G = idGaussianProcess(kernelFunction,kernelParameters,UseLinearFcn,UseOffset) specifies whether the function uses an offset term y₀ parameter.

Input Arguments

kernelFunction — Kernel covariance function

'SquaredExponential' (default) | 'Exponential' | 'Matern32' | 'Matern52' | 'RationalQuadratic' | 'ARDSquaredExponential' | 'ARDExponential' | 'ARDMatern32' | 'ARDMatern52' | 'ARDRationalquadratic'

Kernel covariance function, specified as character array or string. For information about the individual options, see Kernel (Covariance) Function in fitrgp (Statistics and Machine Learning Toolbox).

This argument sets the G.Kernel.KernelFunction property.

kernelParameters — Initial values for kernel parameters

vector

Initial values for the kernel parameters, specified as a vector. The size of the vector and the values depend on the choice of kernelFunction. For more information, see Kernel Parameters in fitrgp (Statistics and Machine Learning Toolbox).

This argument sets the G.Kernel.Parameters.Value property.

UseLinearFcn — Option to use linear function

true (default) | false

Option to use the linear function subcomponent, specified as true or false. This argument sets the value of the G.LinearFcn.Use property.

UseOffset — Option to use offset term

true (default) | false

Option to use an offset term, specified as true or false. This argument sets the value of the G.Offset.Use property.

Properties

Inputs — Input signal names

cell array

Input signal names for the inputs to the mapping object, specified as a 1-by-m cell array, where m is the number of input signals. This property is determined during estimation.

Outputs — Output signal name

cell array

Output signal name for the output of the mapping object, specified as a 1-by-1 cell array. This property is determined during estimation.

Kernel — Properties of GP kernel

GP kernel property values

Properties of the GP kernel, specified as follows:

• KernelFunction —Covariance kernel function K, specified as one of the values listed in the kernelFunction argument description. For more information about these options, see Kernel (Covariance) Function in fitrgp (Statistics and Machine Learning Toolbox).

• Parameters — Parameters used in the kernel function, specified as the following properties:

  • Value — Values of the kernel parameters, specified as a vector.

  • Names — Names of the kernel parameters

  • InputProjection — Projection matrix used to project inputs onto a lower dimensional subspace.

  The size of the value vector and the values depend on the choice of KernelFunction. For more information, see Kernel Parameters in fitrgp (Statistics and Machine Learning Toolbox).

• Free — Option to estimate parameters, specified as a logical scalar. If all the parameters have finite values, such as when the idGaussianProcess object corresponds to a previously estimated model, then setting Free to false causes the parameters of the kernel G(X) to remain unchanged during estimation. The default value is true.

LinearFcn — Parameters of linear function

linear function property values

Parameters of the linear function, specified as follows:

• Use — Option to use the linear function in the idGaussianProduct model, specified as a scalar logical. The default value is true.

• Value — Linear weights that compose L', specified as a 1-by-p vector.

• InputProjection — Input projection matrix P, specified as an m-by-p matrix, that transforms the detrended input vector of length m into a vector of length p.

• Free — Option to update entries of Value during estimation, specified as a 1-by-p logical vector. The software honors the Free specification only if the starting value of Value is finite. The default value is true.

Offset — Parameters of offset term

offset property values

Parameters of the offset term, specified as follows:

• Use — Option to use the offset in the idGaussianProcess model, specified as a scalar logical. The default value is true.

• Value — Offset value, specified as a scalar.

• Free — Option to update Value during estimation, specified as a scalar logical. The software honors the Free specification of false only if the value of Value is finite. The default value is true.

Estimation Options — Estimation options

estimation option property values

Estimation options for the nonlinear block of the idGaussianProcess model, specified as follows. For more information on any of these options, see fitrgp (Statistics and Machine Learning Toolbox).

• FitMethod — Method to use for estimating the parameters of the idGaussianProcess nonlinear model, specified as one of the items in the following table.

  Option	Description
  'auto'	Software selects the method automatically (default)
  'exact'	Exact GP regression
  'sd'	Subset of data points approximation
  'sr'	Subset of regressors approximation
  'fic'	Fully independent conditional approximation

• ActiveSetMethod — Active set selection method, specified as one of the items in the following table.

  Option	Description
  'random'	Random selection (default)
  'sgma'	Sparse greedy matrix approximation
  'entropy'	Differential entropy-based selection
  'likelihood'	Subset of regressors log likelihood-based selection

• SparseFitRegularization — Regularization standard deviation for the sparse methods subset of regressors ('sr') and the fully independent conditional approximation ('fic'), specified as a positive scalar value.

• Optimizer — Optimizer to use for parameter estimation, specified as one of the items in the following table.

  Option	Description
  'quasinewton'	Dense, symmetric rank-1-based, quasi-Newton approximation to the Hessian (default)
  'lbfgs'	LBFGS-based quasi-Newton approximation to the Hessian
  'fminsearch'	Unconstrained nonlinear optimization using the simplex search method of Lagarias et al. [see fitrgp (Statistics and Machine Learning Toolbox)]
  'fminunc'	Unconstrained nonlinear optimization (requires an Optimization Toolbox™ license)
  'fmincon'	Constrained nonlinear optimization (requires an Optimization Toolbox license)

• OptimizerOptions — Options for the optimizer, specified as a structure or object. When Optimizer is set or changed, the software automatically updates the value of OptimizerOptions to match the defaults for the corresponding optimizer. Use the properties of the OptimizerOptions option set to change the values from their defaults.

Examples

Estimate Nonlinear ARX Model with idGaussianProcess as Output Function

Load the input/output data from twotankdata and construct an iddata object z.

load twotankdata u y
z = iddata(y,u,0.8,'timeunit','hours');

Create an idGaussianProduct mapping object g that uses a Matern kernel with the parameter 3/2.

g = idGaussianProcess('Matern32');

Estimate a nonlinear ARX model that uses g as the output function.

sys = nlarx(z,[4 4 1],g)

sys =

Nonlinear ARX model with 1 output and 1 input
  Inputs: u1
  Outputs: y1

Regressors:
  Linear regressors in variables y1, u1

Output function: Gaussian process function using a Matern32 kernel
Sample time: 0.8 hours

Status:                                          
Estimated using NLARX on time domain data "z".   
Fit to estimation data: 97.14% (prediction focus)
FPE: 2.82e-05, MSE: 2.795e-05

Display the postestimation properties of g.

disp(sys.OutputFcn.Input)

Function inputs

     Name: {'y1(t-1)'  'y1(t-2)'  'y1(t-3)'  'y1(t-4)'  'u1(t-1)'  'u1(t-2)'  'u1(t-3)'  'u1(t-4)'}
     Mean: [-4.7062e-17 -5.3807e-17 -5.2324e-17 -8.3748e-18 1.0174e-15 1.0364e-15 1.0744e-15 1.1123e-15]
    Range: [2x8 double]

disp(sys.outputFcn.Offset)

Output Offset: initialized to -8.11e-17
      Use: 1
    Value: -8.1058e-17
     Free: 1

disp(sys.outputFcn.NonlinearFcn)

GP kernel and its parameters

    KernelFunction: 'Matern32'
        Parameters: '<Kernel parameters>'
              Free: 1
            Inputs: {'y1(t-1)'  'y1(t-2)'  'y1(t-3)'  'y1(t-4)'  'u1(t-1)'  'u1(t-2)'  'u1(t-3)'  'u1(t-4)'}
           Outputs: {'y1(t):Nonlinear'}

Compare the output of sys with the measured output z.

compare(z,sys)

The nonlinear model shows a good fit to the estimation data.

Create idGaussianProcess object with no Linear Block

Load the data z3.

load iddata3 z3

Create an idGaussianProcess object G that sets the UseLinearFcn argument to 0. Since this argument is third in the syntax, you must also specify kernelFunction and kernelParameters. Set kernelFunction to its default value of 'SquaredExponential'. Set the kernelParameters argument to [], which specifies no initialization for the parameters.

kernelFunction = 'SquaredExponential';
kernelParameters = [];
UseLinearFcn = 0;
G = idGaussianProcess(kernelFunction,kernelParameters,UseLinearFcn)

G = 
Gaussian Process Function

 Nonlinear Function: Gaussian process function using a SquaredExponential kernel
 Linear Function: not in use
 Output Offset: uninitialized

               Kernel: '<GP kernel and its parameters>'
            LinearFcn: '<Linear function parameters>'
               Offset: '<Offset parameters>'
    EstimationOptions: '<Estimation option set>'

The properties of G are consistent with your inputs.

Create idGaussianProcess Object with no Offset Block

Load the data z3.

load iddata3 z3

Create an idGaussianProcess object that has no offset block by first creating a default object, and then, using dot notation to set the G.Offset.Use property directly.

G = idGaussianProcess;
G.Offset.Use = 0

G = 
Gaussian Process Function

 Nonlinear Function: Gaussian process function using a SquaredExponential kernel
 Linear Function: uninitialized
 Output Offset: not in use

               Kernel: '<GP kernel and its parameters>'
            LinearFcn: '<Linear function parameters>'
               Offset: '<Offset parameters>'
    EstimationOptions: '<Estimation option set>'

The function description identifies the output offset as not in use.

Version History

Introduced in R2021b

R2022a: Use of previous idGaussianProcess NonLinearFcn property is not recommended

Starting in R2022a, the NonLinearFcn property of the idGaussianProcess object has been renamed to Kernel. The previous property name still works. There are no plans to exclude the previous name at this time.

This change has no impact on existing syntaxes for idGaussianProcess. If you have code that uses dot notation to directly set or view this property, consider changing your code to use the new name.

R2022a: Previous idnlarx data normalization information moved from mapping object properties to idnlarx Normalization property

Information related to data normalization was moved from the idGaussianProcess mapping object level to the model level. The Normalization property of the idnlarx model contains the data centering and scaling information that the estimation process computes. In addition, the regressor-selection process for the mapping objects has also moved to the model level. The model now passes the actual regressor names rather than the selection indices to the mapping object, eliminating the need for an index property at the mapping object level.

The following table summarizes the mapping object subproperties that were eliminated. For more information, see the Normalization property of idnlarx.

Main Properties / Subproperties	Input	Output	LinearMdl	Offset	NonlinearMdl
Mean	X	X
Range	X	X
Minimum			X	X	X
Maximum			X	X	X
SelectedInputIndex			X		X