ClassificationDiscriminant

R2026b

Discriminant analysis classification

Description

ClassificationDiscriminant is a discriminant analysis classifier, which is a Gaussian mixture model for data generation. Use the predict function of ClassificationDiscriminant to predict responses for new data. Because the ClassificationDiscriminant object contains the data used for training, you can use the object to compute resubstitution predictions.

Creation

Create a ClassificationDiscriminant object by using fitcdiscr.

Properties

Discriminant Analysis Properties

BetweenSigma — Between-class covariance

Read-only: square matrix

This property is read-only.

Between-class covariance, returned as a p-by-p matrix, where p is the number of predictors.

Data Types: double

Coeffs — Coefficient matrices

Read-only: k-by-k structure | []

This property is read-only.

Coefficient matrices, returned as a k-by-k structure, where k is the number of classes. If the value of the FillCoeffs name-value argument is "off" when you create the classifier object, Coeffs is empty ([]).

Coeffs(i,j) contains the coefficients of the linear or quadratic boundaries between classes i and j. The fields in Coeffs(i,j) are:

• DiscrimType — Character vector corresponding to the discriminant type

• Class1 — ClassNames(i) value

• Class2 — ClassNames(j) value

• Const — Numeric scalar

• Linear — Numeric vector with p components, where p is the number of columns in X

• Quadratic — p-by-p numeric matrix. This field exists only when the discriminant type is quadratic.

The equation of the boundary between class i and class j is

Const + Linear * x + x' * Quadratic * x = 0,

where x is a column vector of length p.

Data Types: struct

Delta — Delta threshold for linear discriminant model

nonnegative scalar

Delta threshold for a linear discriminant model, specified as a nonnegative scalar. If a coefficient of the classifier object has a magnitude smaller than Delta, the software sets the coefficient to 0, which means you can remove the corresponding predictor from the model. Set Delta to a higher value to remove more predictors.

Delta must be 0 for quadratic discriminant models.

To change the value of Delta to newDelta for a classifier object Mdl, you can use dot notation: Mdl.Delta = newDelta.

Data Types: double

DeltaPredictor — Minimum value of Delta coefficient

Read-only: row vector of length p

This property is read-only.

Minimum value of the Delta coefficient needed for the predictor to be included in the model, returned as a row vector of length p, where p is the number of predictors in the classifier object. If DeltaPredictor(i) < Delta, then coefficient i of the model is 0.

If the classifier is a quadratic discriminant model, all elements of DeltaPredictor are 0.

Data Types: double

DiscrimType — Discriminant type

character vector | string scalar

Discriminant type, specified as a character vector or string scalar corresponding to one of these values:

• "linear"

• "quadratic"

• "diagLinear"

• "diagQuadratic"

• "pseudoLinear"

• "pseudoQuadratic"

To change the value of DiscrimType to newDiscrimType for a classifier object Mdl, you can use dot notation: Mdl.DiscrimType = newDiscrimType. You can change one linear type to another, or one quadratic type to another, but cannot change a linear type to a quadratic type (or vice versa).

Data Types: char | string

Gamma — Gamma regularization parameter

scalar from 0 through 1

Gamma regularization parameter, specified as a scalar from 0 through 1. To change the Gamma value to newGamma for a classifier object Mdl, you can use dot notation: Mdl.Gamma = newGamma.

• If you set Gamma to 1 for a linear discriminant model, the software sets the DiscrimType property to 'diagLinear'.

• If you set Gamma to a value between MinGamma and 1 for a linear discriminant model, the software sets the DiscrimType property to 'linear'.

• You cannot set Gamma to a value below the value of the MinGamma property.

• For a quadratic discriminant model, you can set Gamma to 0 (when DiscrimType is 'quadratic') or 1 (when DiscrimType is 'diagQuadratic').

Data Types: double

LogDetSigma — Logarithm of determinant of within-class covariance matrix

Read-only: numeric scalar | numeric vector

This property is read-only.

Logarithm of the determinant of the within-class covariance matrix, returned as a numeric scalar or numeric vector.

• For linear discriminant analysis, LogDetSigma is a numeric scalar.

• For quadratic discriminant analysis, LogDetSigma is a numeric vector of length K, where K is the number of classes.

Data Types: double

MinGamma — Minimal value of Gamma parameter

Read-only: nonnegative scalar

This property is read-only.

Minimal value of the Gamma parameter required to make the correlation matrix invertible, returned as a nonnegative scalar. If the correlation matrix is not singular, MinGamma is 0.

Data Types: double

ModelParameters — Parameters used to train the model

Read-only: DiscriminantParams object

This property is read-only.

Parameters used to train the model, returned as a DiscriminantParams object with the following properties.

Property	Value
DiscrimType	'linear' 'quadratic' 'diagLinear' 'diagQuadratic' 'pseudoLinear' 'pseudoQuadratic'
Gamma	scalar from 0 through 1
Delta	nonnegative scalar
FillCoeffs	logical scalar
SaveMemory	logical scalar
Version	scalar
Method	'Discriminant'
Type	'classification'

Predictor Properties

CategoricalPredictors — Categorical predictor indices

Read-only: empty array []

This property is read-only.

Categorical predictor indices, returned as an empty array [].

Mu — Class means

Read-only: K-by-p matrix

This property is read-only.

Class means, returned as a K-by-p matrix of real values. K is the number of classes, and p is the number of predictors. Each row of Mu represents the mean of the multivariate normal distribution of the corresponding class. The class indices are stored in the ClassNames property.

Data Types: double

PredictorNames — Predictor variable names

Read-only: cell array of character vectors

This property is read-only.

Predictor variable names, returned as a cell array of character vectors. The order of the elements in PredictorNames corresponds to the order in which the predictor names appear in the training data.

Data Types: cell

Sigma — Within-class covariance

Read-only: numeric array

This property is read-only.

Within-class covariance, returned as a numeric array. The dimensions of the array depend on the value of DiscrimType:

• 'linear' (default) — Matrix of size p-by-p, where p is the number of predictors

• 'quadratic' — Array of size p-by-p-by-K, where K is the number of classes

• 'diagLinear' — Row vector of length p

• 'diagQuadratic' — Array of size 1-by-p-by-K

• 'pseudoLinear' — Matrix of size p-by-p

• 'pseudoQuadratic' — Array of size p-by-p-by-K

Data Types: double

X — Predictor values

Read-only: real matrix

This property is read-only.

Predictor values, returned as a real matrix. Each column of X represents one predictor (variable), and each row represents one observation.

Data Types: single | double

Xcentered — X data with class means subtracted

Read-only: real matrix

This property is read-only.

X data with the class means subtracted, returned as a real matrix. If Y(i) is of class j, Xcentered(i,:) = X(i,:) – Mu(j,:), where Mu is the class means property.

Data Types: single | double

Response Properties

ClassNames — Unique class names

Read-only: numeric vector | categorical vector | logical vector | character array | cell array of character vectors

This property is read-only.

Unique class names used in training, returned as a numeric vector, categorical vector, logical vector, character array, or cell array of character vectors. ClassNames has the same data type as the class labels Y. (The software treats string arrays as cell arrays of character vectors.) ClassNames also determines the class order.

Data Types: single | double | categorical | logical | char | cell

ResponseName — Response variable name

Read-only: character vector

This property is read-only.

Response variable name, returned as a character vector.

Data Types: char

Y — Class labels

Read-only: numeric vector | categorical vector | logical vector | character array | cell array of character vectors

This property is read-only.

Class labels used to train the model, returned as a numeric vector, categorical vector, logical vector, character array, or cell array of character vectors. Y has the same data type as the response variable used to train the model. (The software treats string arrays as cell arrays of character vectors.)

Each row of Y represents the classification of the corresponding observation in X.

Data Types: single | double | categorical | logical | char | cell

Other Data Properties

HyperparameterOptimizationResults — Cross-validation optimization of hyperparameters

Read-only: SupervisedLearningBayesianOptimization object | table | []

This property is read-only.

Cross-validation optimization of hyperparameters, returned as a SupervisedLearningBayesianOptimization object or a table of hyperparameters and associated values. This property is nonempty if the OptimizeHyperparameters name-value argument is nonempty when you create the model. The value of HyperparameterOptimizationResults depends on the setting of the Optimizer option in the HyperparameterOptimizationOptions value when you create the model.

Value of Optimizer Option	Value of HyperparameterOptimizationResults
"bayesopt" (default)	SupervisedLearningBayesianOptimization object
"gridsearch" or "randomsearch"	Table of hyperparameters used, observed objective function values (cross-validation loss), and observation ranks from lowest (best) to highest (worst)

NumObservations — Number of observations in training data

Read-only: positive integer

This property is read-only.

Number of observations in the training data, returned as a positive integer. NumObservations can be less than the number of rows of input data when there are missing values in the input data or response data.

Data Types: double

RowsUsed — Rows of original predictor data X used for fitting

Read-only: logical vector

This property is read-only.

Rows of the original predictor data X used for fitting, returned as an n-element logical vector, where n is the number of rows of X. If the software uses all rows of X to create the object, then RowsUsed is an empty array ([]).

Data Types: logical

W — Scaled observation weights

Read-only: numeric vector

This property is read-only.

Scaled observation weights, returned as a numeric vector of length n, where n is the number of rows in X.

Data Types: double

Other Classification Properties

Cost — Misclassification costs

square numeric matrix

Misclassification costs, returned as a square numeric matrix. Cost has K rows and columns, where K is the number of classes.

Cost(i,j) is the cost of classifying a point into class j if its true class is i. The order of the rows and columns of Cost corresponds to the order of the classes in ClassNames.

To change the value of Cost to costMatrix for a classifier object Mdl, you can use dot notation: Mdl.Cost = costMatrix.

Data Types: double

Prior — Prior probabilities for each class

numeric vector

Prior probabilities for each class, returned as a K-element numeric vector, where K is the number of unique classes in the response. The order of the elements of Prior corresponds to the order of the classes in ClassNames.

To change the value of Prior to priorVector for a classifier object Mdl, you can use dot notation: Mdl.Prior = priorVector.

Data Types: double

ScoreTransform — Function for transforming scores

function handle | name of a built-in transformation function | "none"

Function for transforming scores, specified as a function handle or the name of a built-in transformation function. "none" means no transformation; equivalently, "none" means @(x)x. For a list of built-in transformation functions and the syntax of custom transformation functions, see ScoreTransform.

To change the value of ScoreTransform for a classifier object Mdl, you can use dot notation:

Mdl.ScoreTransform = "function"
% or
Mdl.ScoreTransform = @function

Data Types: char | string | function_handle

Object Functions

compact	Reduce size of machine learning model
compareHoldout	Compare accuracies of two classification models using new data
crossval	Cross-validate machine learning model
cvshrink	Cross-validate regularization of linear discriminant
edge	Classification edge for discriminant analysis classifier
lime	Local interpretable model-agnostic explanations (LIME)
logp	Log unconditional probability density for discriminant analysis classifier
loss	Classification loss for discriminant analysis classifier
mahal	Mahalanobis distance to class means of discriminant analysis classifier
margin	Classification margins for discriminant analysis classifier
nLinearCoeffs	Number of nonzero linear coefficients in discriminant analysis classifier
partialDependence	Compute partial dependence
plotPartialDependence	Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots
predict	Predict labels using discriminant analysis classifier
resubEdge	Resubstitution classification edge for discriminant analysis classifier
resubLoss	Resubstitution classification loss for discriminant analysis classifier
resubMargin	Resubstitution classification margins for discriminant analysis classifier
resubPredict	Classify observations in discriminant analysis classifier by resubstitution
shapley	Shapley values
testckfold	Compare accuracies of two classification models by repeated cross-validation

Examples

Train Discriminant Analysis Model

Load Fisher's iris data set.

load fisheriris

Train a discriminant analysis model using the entire data set.

Mdl = fitcdiscr(meas,species)

Mdl = 
  ClassificationDiscriminant
             ResponseName: 'Y'
    CategoricalPredictors: []
               ClassNames: {'setosa'  'versicolor'  'virginica'}
           ScoreTransform: 'none'
          NumObservations: 150
              DiscrimType: 'linear'
                       Mu: [3×4 double]
                   Coeffs: [3×3 struct]


  Properties, Methods

Mdl is a ClassificationDiscriminant model object. To access its properties, use dot notation. For example, display the class means for each predictor.

Mdl.Mu

ans = 3×4

    5.0060    3.4280    1.4620    0.2460
    5.9360    2.7700    4.2600    1.3260
    6.5880    2.9740    5.5520    2.0260

To predict labels for new observations, pass Mdl and predictor data to the predict function.

More About

Discriminant Classification

In the model for discriminant analysis, each class (Y) generates data (X) using a multivariate normal distribution. That is, the model assumes X has a Gaussian mixture distribution (gmdistribution).

• For linear discriminant analysis, the model has the same covariance matrix for each class; only the means vary.

• For quadratic discriminant analysis, both the means and the covariances of each class vary.

The predict function classifies observations in a way that minimizes the expected classification cost:

$$\widehat{y}=\underset{y=1,\mathrm{...},K}{\arg \min }{\displaystyle \sum _{k=1}^K\widehat{P}\left(k|x\right)C\left(y|k\right)},$$

where

• $$\widehat{y}$$ is the predicted classification.

• K is the number of classes.

• $$\widehat{P}\left(k|x\right)$$ is the posterior probability of class k for observation x.

• $$C\left(y|k\right)$$ is the cost of classifying an observation as y when its true class is k.

For details, see Prediction Using Discriminant Analysis Models.

Regularization

Regularization is the process of finding a small set of predictors that yield an effective predictive model. Linear discriminant analysis includes two parameters that control regularization, γ and δ. cvshrink helps you select appropriate values of the parameters.

Let Σ represent the covariance matrix of the data X, and let $$\widehat{X}$$ be the centered data (the data X minus the mean by class).

$$D=\text{diag}\left({\widehat{X}}^T*\widehat{X}\right).$$

The regularized covariance matrix $$\tilde{\Sigma }$$ is

$$\tilde{\Sigma }=\left(1-\gamma \right)\Sigma +\gamma D.$$

When γ ≥ MinGamma, $$\tilde{\Sigma }$$ is nonsingular.

Let μ_k be the mean vector for those elements of X in class k, and let μ₀ be the global mean vector (the mean of the rows of X). Let C be the correlation matrix of the data X, and let $$\tilde{C}$$ be the regularized correlation matrix.

$$\tilde{C}=\left(1-\gamma \right)C+\gamma I,$$

where I is the identity matrix.

The linear term in the regularized discriminant analysis classifier for a data point x is

$${\left(x-{\mu }_0\right)}^T{\tilde{\Sigma }}^{-1}\left({\mu }_k-{\mu }_0\right)=\left[{\left(x-{\mu }_0\right)}^T{D}^{-1/2}\right]\left[{\tilde{C}}^{-1}{D}^{-1/2}\left({\mu }_k-{\mu }_0\right)\right].$$

The parameter δ enters into this equation as a threshold on the final term in square brackets. Each component of the vector $$\left[{\tilde{C}}^{-1}{D}^{-1/2}\left({\mu }_k-{\mu }_0\right)\right]$$ is set to zero if it is smaller in magnitude than the threshold δ. Therefore, for class k, if component j is thresholded to zero, component j of x does not enter into the evaluation of the posterior probability.

The DeltaPredictor property is a vector related to this threshold. When δ ≥ DeltaPredictor(i), all classes k have

$$\left|{\tilde{C}}^{-1}{D}^{-1/2}\left({\mu }_k-{\mu }_0\right)\right|\le \delta .$$

Therefore, when δ ≥ DeltaPredictor(i), the regularized classifier does not use predictor i.

References

[1] Guo, Y., T. Hastie, and R. Tibshirani. "Regularized Linear Discriminant Analysis and Its Application in Microarrays." Biostatistics, Vol. 8, No. 1, pp. 86–100, 2007.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• The predict function supports code generation.

• When you train a discriminant analysis model by using fitcdiscr or create a compact discriminant analysis model by using makecdiscr, the value of the ScoreTransform name-value argument cannot be an anonymous function.

For more information, see Introduction to Code Generation for Statistics and Machine Learning Functions.

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Refer to the usage notes and limitations in the C/C++ Code Generation section. The same usage notes and limitations apply to GPU code generation.

Version History

Introduced in R2011b

R2026a: Model stores Bayesian optimization results in a new object

If you perform Bayesian hyperparameter optimization by using a supervised learning fit function, the optimization results are stored in a SupervisedLearningBayesianOptimization object. In previous releases, the optimization results are stored in a BayesianOptimization object.

R2023b: Observations with missing predictor values are stored in the model

Starting in R2023b, training observations with missing predictor values are included in the X, Xcentered, Y, and W data properties. The RowsUsed property indicates the training observations stored in the model, rather than those used for training. Observations with missing predictor values continue to be omitted from the model training process.

In previous releases, the software omitted training observations that contained missing predictor values from the data properties of the model.