resubLoss

Class: ClassificationNaiveBayes

Classification loss for naive Bayes classifiers by resubstitution

Syntax

L = resubLoss(Mdl)
L = resubLoss(Mdl,Name,Value)

Description

example

L = resubLoss(Mdl) returns the in-sample minimum misclassification cost loss (L), which is a scalar representing how well the trained naive Bayes classifier Mdl classifies the predictor data stored in Mdl.X as compared to the true class labels stored in Mdl.Y.

example

L = resubLoss(Mdl,Name,Value) returns the in-sample classification loss with additional options specified by one or more Name,Value pair arguments.

Input Arguments

expand all

A fully trained naive Bayes classifier, specified as a ClassificationNaiveBayes model trained by fitcnb.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Loss function, specified as the comma-separated pair consisting of 'LossFun' and a built-in, loss-function name or function handle.

  • The following table lists the available loss functions. Specify one using its corresponding character vector or string scalar.

    ValueDescription
    'binodeviance'Binomial deviance
    'classiferror'Classification error
    'exponential'Exponential
    'hinge'Hinge
    'logit'Logistic
    'mincost'Minimal expected misclassification cost (for classification scores that are posterior probabilities)
    'quadratic'Quadratic

    'mincost' is appropriate for classification scores that are posterior probabilities. Naive Bayes models return posterior probabilities as classification scores by default (see predict).

  • Specify your own function using function handle notation.

    Suppose that n be the number of observations in X and K be the number of distinct classes (numel(Mdl.ClassNames), Mdl is the input model). Your function must have this signature

    lossvalue = lossfun(C,S,W,Cost)
    where:

    • The output argument lossvalue is a scalar.

    • You choose the function name (lossfun).

    • C is an n-by-K logical matrix with rows indicating which class the corresponding observation belongs. The column order corresponds to the class order in Mdl.ClassNames.

      Construct C by setting C(p,q) = 1 if observation p is in class q, for each row. Set all other elements of row p to 0.

    • S is an n-by-K numeric matrix of classification scores. The column order corresponds to the class order in Mdl.ClassNames. S is a matrix of classification scores, similar to the output of predict.

    • W is an n-by-1 numeric vector of observation weights. If you pass W, the software normalizes them to sum to 1.

    • Cost is a K-by-K numeric matrix of misclassification costs. For example, Cost = ones(K) - eye(K) specifies a cost of 0 for correct classification, and 1 for misclassification.

    Specify your function using 'LossFun',@lossfun.

For more details on loss functions, see Classification Loss.

Data Types: char | string | function_handle

Output Arguments

expand all

Classification loss, returned as a scalar. L is a generalization or resubstitution quality measure. Its interpretation depends on the loss function and weighting scheme, but, in general, better classifiers yield smaller loss values.

Examples

expand all

Load Fisher's iris data set.

load fisheriris
X = meas;    % Predictors
Y = species; % Response

Train a naive Bayes classifier. It is good practice to specify the class order. Assume that each predictor is conditionally, normally distributed given its label.

Mdl = fitcnb(X,Y,'ClassNames',{'setosa','versicolor','virginica'});

Mdl is a trained ClassificationNaiveBayes classifier.

Estimate the default resubstitution loss, which is the in-sample minimum misclassification cost.

L = resubLoss(Mdl)
L = 0.0400

The average, in-sample cost of classification is 0.04.

Load Fisher's iris data set.

load fisheriris
X = meas;    % Predictors
Y = species; % Response

Train a naive Bayes classifier. It is good practice to specify the class order. Assume that each predictor is conditionally, normally distributed given its label.

Mdl = fitcnb(X,Y,'ClassNames',{'setosa','versicolor','virginica'});

Mdl is a trained ClassificationNaiveBayes classifier.

Estimate the in-sample proportion of misclassified observations.

L = resubLoss(Mdl,'LossFun','classiferror')
L = 0.0400

The naive Bayes classifier misclassifies 4% of the training observations.

More About

expand all

References

[1] Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, second edition. Springer, New York, 2008.