Compute the performance metrics (FPR, TPR, and expected cost) for a multiclass classification problem when you create a rocmetrics object. Compute additional metrics, the positive predictive value (PPV) and the negative predictive value (NPV), and add them to the object.
Load the fisheriris data set. The matrix meas contains flower measurements for 150 different flowers. The vector species lists the species for each flower. species contains three distinct flower names.
Train a classification tree that classifies observations into one of the three labels. Cross-validate the model using 10-fold cross-validation.
Compute the classification scores for validation-fold observations.
Scores is a matrix of size 150-by-3. The column order of Scores follows the class order in Mdl. Display the class order stored in Mdl.ClassNames.
ans = 3x1 cell
{'setosa' }
{'versicolor'}
{'virginica' }
Create a rocmetrics object by using the true labels in species and the classification scores in Scores. Specify the column order of Scores using Mdl.ClassNames. By default, rocmetrics computes the FPR and TPR. Specify AdditionalMetrics="ExpectedCost" to compute the expected cost as well.
The table in the Metrics property of rocObj contains performance metric values for all three classes, vertically concatenated according to the class order. Find and display the rows for the second class in the table.
ans=13×5 table
ClassName Threshold FalsePositiveRate TruePositiveRate ExpectedCost
______________ _________ _________________ ________________ ____________
{'versicolor'} 1 0 0 0.074074
{'versicolor'} 1 0.01 0.7 0.023704
{'versicolor'} 0.95455 0.02 0.8 0.017778
{'versicolor'} 0.91304 0.03 0.9 0.011852
{'versicolor'} -0.2 0.04 0.9 0.013333
{'versicolor'} -0.33333 0.06 0.9 0.016296
{'versicolor'} -0.6 0.08 0.9 0.019259
{'versicolor'} -0.86957 0.12 0.92 0.023704
{'versicolor'} -0.91111 0.16 0.96 0.026667
{'versicolor'} -0.95122 0.31 0.96 0.048889
{'versicolor'} -0.95238 0.38 0.98 0.057778
{'versicolor'} -0.95349 0.44 0.98 0.066667
{'versicolor'} -1 1 1 0.14815
The table in Metrics contains the variables for the class names, threshold, false positive rate, true positive rate, and expected cost (the additional metric).
After creating a rocmetrics object, you can compute additional metrics using the classification model information stored in the object. Compute the PPV and NPV by using the addMetrics function. To overwrite the input argument rocObj, assign the output of addMetrics to the input.
Display the Metrics property.
ans=13×7 table
ClassName Threshold FalsePositiveRate TruePositiveRate ExpectedCost PositivePredictiveValue NegativePredictiveValue
______________ _________ _________________ ________________ ____________ _______________________ _______________________
{'versicolor'} 1 0 0 0.074074 NaN 0.66667
{'versicolor'} 1 0.01 0.7 0.023704 0.97222 0.86842
{'versicolor'} 0.95455 0.02 0.8 0.017778 0.95238 0.90741
{'versicolor'} 0.91304 0.03 0.9 0.011852 0.9375 0.95098
{'versicolor'} -0.2 0.04 0.9 0.013333 0.91837 0.9505
{'versicolor'} -0.33333 0.06 0.9 0.016296 0.88235 0.94949
{'versicolor'} -0.6 0.08 0.9 0.019259 0.84906 0.94845
{'versicolor'} -0.86957 0.12 0.92 0.023704 0.7931 0.95652
{'versicolor'} -0.91111 0.16 0.96 0.026667 0.75 0.97674
{'versicolor'} -0.95122 0.31 0.96 0.048889 0.60759 0.97183
{'versicolor'} -0.95238 0.38 0.98 0.057778 0.56322 0.98413
{'versicolor'} -0.95349 0.44 0.98 0.066667 0.52688 0.98246
{'versicolor'} -1 1 1 0.14815 0.33333 NaN
The table in Metrics now includes the PositivePredictiveValue and NegativePredictiveValue variables in the last two columns, in the order you specified. Note that the positive predictive value (PPV = TP/(TP+FP)) is NaN for the reject-all threshold (largest threshold), and the negative predictive value (NPV = TN/(TN+FN)) is NaN for the accept-all threshold (lowest threshold). TP, FP, TN, and FN represent the number of true positives, false positives, true negatives, and false negatives, respectively.
