Reliability Diagram for calibration of two-class predictor.
PLOTRELIABILITY(TARGETS, PREDICTIONS) uses TARGETS and PREDICTIONS vectors to compute TARGETS_MEANS and PREDICTIONS_MEANS. FIG is a rudimentary reliability diagram plotted using these means.
Values of TARGETS are either 0 or 1, representing the two classes. Values of PREDICTIONS are probabilities of the positive class (1), previously obtained using a probabilistic predictive model. TARGETS and PREDICTIONS are row or column vectors of the same size.
The predictions are discretized into ten bins. Cases with predicted value between 0 and 0.1 fall in the first bin, between 0.1 and 0.2 in the second bin, etc. For each bin, the mean predicted value (PREDICTIONS_MEANS) and the true fraction of positive cases (TARGETS_MEANS) are computed. These points are then plotted on the X and Y axes respectively. If the model is well calibrated, the points will fall near the diagonal line.
SHOW_FIG is an optional logical input argument. Its default value is true. Setting it to false prevents the automatic display of the reliability diagram.
Syntax:
plotreliability(targets, predictions);
[fig, targets_means, predictions_means] = plotreliability(targets, predictions, false);
Remarks:
This function does not allow the number of bins or the width of each bin to be changed.
References:
Alexandru Niculescu-Mizil and Rich Caruana (2005) Predicting Good Probabilities With Supervised Learning, in Proceedings of the 22nd International Conference on Machine Learning. See section 4 (Qualitative Analysis of Predictions).
Cite As
Skynet (2024). Reliability Diagram for calibration of two-class predictor. (https://www.mathworks.com/matlabcentral/fileexchange/25704-reliability-diagram-for-calibration-of-two-class-predictor), MATLAB Central File Exchange. Retrieved .
MATLAB Release Compatibility
Platform Compatibility
Windows macOS LinuxCategories
Tags
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!Discover Live Editor
Create scripts with code, output, and formatted text in a single executable document.
Version | Published | Release Notes | |
---|---|---|---|
1.3.0.0 | Fixed error in Description. |
||
1.2.0.0 | (1) Changed function name to plotreliability.
|
||
1.1.0.0 | (1) Added SHOW_FIG, an optional logical input argument. Its default value is false. Setting it to true displays the reliability diagram.
|
||
1.0.0.0 |