Classification Ensembles

Boosting, random forest, bagging, random subspace, and ECOC ensembles for multiclass learning

A classification ensemble is a predictive model composed of a weighted combination of multiple classification models. In general, combining multiple classification models increases predictive performance.

To explore classification ensembles interactively, use the Classification Learner app. For greater flexibility, use fitcensemble in the command-line interface to boost or bag classification trees, or to grow a random forest. For details on all supported ensembles, see Ensemble Algorithms. To reduce a multiclass problem into an ensemble of binary classification problems, train an error-correcting output codes (ECOC) model. For details, see fitcecoc.

To boost regression trees using LSBoost, or to grow a random forest of regression trees, see Regression Ensembles.

Apps

Classification LearnerTrain models to classify data using supervised machine learning

Blocks

ClassificationEnsemble PredictClassify observations using ensemble of decision trees (Since R2021a)
ClassificationECOC PredictClassify observations using error-correcting output codes (ECOC) classification model (Since R2023a)
IncrementalClassificationECOC PredictClassify observations using incremental ECOC classification model (Since R2024a)

Functions

expand all

templateDiscriminantDiscriminant analysis classifier template
templateECOCError-correcting output codes learner template
templateEnsembleEnsemble learning template
templateKNNk-nearest neighbor classifier template
templateLinearLinear learner template
templateNaiveBayesNaive Bayes classifier template
templateSVMSupport vector machine template
templateTreeCreate decision tree template

Create Classification Ensemble

fitcensembleFit ensemble of learners for classification
compactReduce size of machine learning model

Modify Classification Ensemble

resumeResume training of classification ensemble model
removeLearnersRemove members of compact classification ensemble

Interpret Classification Ensemble

limeLocal interpretable model-agnostic explanations (LIME)
partialDependenceCompute partial dependence
permutationImportancePredictor importance by permutation (Since R2024a)
plotPartialDependenceCreate partial dependence plot (PDP) and individual conditional expectation (ICE) plots
predictorImportanceEstimates of predictor importance for classification ensemble of decision trees
shapleyShapley values (Since R2021a)

Cross-Validate Classification Ensemble

crossvalCross-validate machine learning model
kfoldEdgeClassification edge for cross-validated classification model
kfoldLossClassification loss for cross-validated classification model
kfoldMarginClassification margins for cross-validated classification model
kfoldPredictClassify observations in cross-validated classification model
kfoldfunCross-validate function for classification

Measure Performance

lossClassification loss for classification ensemble model
resubLossResubstitution classification loss for classification ensemble model
compareHoldoutCompare accuracies of two classification models using new data
edgeClassification edge for classification ensemble model
marginClassification margins for classification ensemble model
resubEdgeResubstitution classification edge for classification ensemble model
resubMarginResubstitution classification margins for classification ensemble model
testckfoldCompare accuracies of two classification models by repeated cross-validation

Classify Observations

predictPredict labels using classification ensemble model
resubPredictClassify observations in classification ensemble by resubstitution
oobPredictPredict out-of-bag labels and scores of bagged classification ensemble

Gather Properties of Classification Ensemble

gatherGather properties of Statistics and Machine Learning Toolbox object from GPU
fitcensembleFit ensemble of learners for classification
TreeBaggerEnsemble of bagged decision trees
predictPredict responses using ensemble of bagged decision trees
oobPredictEnsemble predictions for out-of-bag observations

Create ECOC

fitcecocFit multiclass models for support vector machines or other classifiers
compactReduce size of machine learning model

Modify ECOC

discardSupportVectorsDiscard support vectors of linear SVM binary learners in ECOC model

Interpret ECOC

limeLocal interpretable model-agnostic explanations (LIME)
partialDependenceCompute partial dependence
plotPartialDependenceCreate partial dependence plot (PDP) and individual conditional expectation (ICE) plots
shapleyShapley values (Since R2021a)

Cross-Validate ECOC

crossvalCross-validate machine learning model
kfoldEdgeClassification edge for cross-validated ECOC model
kfoldLossClassification loss for cross-validated ECOC model
kfoldMarginClassification margins for cross-validated ECOC model
kfoldPredictClassify observations in cross-validated ECOC model
kfoldfunCross-validate function using cross-validated ECOC model

Measure Performance

lossClassification loss for multiclass error-correcting output codes (ECOC) model
resubLossResubstitution classification loss for multiclass error-correcting output codes (ECOC) model
compareHoldoutCompare accuracies of two classification models using new data
edgeClassification edge for multiclass error-correcting output codes (ECOC) model
marginClassification margins for multiclass error-correcting output codes (ECOC) model
resubEdgeResubstitution classification edge for multiclass error-correcting output codes (ECOC) model
resubMarginResubstitution classification margins for multiclass error-correcting output codes (ECOC) model
testckfoldCompare accuracies of two classification models by repeated cross-validation

Classify Observations

predictClassify observations using multiclass error-correcting output codes (ECOC) model
resubPredictClassify observations in multiclass error-correcting output codes (ECOC) model

Gather Properties of ECOC

gatherGather properties of Statistics and Machine Learning Toolbox object from GPU

Objects

expand all

ClassificationEnsembleEnsemble classifier
CompactClassificationEnsembleCompact classification ensemble
ClassificationPartitionedEnsembleCross-validated classification ensemble
TreeBaggerEnsemble of bagged decision trees
CompactTreeBaggerCompact ensemble of bagged decision trees
ClassificationBaggedEnsembleClassification ensemble grown by resampling

Classes

expand all

ClassificationECOCMulticlass model for support vector machines (SVMs) and other classifiers
CompactClassificationECOCCompact multiclass model for support vector machines (SVMs) and other classifiers
ClassificationPartitionedECOCCross-validated multiclass ECOC model for support vector machines (SVMs) and other classifiers

Topics