How to compute Gini impurity in random forest (treebagger)?

16 views (last 30 days)
Hello. I am doing classification using treebagger random forest. I should compute the gini index or gini impurity to understand each feature importance in classification. Can any one help me with that? thanks. Here is my random forest classifier code:
nTrees=50;
B2 = TreeBagger(nTrees,train_data,train_label, 'Method', 'classification');
predChar2 = B2.predict(test_data);
predictedClass2 = str2double(predChar2);

Answers (1)

Shubham
Shubham on 6 Sep 2024
Hi Alex,
To compute feature importance using the Gini index (or Gini impurity) in a random forest model with MATLAB's TreeBagger, you can use the OOBPermutedPredictorDeltaError property. This property provides a measure of feature importance based on the increase in prediction error when the values of a given feature are permuted. Here's how you can do it:
Step-by-Step Guide to Compute Feature Importance
  1. Train Your Random Forest Model: You already have this step covered with your existing code.
  2. Calculate Feature Importance: Use the OOBPermutedPredictorDeltaError property to get the importance of each feature.
Here's how you can modify your code to include feature importance calculation:
% Number of trees
nTrees = 50;
% Train the TreeBagger model
B2 = TreeBagger(nTrees, train_data, train_label, 'Method', 'classification', 'OOBPredictorImportance', 'on');
% Predict the test data
predChar2 = B2.predict(test_data);
predictedClass2 = str2double(predChar2);
% Compute feature importance
featureImportance = B2.OOBPermutedPredictorDeltaError;
% Display feature importance
fprintf('Feature Importance (using Gini index):\n');
for i = 1:length(featureImportance)
fprintf('Feature %d: %.4f\n', i, featureImportance(i));
end
% Plot feature importance
figure;
bar(featureImportance);
title('Feature Importance (Gini Index)');
xlabel('Feature Number');
ylabel('Importance');

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!