ROC curve with multiclass SVM

Hello All,

I am plotting the ROC curve for SVM multiclass(3 classes) task. Getting the error as "Index in position 2 exceeds array bounds (must not exceed 3)"

Error in ROC_SVM (line 70)

scores = double(score(:,final_best_SVM.ClassNames + 1))'; % Compute the posterior probabilities (scores)

My code is

%% Plotting ROC curve for SVM
[predictedLabels,score,cost] = predict(final_best_SVM, features_Testing); 
scores = double(score(:,final_best_SVM.ClassNames + 1))'; % Compute the posterior probabilities (scores)
figure(2)
plotroc(dtTest_lab,scores)
title('ROC Curve for SVM')

[Gorkem Akgul]

Could you solve the problem ?

Answer 1

[Harsha Priya Daggubati]

Hi,

The number of columns of score matrix will be equal to your classes, in your case it is 3. Since you are using +1 in the following line, this issue pops up.

scores = double(score(:,final_best_SVM.ClassNames + 1))

[kav]

Thank you for the answer. If i change ClassNames + 1 it to ClassNames + 0 it should work right but isn't.

What other changes should i make?

Thank you

[Harsha Priya Daggubati]

What is the issue when changed to 0?

[kav]

it just gives one diagonal line no curves.

[Harsha Priya Daggubati]

Can you try using this:

[Xnb,Ynb,Tnb,AUCnb] = perfcurve(dtTest_lab,score(:,final_best_SVM.ClassNames(1)),'true'); 
plot(Xnb,Ynb);

Do the same for each class.

[kav]

tried the above code got error as

Error using perfcurve>preparedata (line 1270)

You must pass scores as a vector of floating-point values.

Error in perfcurve (line 393)

[scores,cls,weights,ncv] = preparedata(scores,cls,weights);

Error in ROC_SVM (line 99)

[Xnb,Ynb,Tnb,AUCnb] = perfcurve(dtTest_lab,score(:,final_best_SVM.ClassNames),'true');

[Harsha Priya Daggubati]

I edited the recent reply. Can you try using it.

[kav]

Thank you so much for your time. But still i am getting error

Error using perfcurve>preparedata (line 1282)

The size of scores does not match the size of labels.

Error in perfcurve (line 393)

[scores,cls,weights,ncv] = preparedata(scores,cls,weights);

if i send my full code will it be easier for to solve?

[Harsha Priya Daggubati]

Yeah that would help me better!

[Harsha Priya Daggubati]

Hi,

I assume the SVM trained is working fine. ROC usually plots TPR Vs FPR and is mostly used for binary classification. To extend it for multi-class classification you have to binarize the output - one ROC curve can be drawn for label.

So, I guess doing this for every class works:

[Xnb,Ynb,Tnb,AUCnb] = perfcurve(dtTest_lab(1),score(:,final_best_SVM.ClassNames(1)),'true');   

[kav]

Thanks again, i edited my code. but same error

figure
[predictedLabels,score,cost] = predict(final_best_SVM, features_Testing);
    
[Xnb,Ynb,Tnb,AUCnb] = perfcurve(dtTest_lab(1),score(:,final_best_SVM.ClassNames(1)),'true');   
plot(Xnb,Ynb);
hold on 
[Xnb2,Ynb2,Tnb2,AUCnb2] = perfcurve(dtTest_lab(2),score(:,final_best_SVM.ClassNames(2)),'true');
plot(Xnb2,Ynb2);

hold on 
[Xnb3,Ynb3,Tnb3,AUCnb3] = perfcurve(dtTest_lab(3),score(:,final_best_SVM.ClassNames(3)),'true');
plot(Xnb3,Ynb3);
xlabel('False positive rate')
ylabel('True Positive rate') 
hold off

[Harsha Priya Daggubati]

What was the error? Try converting scores to double as you did previously

[kav]

edited the code, and got the error as below

figure
[predictedLabels,score,cost] = predict(final_best_SVM, features_Testing);
score = double(score(:,final_best_SVM.ClassNames));
    
[Xnb,Ynb,Tnb,AUCnb] = perfcurve(dtTest_lab(1),score(:,final_best_SVM.ClassNames(1)),'true');   
plot(Xnb,Ynb);
hold on 
[Xnb2,Ynb2,Tnb2,AUCnb2] = perfcurve(dtTest_lab(2),score(:,final_best_SVM.ClassNames(2)),'true');
plot(Xnb2,Ynb2);

hold on 
[Xnb3,Ynb3,Tnb3,AUCnb3] = perfcurve(dtTest_lab(3),score(:,final_best_SVM.ClassNames(3)),'true');
plot(Xnb3,Ynb3);
xlabel('False positive rate')
ylabel('True Positive rate') 
hold off

Error using perfcurve>preparedata (line 1282)

The size of scores does not match the size of labels.

Error in perfcurve (line 393)

[scores,cls,weights,ncv] = preparedata(scores,cls,weights);

Error in ROC_SVM (line 110)

[Xnb,Ynb,Tnb,AUCnb] = perfcurve(dtTest_lab(1),score(:,final_best_SVM.ClassNames(1)),'true');

[Harsha Priya Daggubati]

This should work fine. Can you share the csv file too. Ill look into this in much detail

[kav]

yes attached the file.

[Harsha Priya Daggubati]

Hi, the target you are specifying must be in a binary format (0 and 1) as you are dealing with each class.

figure
[predictedLabels,score,cost] = predict(final_best_SVM, features_Testing);
score = double(score(:,final_best_SVM.ClassNames));
    
[Xnb,Ynb,Tnb,AUCnb] = perfcurve(classes_Testing==1,score(:,final_best_SVM.ClassNames(1)),'true');   
plot(Xnb,Ynb);
hold on 
[Xnb2,Ynb2,Tnb2,AUCnb2] = perfcurve(classes_Testing==2,score(:,final_best_SVM.ClassNames(2)),'true');
plot(Xnb2,Ynb2);
hold on 
[Xnb3,Ynb3,Tnb3,AUCnb3] = perfcurve(classes_Testing==3,score(:,final_best_SVM.ClassNames(3)),'true');
plot(Xnb3,Ynb3);
xlabel('False positive rate')
ylabel('True Positive rate') 
hold off

[kav]

Thank you so much for your time and help. It worked!

[Harsha Priya Daggubati]

What is the issue with this case?

[kav]

I think its same as before problem.

Error using network/sim (line 270)

Input data sizes do not match net.inputs{1}.size.

Error in ROC_MLP (line 79)

simpleclusterOutputs = sim(netBest,features_Training)