How to count up an error in svm

Hello, I'm using 'svmtrain' in MATLAB R2013a and I'm looking for function counting a training error.

I drew a picture where data are separated by the classifier, but there is one red dot in wrong side and that's the error I mean...

So how can i count it up??

Thanks

Answer 1

[Ilya]

Classification error by definition is the fraction of misclassified observations. So if you have one misclassified point, divide 1 by the number of observations in your data.

Take a look at 'help confusionmat' or 'doc confusionmat'.

[Josef]

OK, that's nice, but I need a value, which gives me the result... I don't know what shows me the matrix :( I am using this code...

And I need errorTRN and margin, but the error is more important for me...

if true
tic;
figure
model = svmtrain(trn.X,trn.y,'Kernel_Function','rbf', ...
  'boxconstraint',Inf,'showplot',true);
time=toc;
hold on
axis equal
hold off
y_pred = svmclassify(model,tst.X,'showplot',true);
errorTST=sum(y_pred~=tst.y)/length(tst.y)*100; 
 end

[Ilya]

Since you know how to compute the error for the test data, use the same recipe to compute the error for the training data. Just replace tst with trn in these two lines (copied from your code):

y_pred = svmclassify(model,tst.X,'showplot',true);
errorTST=sum(y_pred~=tst.y)/length(tst.y)*100;

To see the confusion matrix for the test data, do

confusionmat(tst.y,y_pred)

and similarly for the training data.

I don't know what you mean by "margin". Classification margin is defined per observation. Using that definition, you would have a vector of margins of size numel(tst.y) for your test data.

[Josef]

I can't use

errorTST=sum(y_pred~=tst.y)/length(tst.y)*100;

because y_pred is a vector of species formed during svmclassify and it's another vector, than I need to use...even lengths are not the same... but svmtrain doesn't make any vector of species :( I have an idea, but I don't know how to solve a problem. How to get values from the plot into matrix... If I get this data, I'll get a vector of species...

And the margin is simply absolute distance from SVM to classifier by both sides. I don't know how to say it better :/

[Josef]

I have finaly ascertained the problem, because I used the svmclassify with training dat to get a vector of species of trained data. Here is the result:

y_trn = svmclassify(model,trn.X,'showplot',false); % get a species of training data
errorTRN = sum(y_trn~=trn.y)/length(trn.y)*100;

But I still don't know how to get margin value :(

[Ilya]

It is still unclear to me what you mean by "margin". You could mean the minimal margin in the training data or you could mean the geometric margin, that is, distance between the hyperplanes which represent the class boundaries. Either way, the svmtrain and svmclassify functions do not provide a convenient way to obtain this info. See this thread on computing SVM scores. Use that recipe and the mathematical definition of the margin (whichever you choose) to figure this out.

[Josef]

Aha, I mean geometric margin, but I don't know how to do it. I know, that margin=2/ norm(w), but i'm not sure, what is the "w"

[Ilya]

Using notation from the thread I pointed at, w is given by

w = sv'*alphaHat

Keep in mind that if you set 'autoscale' to true when you called svmtrain, input variables are standardized. Then you get w in the space of standardized variables.

[Josef]

Yes, I was trying that, but there is a problem... It works good, but only in linear svm... Another kernels are using own kernel transform of the sv to other dimensions, but I have not found the kernel functions...(or I found them a lot, but which one can I use) :/

[Ilya]

I have never seen anyone trying to estimate the geometric margin for non-linear SVM. The geometric margin is only used in textbooks for pedagogical reasons, using the picture just like the one you've attached.

The margin defined as y*f for class label y in set {-1,+1} and soft score f is a useful concept in the learning theory. But that's not what you want.

I suspect you are somewhat confused about what you really want to compute. Either that or I have no idea what you are doing.