How do I get the distance between the point and the hyperplane using libsvm?

I am using libsvm. I need to know, which observations are farest away from the hyperplane. libsvm returns me the "decision_value" but how can I use it to get the distance from the hyperplane? Taking the largest positive and smallest negative values or do I have to compute it manually and if yes, how?

Answer 1

[Rishabh Gupta]

Hey Stef

You can find the distance of a point i from hyperplane as follows:

distance_i = |decision_value|_i / |w|-b

where,

w = (alpha * support_vectors)
|w| = sqrt(sum(w^2))

alphas, support_vectors and b is generated from SVM model

[Stef]

Thank you for your answer. Here you can see the parameters I receive.

I assume the bias b is model.rho. But what about w, is w the model.sv_coef? And what about alpha? Could you please explain

[Rishabh Gupta]

Hi Stef

Yes, b is the model.rho

w is calculated as written in the answer

w = (alpha * support_vectors)
|w| = sqrt(sum(w^2))

alpha = sv_coef in your model

support_vector = SVs in your model

Using the formula above calculate w and plug it in below formula

distance_i = |decision_value|_i / |w|-b

[Stef]

Thank you very much. Just one last question: If I want to have the distances separately per class i.e. the one most far away from the hyperplane belonging to class -1 and the one most far away from the hyperplane belonging to class 1, do I receive these with the largest and the smallest value of distance_i? Or are the values of one class positive and of the other class negative?

[Stef]

SV_indices contrains the index of the Support vectors in the original matrix. The problem is that I want to find the 5% of observations which are most likely in the -1 category. Therefore I take the x observations which are furthest away from the hyperplane in one direction and the rest (5%-x) which are closest to the hyperplane but in class 1.

[Stef]

See here an example for the fisher Iris. Is there a possibility to find the on which side of the hyperplane the observations are?

load 'fisheriris'

% create X and y
X = meas([1:100],[3:4]);
y = grp2idx(species);

% recode 2 to -1 that lables are 1 and -1
y(y==2) = -1;

%create training and testing sample
rand = randperm(100);

y_train = y(rand([1:80]),:);
X_train = X(rand([1:80]),:);
y_test = y(rand([81:100]),:);
X_test = X(rand([81:100]),:);

% SVM
options =[ '-s 0 -t 0']
[model] = svmtrain(y_train, X_train, options)  
[predict_label, accuracy, decision_values] = svmpredict(y_test, X_test, model);

% find distance
w = model.sv_coef' * model.SVs;
w_abs = sqrt(sum(w.^2));
bias = model.rho;
distance = abs(decision_values) ./ (w_abs-bias);