MATLAB Answers

How to check SVM model?

118 views (last 30 days)
Hi everyone,
I need your help for my project.
I have already built an SVM model for classification with 4 labels. The SVM model worked very well. Accuracy classification reaches more than 90%.
However, when I want to check the model with new data ( new data = the original data through an AWGN channel having a 10 dB signal-to-noise ratio (SNR). The classification result is always less than 30% accuracy.
I don't know why despite trying so many ways. Pls help me!!!
untitled.jpg
untitled1.jpg
My code is as follows:
%% preparing data
load('mydata.mat') % including 200 observers and 120 features, 4 labels
output = grp2idx(Y);
rand_num = randperm(size(X,1));
% training data set 70%, test set 30%,
X_train = X(rand_num(1:round(0.7*length(rand_num))),:);
y_train = output(rand_num(1:round(0.7*length(rand_num))),:);
X_test = X(rand_num(round(0.7*length(rand_num))+1:end),:);
y_test = output(rand_num(round(0.7*length(rand_num))+1:end),:);
%% Train a classifier
% This code specifies all the classifier options and trains the classifier.
template = templateSVM(...
'KernelFunction', 'linear', ...
'PolynomialOrder', [], ...
'KernelScale', 'auto', ...
'BoxConstraint', 1, ...
'Standardize', true)
Mdl = fitcecoc(...
X_train, ...
y_train, ...
'Learners', template, ...
'Coding', 'onevsall',...
'OptimizeHyperparameters','auto',...
'HyperparameterOptimizationOptions',...
struct('AcquisitionFunctionName',...
'expected-improvement-plus'));
%% Perform cross-validation
partitionedModel = crossval(Mdl, 'KFold', 10);
% Compute validation predictions
[validationPredictions, validationScores] = kfoldPredict(partitionedModel);
% Compute validation accuracy
validation_error = kfoldLoss(partitionedModel, 'LossFun', 'ClassifError'); % validation error
validationAccuracy = 1 - validation_error;
%% test model
oofLabel_n = predict(Mdl,X_test);
oofLabel_n = double(oofLabel_n); % chuyen tu categorical sang dang double
test_accuracy_for_iter = sum((oofLabel_n == y_test))/length(y_test)*100;
%% save model
saveCompactModel(Mdl,'mySVM');
  3 Comments
Le Truong An
Le Truong An on 8 Jul 2019
Thank for your suprort!
I attach my data and my code is used to test the SVM model.
Look forward to your advice!
clc
clear all
%% preparing data
%load('mydata.mat')
load('dataWithNoiseSNR10dBForTest.mat');
output = grp2idx(Y);
rand_num = randperm(size(X,1));
% training data set 70%, test set 30%,
X_train = X(rand_num(1:round(0.7*length(rand_num))),:);
y_train = output(rand_num(1:round(0.7*length(rand_num))),:);
X_test = X(rand_num(round(0.7*length(rand_num))+1:end),:);
y_test = output(rand_num(round(0.7*length(rand_num))+1:end),:);
%% load and test SVM model with noise
CompactMdl = loadCompactModel('mySVM');
oofLabel_n = predict(CompactMdl,X_test);
test_accuracy_for_iter = sum((oofLabel_n == y_test))/length(y_test)*100; % tinh accuracy rate
%% plotconfusion
isLabels = unique(output);
nLabels = numel(isLabels);
[n,p] = size(X_test);
% Convert the integer label vector to a class-identifier matrix.
[~,grpOOF] = ismember(oofLabel_n,isLabels);
oofLabelMat = zeros(nLabels,n);
idxLinear = sub2ind([nLabels n],grpOOF,(1:n)');
oofLabelMat(idxLinear) = 1; % Flags the row corresponding to the class
[~,grpY] = ismember(y_test,isLabels);
YMat = zeros(nLabels,n);
idxLinearY = sub2ind([nLabels n],grpY,(1:n)');
YMat(idxLinearY) = 1;
figure;
plotconfusion(YMat,oofLabelMat);
h = gca;
h.XTickLabel = [(isLabels); {''}];
h.YTickLabel = [(isLabels); {''}];
title('Add white Gaussian noise to original data (SNR=10dB) ','FontWeight','bold','FontSize',12);

Sign in to comment.

Accepted Answer

Don Mathis
Don Mathis on 8 Jul 2019
If you want your classifier to perform well on data with Gaussian noise added, I suggest training it on your original data with Gaussian noise added. That is, create an "augmented" dataset and train on that.
  5 Comments
Le Truong An
Le Truong An on 9 Jul 2019
It seems that the results are very good. Thank you so much!
I will try it with some other popular classification methods like k-NN, Decision Trees, ANN.
Thank you again, Mr. Don Mathis!

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!