MATLAB Answers

0

Visual bag-off-features evalute vs. fitcecoc classification results are very different

Asked by Preetham Manjunatha on 9 Nov 2017
Latest activity Edited by Preetham Manjunatha on 9 Nov 2017
Dear All:
I have few questions regarding vision toolbox's visual bags-of-features class. Firstly, I have a feature matrix created using (encode, bagOfFeatures functions) visual-bags-of-words using computer vision toolbox, it is 500 x 14404 (Xtrain = 10793, XVal = 1204 and XTest = 2407 samples). There are 14 classes, target matrix is 14 x 14404 or 14404 X 1 categroical classes. When I use Matlab's default method "evaluate" provided in visual bags-of-words example (I presume it uses multiclass SVM "fitcecoc") to classify the test data I am getting a decent precison/recall values around 75% which is decent for the given dataset. Code snippet is given below:
categoryClassifier = trainImageCategoryClassifier(imdsTrainRandomized, bag_Train_BoFOri);
[confMatTr,knownLabelIdxTr,predictedLabelIdxTr,scoreTr] = evaluate(categoryClassifier,imdsTrainRandomized);
[confMatVl,knownLabelIdxVl,predictedLabelIdxVl,scoreVl] = evaluate(categoryClassifier,imdsValRandomized);
[confMatTs,knownLabelIdxTs,predictedLabelIdxTs,scoreTs] = evaluate(categoryClassifier,imdsTestRandomized);
Whereas, if I use fitcecoc to classify train/test dataset using multiclass SVM. I am getting testing accuracy of only ~9-10%. Precision/recall values less than 8%. Also, I noticed runtime takes less than 3 minutes to train, where I am skeptical about. Below is the multiclass SVM code snippet:
t = templateSVM('KernelFunction','polynomial', 'PolynomialOrder',2);
options = statset('UseParallel',0);
MdlSVM = fitcecoc(Xtrain,Labelstrain, 'Coding','onevsone','Learners',t ,...
'Prior','uniform','Options',options);
isLoss = resubLoss(MdlSVM);
CVSVMModel = crossval(MdlSVM);
FirstModel = CVSVMModel.Trained{1};
yTestSVM = predict(FirstModel,Xtest);
percentErrorsSVM = sum(yTestSVM ~= Labelstest)/numel(Labelstest);
accuracySVM = 1 - percentErrorsSVM
Is there a way we can find the optimal parameters tuned in "evaluate" method? Also, I appreciate if there are any suggestions.
Thanks,

  0 Comments

Sign in to comment.

0 Answers