Adding confusion code for LSTM classification on audio files in Matlab

in the following code I have applied LSTM on audio files. Now I want to add confusion matrix for the results. Please advise me.

clear all
close all

TrainRatio=0.8;
ValidationRatio=0.1;

folder='/Users/pooyan/Documents/normal/';  % change this path to your normal data folder
audio_files=dir(fullfile(folder,'*.ogg'));
nfileNum=length(audio_files);
%nfileNum=200
normal=[];
for i = 1:nfileNum 
    normal_name = [folder audio_files(i).name]; 
    normal(i,:) = audioread(normal_name);   
end 
normal=normal';
nLabels = repelem(categorical("normal"),nfileNum,1);

folder='/Users/pooyan/Documents/anomaly/'; % change this path to your anomaly data folder
audio_files=dir(fullfile(folder,'*.ogg'));
afileNum=length(audio_files);
anomaly=[];
for i = 1:afileNum 
    anomaly_name = [folder audio_files(i).name]; 
    anomaly(i,:) = audioread(anomaly_name);   
end 
anomaly=anomaly';
aLabels = repelem(categorical("anomaly"),afileNum,1);

% randomize the inputs if necessary
% normal=normal(:,randperm(nfileNum, nfileNum));
% anomaly=anomaly(:,randperm(afileNum, afileNum));

nTrainNum = round(nfileNum*TrainRatio);
aTrainNum = round(afileNum*TrainRatio);
nValidationNum = round(nfileNum*ValidationRatio);
aValidationNum = round(afileNum*ValidationRatio);

audioTrain = [normal(:,1:nTrainNum),anomaly(:,1:aTrainNum)];
labelsTrain = [nLabels(1:nTrainNum);aLabels(1:aTrainNum)];

audioValidation = [normal(:,nTrainNum+1:nTrainNum+nValidationNum),anomaly(:,aTrainNum+1:aTrainNum+aValidationNum)];
labelsValidation = [nLabels(nTrainNum+1:nTrainNum+nValidationNum);aLabels(aTrainNum+1:aTrainNum+aValidationNum)];

audioTest = [normal(:,nTrainNum+nValidationNum+1:end),anomaly(:,aTrainNum+aValidationNum+1:end)];
labelsTest = [nLabels(nTrainNum+nValidationNum+1:end); aLabels(aTrainNum+aValidationNum+1:end)];

fs=44100;

%  Create an audioFeatureExtractor object 
%to extract the centroid and slope of the mel spectrum over time.
aFE = audioFeatureExtractor("SampleRate",fs, ...    %Fs
    "SpectralDescriptorInput","melSpectrum", ...
    "spectralCentroid",true, ...
    "spectralSlope",true);

featuresTrain = extract(aFE,audioTrain);
[numHopsPerSequence,numFeatures,numSignals] = size(featuresTrain);
numHopsPerSequence;
numFeatures;
numSignals;
%treat the extracted features as sequences and use a
%sequenceInputLayer as the first layer of your deep learning model. 

featuresTrain = permute(featuresTrain,[2,1,3]); %permute switching dimensions in array
featuresTrain = squeeze(num2cell(featuresTrain,[1,2]));%remove dimensions

numSignals = numel(featuresTrain); %number of signals of normal and anomalies

[numFeatures,numHopsPerSequence] = size(featuresTrain{1});


%Extract the validation features.

featuresValidation = extract(aFE,audioValidation);
featuresValidation = permute(featuresValidation,[2,1,3]);
featuresValidation = squeeze(num2cell(featuresValidation,[1,2]));



%Define the network architecture.

layers = [ ...
    sequenceInputLayer(numFeatures)
    lstmLayer(50,"OutputMode","last")
    fullyConnectedLayer(numel(unique(labelsTrain))) %%labelTrain=audio
    softmaxLayer
    classificationLayer];

%To define the training options

options = trainingOptions("adam", ...
    "Shuffle","every-epoch", ...
    "ValidationData",{featuresValidation,labelsValidation}, ... %%labelValidatin=audioValidation
    "Plots","training-progress", ...
    "Verbose",false);

%To train the network

net = trainNetwork(featuresTrain,labelsTrain,layers,options);


%Test the network %10 preccent 


%classify(net,permute(extract(aFE,audioTest),[2 257 35]))

TestFeature=extract(aFE, audioTest);
for i=1:size(TestFeature, 3)

TestFeatureIn = TestFeature(:,:,i)';
classify(net,TestFeatureIn)

end

Answer 1

[Brian Hemmat]

You can use confusionchart: https://www.mathworks.com/help/deeplearning/ref/confusionchart.html

Your output from classify will be your predicted labels, your the targets will be the known labels (it looks like 'labelsTest' in your code.

[Pooyan Mobtahej]

Is that correct ?, since still I got error, How should I fix that?

predict = classify(net,TestFeatureIn);

confusionchart(labelsTest,predict)

Error:

Error using confusionchart (line 68)

Vectors of true and predicted labels must have the same number of observations.

Error in audioDNN (line 118)

confusionchart(labelsTest,predict)

[Pooyan Mobtahej]

Can you also let me know how can apply cross validation for 10 percents of the data and for 10 iterations?

[Brian Hemmat]

As the error says, you need the same number of in labelsTest as in predict... Based on what I can see, labelsTest corresponds to the entire test set, while predict corresponds to the result of one sample (but it should contain the predictions for the entire test set).

This example performs cross-validation:

https://www.mathworks.com/help/audio/ug/speech-emotion-recognition.html#SequentialFeatureSelectionForSpeechEmotionRecognitionExample-5

I'm guessing by iterations you mean what's usually referred to as 'folds'. At a high level, what you'll do is place a loop around all of your code and separate out a random 10% for testing and use the rest for training for 10 iterations. The final metric you report is the average of each iteration/fold.

This page has some general information about cross validation:

https://www.mathworks.com/discovery/cross-validation.html

[Pooyan Mobtahej]

So how can I correct the labelsTest and predict, predict is classification of from test i guess it should be correct numbers, how can I correct that ? Thanks

[Pooyan Mobtahej]

[Pooyan Mobtahej]

You can look at the results here

[Pooyan Mobtahej]

Should I make an array for put the classification data in it for prediction ? if you let me know the function that would be great