VGGishPreprocess not accepting input

Hello,

I'm trying to do some transfer learning on the VGGish network with my own data but can't seem to get VGGishPreprocess to accept it.

The code is:

net=vggish;


downloadFolder = fullfile(tempdir,'Data_Test1');
datasetLocation = tempdir;

ads = audioDatastore(downloadFolder,'IncludeSubfolders',true,'LabelSource','foldernames');
labelTable = countEachLabel(ads)

numClasses = size(labelTable,1);

[adsTrain, adsValidation] = splitEachLabel(ads,0.8);
countEachLabel(adsTrain)

countEachLabel(adsValidation)

overlapPercentage = 75;
fs = 16000;

[trainFeatures,trainLabels] = vggishPreprocessNEW(adsTrain,overlapPercentage); 

I should note that

vggishPreprocessNEW

is the variation of vggishPreprocess that is supplied in this article, rather than the one offered in the Audio Toolbox.

The error i end up with is:

I have run the same code on the ESC-10 dataset that is provided, which works fine.

I have copied ESC-10's file structure and format, reducing all my input files to the same size and length.

I am now unsure what else is left to test, I feel I am running my test in exactly the same way but with a different output.

System is MacBook Pro M1, 2020, Monterey 12.1, 8GB

Many thanks in advance for any help, it is greatly appreciated.

Answer 1

[jibrahim]

Hi Joshua,

First, sorry we had a local funtion name with the same name as a function we ship. We are fixing this.

The function you copied and you're working off does not do well when the audio has multiple channels (for example, stereo). I suspect this is why you are seeing this error. If you only care about one channel, you can use indexing to just use the first channel of the audio. If you care about all channels, use this code instead (it uses the documented vggishPreprocess function. Make sure you do not have local function with that name around):

function [predictor,response,segmentsPerFile] = vggishPreprocessDatastore(ads,overlap)

% Preallocate cell arrays for the predictors and responses
numFiles = numel(ads.Files);
predictor = cell(numFiles,1);
response = predictor;
segmentsPerFile = zeros(numFiles,1);

% Extract predictors and responses for each file
for ii = 1:numFiles
    [audioIn,info] = read(ads);

    duration = size(audioIn,1)/info.SampleRate;

    if duration>=0.975
        % Use the documented vggishPreprocess function
        predictor{ii} = vggishPreprocess(audioIn,info.SampleRate,"OverlapPercentage",overlap);
        numHops = size(predictor{ii},4);
        response{ii} = repelem(info.Label,numHops);
        segmentsPerFile(ii) = numHops;
    else % signal too short - no results
        predictor{ii} = [];
        response{ii} = [];
        segmentsPerFile(ii)=0;
    end
end

% Concatenate predictors and responses into arrays
predictor = cat(4,predictor{:});
response = cat(2,response{:});
end

[Joshua Gardner]

Hi Jibrahim,

both these solutions worked, I forgot my dataset was in stereo!

thanks so much for the response, I really appreciate it