Out of memory gather()

I am reading the documentation on this topic Deep Learning Denoise. After this line my computer is not responding.

[targets,predictors] = gather(targets,predictors);

Error: Out of memory.

I change number of dataset in this line to 300 and all works good

adsTrain = audioDatastore(fullfile(dataFolder,'train'),'IncludeSubfolders',true);

reduceDataset = true;
if reduceDataset
    adsTrain = shuffle(adsTrain);
    adsTrain = subset(adsTrain,1:1000); % to 300
end

but sometimes i get this Error.

How can I examine the entire dataset if it is really big (87000 files). Or at least 1000. I read the documentation about gather (), but I still don't understand how to get around this Error.

Answer 1

[jibrahim]

Hi Maxim,

In many practical situations, your data might not all fit in memory. There is a modified workflow for that case. See, for example:

https://www.mathworks.com/help/signal/ug/train-spoken-digit-recognition-network-using-out-of-memory-features.html?searchHighlight=out%20of%20memory%20features&s_tid=srchtitle

and

https://www.mathworks.com/help/audio/ug/train-spoken-digit-recognition-network-using-out-of-memory-audio-data.html?searchHighlight=out%20of%20memory%20features&s_tid=srchtitle

[Maxim Rozhkov]

jibrahim, thank you

Answer 2

[neal paze]

I also have a problem of gather in this document. Have you solved your problem? Can you share how to modify this line of code?