Using datastore for a matrix
8 views (last 30 days)
Show older comments
Hi
I am trying to set up a datastore in order to train a feedforward neural network with a trainingset that is too large for my workspace.
I have a training set, X, that is 80002x10000 which I stored in a .mat file called ¨TrainingData'. I am now trying to set up a datastore so I can load mini batches and train on those, but I can't get it working.
I have tried datastore and arrayDatastore.
datastore cannot determine the datastore type for the specified location and arrayDatastore just output the name of the .mat-file whenever i read, i.e.
>> read(ds)
ans =
1×1 cell array
{'X.mat'}
What am I doing wrong here?
EDIT
I just read that the arrayDatastore is only for in-memory data which makes it useless in my case
Thanks
0 Comments
Accepted Answer
Catalytic
on 9 Sep 2023
If you're not going to store your data in memory, then the training samples have to be split up into separate .mat files. You could then use a fileDatastore to read from and manage them.
2 Comments
More Answers (0)
See Also
Categories
Find more on Datastore in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!