Loading multiple .mat files by using for loop

Hello, I am trying to load some .mat files and run some specific commands on. The .mat files are named as the following:

 % Data Files
dataset1_1.mat     dataset2_1.mat     dataset3_1.mat     ...and so on
dataset1_2.mat     dataset2_2.mat     ...and so on
dataset1_3.mat     ...and so on 
...and so on

I want to first load all the dataset1_*.mat files and then perform some calculations on each of them in order (i.e. perform calculations on dataset1_1.mat, then dataset1_2.mat...and so on).

After completing all of the dataset1_*.mat files, I would like to move onto the next batch of files, dataset2_*.mat, and repeat(i.e. perform calculations on dataset2_1.mat, then dataset2_2.mat...and so on).

Process will continue until the last dataset*_*.mat file.

Here is my code so far

% code
files = dir('dataset*_1.mat');
for i = length(files)
  files2 = dir('dataset(i)_*.mat');
  for j = 1:length(files2)
      a = load(files2(j).name);

        %Perform some calculations on dataset

    end
end

I am using 2 for loops, not sure where I am going wrong. Thanks for the help!

Answer 1

[Orion]

the line

_files = dir('dataset*_1.mat');_

returns dataset1_1, dataset2_1,... not what you want. then

files2 = dir('dataset(i)_*.mat')

will return nothing, because (i) is not interpreted, you're asking for a file exactly named 'dataset(i)_*.mat'.

you should do something like

% number of set (dataset1, dataset2, ...if unknown, figure it out).
NumberOfDataset = 3;
for i = 1:NumberOfDataset 
    % get allfiles matching the pattern 'dataset(i)_*'
    files = dir(sprintf('dataset%d_*.mat',i));
    for j = 1:length(files)
        fprintf('Current file : %s\n',files(j).name)
        a = load(files(j).name);
        %Perform some calculations on dataset
    end
end

[Guillaume]

Orion, the first dir makes sense, it allows to retrieve the number of prefixes, i.e. finding the NumberOfDataset in your example.

Both approaches are a bit brittle though. What if a number is mising in the middle of a set? You then have a mismatch between your loop count and true file number.

[Orion]

@Guillaume

I agree with you, the line

files = dir('dataset*_1.mat');

can be used to determine the number of dataset (what i meant by figure it out). maybe i was not enough clear.

so at the end :

% number of set
files_1 = dir('dataset*_1.mat');
NumberOfDataset = length(files_1);

for i = 1:NumberOfDataset
    % get allfiles matching the pattern 'dataset(i)_*'
    files = dir(sprintf('dataset%d_*.mat',i));
    for j = 1:length(files)
        fprintf('Current file : %s\n',files(j).name)
        a = load(files(j).name);
        %Perform some calculations on dataset
    end
end

And about your 2nd comment, if there is a missing file, there will obviously be some trouble, but that's another problem.

[Tasadduq]

Thanks, it works!

This script is assuming iset goes up to 3, but what about if it does not go up to 3 (it is unknown). How would I incorporate that into the script?

[Orion]

same remark Guillaume did.

look at the 2nd comment I posted. here's your answer (determination of NumberOfDataset, and no more use of iset).

[Guillaume]

Personally, I would extract the numbers from the file names, sort them, and iterate over these real numbers rather than a count of them. This way you would avoid (or detect) problems when a file is missing.

[Tasadduq]

Thank you both. Yes, extracting the numbers from the file names and sorting & iterating is a great idea.