MATLAB Answers

Import range of files based on file name

10 views (last 30 days)
Tessa Kol
Tessa Kol on 3 Sep 2020
Commented: Tessa Kol on 9 Sep 2020
I have a folder containing around 300,000 files. I don't need to import all the files.
Problem: How can I import a range of files based on specific file name?
This is an extension on the problem I asked in another post, link: https://www.mathworks.com/matlabcentral/answers/587738-import-files-based-on-file-name#answer_488336
Problem example:
In the picture below I have a section of the files, which are all in the same folder. I only want to import the .data files. But I don't need all the .data files to be import only the last 5 of every serie.
Up until now I have the following code (thanks to Stephen Cobeldick):
% Select only .data file from the last time step of each simulation
Folder = uigetdir;
files = dir(fullfile(Folder,'*.data*'));
spl = regexp({files.name},'\.data\.','split','once');
spl = vertcat(spl{:});
vec = str2double(spl(:,2));
[~,idx] = sort(vec);
[~,idy,idz] = unique(spl(idx,1),'last');
out = {files(idx(idy)).name};
expData = cell(length(out),1);
for i = 1:length(out)
fid = fopen(fullfile(Folder,out{i}),'r');
%% Reading the data
% Read all the data from the file
dataRead = textscan(fid,'%f %f %f %f %f %f %f %f %f %f %f %f %f %f','HeaderLines',1);
end
This code only extracts the last .data file of every serie, but as I explained I want to have the last 5 files of every serie.

  0 Comments

Sign in to comment.

Accepted Answer

Cris LaPierre
Cris LaPierre on 4 Sep 2020
Acknowledging there are a lot of ways to do this, here's how I thought to do it. It borrows from the code you've shared.
% Select only .data file from the last time step of each simulation
Folder = "datafiles";
files = dir(fullfile(Folder,'*.data*'));
spl = regexp({files.name},'\.data\.','split','once')';
% Convert the cell to a table so that you can sort and group two different data types
tbl = splitvars(cell2table(spl),1,"NewVariableNames",["Name","Ext"]);
tbl.Ext = str2double(tbl.Ext);
% this gives you the unique experiments and the last extension in the series
exp = groupsummary(tbl,"Name","max","Ext");
for n = 1:height(exp)
for i = 4:-1:0
% Use the Name and Ext values to rebuild the file names
fid = fopen(fullfile(Folder,exp.Name(n)+".data." + string(exp.max_Ext(n)-i)))
% Read all the data from the file
dataRead = textscan(fid,'%f %f %f %f %f %f %f %f %f %f %f %f %f %f','HeaderLines',1);
end
end

  13 Comments

Tessa Kol
Tessa Kol on 9 Sep 2020
I ran that whole code so many times and I didn't got an error. But how can I perform the same procedure for the last 5 data files?
Cris LaPierre
Cris LaPierre on 9 Sep 2020
Ah, I misunderstood how the code ran.
How to modify it? Replace the code that opens every file with the code I shared already that opens the last 5 in every series.
rhoPart = 2500;
files = dir(fullfile(uigetdir,'*.data*'));
spl = regexp({files.name},'\.data\.','split','once')';
% Convert the cell to a table so that you can sort and group two different data types
tbl = splitvars(cell2table(spl),1,"NewVariableNames",["Name","Ext"]);
tbl.Ext = str2double(tbl.Ext);
% this gives you the unique experiments and the last extension in the series
exp = groupsummary(tbl,"Name","max","Ext");
c=1;
for n = 1:height(exp)
for i = 4:-1:0
% Use the Name and Ext values to rebuild the file names
fid = fopen(fullfile(Folder,exp.Name(n)+".data." + string(exp.max_Ext(n)-i)))
%% Reading the data
dataRead = textscan(fid,'%f %f %f %f %f %f %f %f %f %f %f %f %f %f','HeaderLines',1);
frewind(fid);
% Write headerline N, time, xmin, ymin, zmin, xmax, ymax, zmax
runData{c} = strsplit(fgetl(fid), {' ','\t'});
% Write only the x, y, and z components of the particles, particle radius,
% z component+ particle radius and volume of the particle
expData{c} = [dataRead{1}(:,1) dataRead{2}(:,1) dataRead{3}(:,1) dataRead{7}(:,1) dataRead{3}(:,1)+dataRead{7}(:,1) rhoPart*(4/3)*pi*(dataRead{7}(:,1).^3)];
% Write only the vx,vy,vz of the particles and magnitude
velData{c} = [dataRead{4}(:,1) dataRead{5}(:,1) dataRead{6}(:,1) sqrt(dataRead{4}(:,1).^2 + dataRead{5}(:,1).^2 + dataRead{6}(:,1).^2)];
fclose(fid);
c=c+1;
end
end
Tessa Kol
Tessa Kol on 9 Sep 2020
Yes. Thank you so much for all the help!

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!