How to execute the same operation on various .csv files?

I have a list (62 items) of .csv files, all within a single folder. My goal would be to make the same operation on all of these files; I managed to write the code on a single .csv file, which is as follows (actually it is a trapz operation but written extensively):

%my code on a single .csv file
T = readtable('M_20210325-1519_90_l90_.csv','HeaderLines', 11); %read single .csf file and skip 11 lines
[numRows,numCols] = size(T);
cont=0;
tB= T(:,3);                 %consider 3rd column
B=table2array(tB)           %consider it as an array
bvB=B(1,1);                 %consider the first value
i=1;
intB=zeros(numRows,1);      % create an array full of zeros
newB=B(:,1)-bvB;            %subtract the first value
z=numRows-1;
for cont=1:z
    if (newB(i+1,1)+newB(i,1))>=0
       intB(i,1)= (newB(i+1,1)+newB(i,1))/2;
    else
       intB(i,1)=0;
    end 
     dbstop if error;
     i=i+1;
end
sommaB=sum(intB);       %useful result

I had been looking for some scripts that allow to consider and read all the data from the .csv files, such as

files = dir('*.csv')
N = length(files) ;
% loop for each file 
fileNames = {files.name};
for k = 1:N
    data{k} = csvread(fileNames{k});
    %%do whatever you want
end

(which gives the Trouble reading 'Numeric' field from file error, because I guess it does not skip the header lines), or

fds = fileDatastore('*.csv', 'ReadFcn', @importdata)
fullFileNames = fds.Files
numFiles = length(fullFileNames)
% Loop over all files reading them in and plotting them.
for k = 1 : numFiles
    fprintf('Now reading file %s\n', fullFileNames{k});
    % do the operation --------> my single operation
end

Both are useful because I get all the .csv files, already sorted in a proper way, but I'm missing the link between considering them one at a time and execute the operation. Probably it's relatively easy, but I'm a beginner.

I attach two .csv files as an example.

Thank you

Answer 1

[David Fletcher]

You would need to incorporate your code into one of the examples, also you would need to store the intended output as the loop progresses thorugh your examples. As a guide: (you will of course have to change the path to where you have the .csv files stored on your system)

path='H:\MATLAB\R2021a\bin\TestDir\' %directory of the csv files on your system
files = dir(strcat(path,'*.csv'))
N = length(files) ;
% loop for each file 
fileNames = {files.name};


for k = 1:N
    %my code on a single .csv file
    T = readtable(strcat(path,fileNames{k}),'HeaderLines', 11); %read single .csf file and skip 11 lines
    [numRows,numCols] = size(T);
    cont=0;
    tB= T(:,3);                 %consider 3rd column
    B=table2array(tB)           %consider it as an array
    bvB=B(1,1);                 %consider the first value
    i=1;
    intB=zeros(numRows,1);      % create an array full of zeros
    newB=B(:,1)-bvB;            %subtract the first value
    z=numRows-1;
    for cont=1:z
        if (newB(i+1,1)+newB(i,1))>=0
           intB(i,1)= (newB(i+1,1)+newB(i,1))/2;
        else
           intB(i,1)=0;
        end 
         dbstop if error;
         i=i+1;
    end
     %useful result
    sommaB(k)=sum(intB); %Save output in a vector for each loop iteration

end

[Simone Speltoni]

Hi David, thanks for the useful tip. I did not know the use of strcat, to concatenate strings... the aim here would be to concatenate the names of my .csv files..?

Yes, of course I changed the path to the directory that contains the .csv files, anyway N, the cell fileNames, the struct files all result empty or equal to zero, as if no files would have been found... Am I missing something obvious?

[David Fletcher]

Yes, the strcat just adds the file name to the path to make a valid file location - I had to do this on my system because I put your files in an separate test directory (you might get away without it if everything on your system is in the same diectory). If the files struct is empty it indicates that there is a problem in locating the files. It works on my system, but obviously I have no idea how your system is setup or where you are storing your files. If you've got everything in the same directory you could get rid of the path entirely and revert to your original usage:

files = dir('*.csv')
N = length(files) ;
% loop for each file 
fileNames = {files.name};

for k = 1:N
    T = readtable(fileNames{k}),'HeaderLines', 11); 
% ... and so on

[Simone Speltoni]

Thanks for the help David, I avoided the part regarding the path and it works, I really appreciate! Thanks!