GUI selecting multiple files with same amount of digits in filename

Hello everyone!

A GUI that I am trying to develop is analysing data from a device. The files with the data that need to be analysed differ in the number of digits to save the files or prefix. For example: ID103215sec.csv or ID0915sec.csv or PreID0392sec.csv. The sec.csv is always fixed. Now I already figured out how to let the user input the prefix 'ID' or 'PreID' and let MATLAB search for files using this prefix and ending in sec.csv by using this code:

function prefix_Callback(hObject, eventdata, handles)
% hObject    handle to prefix (see GCBO)
% eventdata  reserved - to be defined in a future version of MATLAB
% handles    structure with handles and user data (see GUIDATA)

% Hints: get(hObject,'String') returns contents of prefix as text
%        str2double(get(hObject,'String')) returns contents of prefix as a double
Prefix = get(handles.prefix, 'String');
Pattern = [Prefix '*'];
handles.Pattern = Pattern;
if isempty(Prefix) == 1;
    Prefixval = 0;
else
    Prefixval = 1;
end
switch Prefixval
    case 1
    set(handles.findspecfiles,'Enable','On');
    case 0
    set(handles.findspecfiles,'Enable','Off');
end
guidata (hObject,handles);

% --- Executes on button press in findspecfiles.
function findspecfiles_Callback(hObject, eventdata, handles)
% hObject    handle to findspecfiles (see GCBO)
% eventdata  reserved - to be defined in a future version of MATLAB
% handles    structure with handles and user data (see GUIDATA)
foldername = handles.foldername;
Pattern = handles.Pattern;

filePattern = fullfile(foldername, Pattern);
theFiles = dir(filePattern);
Filenames = struct2cell(theFiles);
Filenames = Filenames(1,1:end)';
Filestringcheck = strfind(Filenames,'sec.csv');
Filestringlogic = cellfun(@isempty,Filestringcheck);
if isempty(Filestringlogic) == 1;
    ErrorFileName = msgbox('No files found with input criteria', 'Error','error');
end
Filenames = Filenames(Filestringlogic==0);
set(handles.text7, 'String', Filenames);

Now, if I want the user to use a edittext or popupmenu for the number of digits between the prefix and sec.csv, how can MATLAB find the files with those specific amount of numbers? MATLAB should then be able analyse all the files in the folder with those specific amount of digits.

[Stephen23]

Note this complex code:

if isempty(Prefix) == 1;
    Prefixval = 0;
else
    Prefixval = 1;
end
switch Prefixval
    case 1
    set(handles.findspecfiles,'Enable','On');
    case 0
    set(handles.findspecfiles,'Enable','Off');
end

could be simplified to just half the code:

if isempty(Prefix)
    set(handles.findspecfiles,'Enable','Off');
else
    set(handles.findspecfiles,'Enable','On');
end

Why make it more complex than it needs to be? Also note that testing isempty(...)==1 serves no purpose whatsoever: why test a logical value against 1 just to produce another exactly identical logical value? Some beginners like to do this, for reasons that have never been made clear to me.

I would also suggest that your Pattern should include the file extension: [Prefix,'*.csv'], just to make it a tiny bit more robust. Perhaps even [Prefix,'*sec.csv'], then dir already finds only the filenames that match that format, and so you do not need to use that code to filter a larger list of filenames.

Answer 1

[Stephen23]

You could try something like this:

Prefix = 'ID';
Pattern = [Prefix,'*sec.csv']; % better to match both prefix and fixed parts of the filenames.
foldername = '.';
filePattern = fullfile(foldername, Pattern);
S = dir(filePattern);
S = S(~[S.isdir]); % remove folders (probably none, but more robust).
C = {S.name};      % get filenames.
D = cellfun(@nnz,isstrprop(C,'digit')); % count number of digits in names.
N = 4;    % requested number of digits.
X = D==N; % index of filenames with that number of digits.
C{X}      % show those filenames!

I created three files with the names that you showed in your question, and when I ran the code it showed this:

ans = ID0915sec.csv

Note that this code counts all digits in the filename! If your filename format changes to have multiple groups of digits them you will need to consider an alternative, e.g. a regular expression.

[Debbie Oomen]

This works great. If I want to indicate that the cell array is 0x0, how can MATLAB provide this as an error? I have tried:

if nnz = 1
         set(handles.text1, 'String', C{X})
else 
         Errorfilename = msgbox('No files found with input criteria', 'Error', 'error')
end

But it says that it does not have enough input arguments

[dpb]

>> which nnz
built-in (C:\ML_R2017\toolbox\matlab\sparfun\@double\nnz)  % double method
>> help nnz
nnz    Number of nonzero matrix elements.
  nz = nnz(S) is the number of nonzero elements in S.

[Stephen23]

@Debbie Oomen. I suspect that you want the function isempty, rather than the function nnz.

[Debbie Oomen]

I am trying that now. This is the code I have tried:

foldername = handles.foldername; %see ChooseFolder callback
Pattern = handles.Pattern; %see prefix callback

filepattern = fullfile(foldername, Pattern); %all files in folder with ID*sec.csv* pattern
filesfolder = dir(filepattern); %files with the pattern
filenames = struct2cell(filesfolder); 
filenames = filenames(1,1:end)'; %filenames
Filestringcheck = strfind(filenames,'csv'); %# of strings in filename without prefix
Filestringlogic = cellfun(@isempty,Filestringcheck); 
filesfolder = filesfolder(~[filesfolder.isdir]); %remove folders 
NoD= cellfun(@isempty, filenames,'digit'); %count number of digits in filename
Dig = str2double(get(handles.digitsfile, 'String')); %gets desired number of digits from user
Matches = NoD == Dig;
Filenames{Matches}

if isempty(Filestringlogic) == 1;
    ErrorFileName = msgbox('No files found with input criteria', 'Error','error'); 
end
if is empty ('digit')

This gives me an error. I have no idea where to put the @isempty

[Stephen23]

Matches is a logical vector, where its true values correspond to matched filenames that you want... if you want to check if there were no matches, then you can do this trivially like this:

if ~any(Matches)

PS: Note that for some reason you deleted the isstrprop function... check my answer again, because this will not work and it is not what I showed you in my answer:

NoD= cellfun(@isempty, filenames,'digit');

PPS: This code:

Filenames{Matches}

simply displays the contents of the cell array Filenames for the cells select by Matches, it does nothing else at all. But if you want to put those selected names in another variable, then you will need to allocate them to that variable:

   Z = Filenames(Matches);

PPPS: I really would recommend that you put the file extension pattern into the dir pattern. I.e. instead of this (and associated code):

Filestringcheck = strfind(filenames,'csv');

use a suitable dir pattern. There is nothing that strfind can do that dir cannot do, so there is no point in making your code more complex by replicating functionality in two places.

[Debbie Oomen]

Thank you so much for your help and the tips!

For the matches of the filenames, I want the GUI to read all the csv files that match the number of digits. With an individual file, readtable(filename) works because the csv file contains strings, numbers, dates, times, etc. However, when I try a loop like this:

foldername = handles.foldername;
if get(handles.specfiles, 'value') == 1 %if this checkbox is selected
FNMS = handles.FNMS; %FNMS, string of the matches with user input
for k = 1:length(FNMS)
basefilename = FNMS(k).name;
fullfilename = fullfile(foldername, basefilename);
cspecfiles = readtable(fullfilename)
end
end

However, this gives me an array and says that readtable needs string array. But FNMS is a string. What am I doing wrong?

[Stephen23]

Is the input to readtable really a string or char vector? Clearly not. What is it really?

You need to learn how to debug your own code, because debugging using random strangers on the internet is very inefficient. When beginners get a bug, their response is often something like "but that is impossible, because I know that XXX is a string!". Well, the fact that you are getting an error shows that it is not impossible. It is your task to investigate what is really happening in the code (not in what you imagine/want/believe/hope/wish/assure/... is happening)... sitting there saying "that is impossible" will not help you to solve anything. Start from readtable's input: check it, what class it is, what size it is, what value it has:

whos fullfilename
class(fullfilename)

And from there you can figure out why and what to do about it.

Your code is complex enough that you need to learn how to debug, and actually looking at what your code is doing is an important step to doing that. I would also recommend that you learn how to set breakpoints and use the debugging tools:

https://www.mathworks.com/help/matlab/matlab_prog/debugging-process-and-features.html

[Debbie Oomen]

>> whos FNM
  Name      Size            Bytes  Class     Attributes

    FNM       1x2               212  string

It says it is a string array. Is it possible to readtable with a string array? I have tried looking up ways to convert it to string vectors but come up empty. Is there another function that will read the contents of the files?

[Stephen23]

"Is it possible to readtable with a string array?"

According to the documentation the first input of readtable can be a string, but I suspect that it must a scalar string. You can test this yourself in about ten seconds.

But in any case you are still looking at FNM, whereas you need to look at fullfilename (as I suggested in my previous comment) and work backwards from there: what class is it, what size does it have, what value?

[Debbie Oomen]

Yes it must be a scalar but I do not know how to get a scalar string from an array or character. This is the code for finding the matches and thus the files that need to be loaded into MATLAB:

foldername = handles.foldername; %see ChooseFolder callback
Pattern = handles.Pattern; %see prefix callback

filepattern = fullfile(foldername, Pattern); %all files in folder with ID*sec.csv* pattern
filesfolder = dir(filepattern); %files with the pattern
handles.filesfolder = filesfolder;
filenames = struct2cell(filesfolder); 
filenames = filenames(1,1:end)'; %filenames
Filestringcheck = strfind(filenames,'csv'); %# of strings in filename without prefix
Filestringlogic = cellfun(@isempty,Filestringcheck); 
filesfolder = filesfolder(~[filesfolder.isdir]); %remove folders 
NoD= cellfun(@nnz, isstrprop(filenames,'digit')); %count number of digits in filename
Dig = str2double(get(handles.digitsfile, 'String')); %gets desired number of digits from user
Matches = NoD == Dig;
FNM = string(filenames(Matches))%string array
handles.FNM = FNM;

Then, to load the files into MATLAB, I wrote this code:

if get(handles.specfiles, 'value') == 1 
    FNM = handles.FNM;
    for k = 1:length(FNM)
        fullfilename = fullfile(foldername, FNM);
        cspecfiles = readtable(fullfilenames)
    end

The class for fullfilename was a cell array but with the code below it is now a character array. MATLAB still gives me an error that it is not a row vector of characters or string scalar. I have been looking on mathwork but I can't find anything that helps me with this issue

[Stephen23]

"Yes it must be a scalar ..."

It must be a scalar string, or a row char vector. Either is permitted.

As I wrote earlier, you need to look as fullfilename, its size, value, etc. Perhaps it contains multiple filenames (e.g. in a non-scalar string), which you then provide to readtable. But because readtable only reads data from one file at a time, as its help clearly states: "... creates a table by reading column oriented data from a file," then asking it to read multiple files throws that error.

Please show me the output of this command:

whos fullfilename

Please also show the value or contents of fullfilename, e.g. the contents of its cells if it is a cell array, etc. It would help if you uploaded it in a .mat file.

[Debbie Oomen]

I think you mean this?

fullfilename =

2×1 string array

    "/Users/debbieoomen/Desktop/Data Files/ID115sec.csv"
    "/Users/debbieoomen/Desktop/Data Files/ID576sec.csv"

Name              Size            Bytes  Class     Attributes

fullfilename      2x1               372  string

[Debbie Oomen]

Doesn't a loop ensure that all files are read?

[Stephen23]

If you have multiple filenames then you will have to use a loop.

[dpb]

The problem we've got is we now don't know at what point in the code fullfilename contains that string array -- is that before the loop? It would almost seem so because the loop won't run w/o error but if is before then why are you appending the content of FNM onto an what appears to already be a fully-qualified file name???

Looks to me like you need something more like

for k = 1:length(fullfilename)        % loop over the string array
  data = readtable(fullfilename(k));  % read the file
  ...do whatever here with that table's data...
  ...
  end

NB: When you read the file; if you immediately read the next, you'll just write over the previous data set; you have to either read and process each file in turn or have a plan for how to deal with multiple sets of data at once.

[Debbie Oomen]

foldername = handles.foldername; %see ChooseFolder callback
Pattern = handles.Pattern; %see prefix callback
filepattern = fullfile(foldername, Pattern); %all files in folder with ID*sec.csv* pattern
filesfolder = dir(filepattern); %files with the pattern
handles.filesfolder = filesfolder;
filenames = struct2cell(filesfolder); 
filenames = filenames(1,1:end)'; %filenames
Filestringcheck = strfind(filenames,'csv'); %# of strings in filename without prefix
Filestringlogic = cellfun(@isempty,Filestringcheck); 
filesfolder = filesfolder(~[filesfolder.isdir]); %remove folders 
NoD= cellfun(@nnz, isstrprop(filenames,'digit')); %count number of digits in filename
Dig = str2double(get(handles.digitsfile, 'String')); %gets desired number of digits from user
Matches = NoD == Dig;
FNM = string(filenames(Matches))%string array
handles.FNM = FNM;

In this code, FNM is important. These are the filenames that match the files that the user wants to analyze. This code is only to look for these files, the code with the loop is meant to analyze the data in all the files with the pattern that the user put in. I don't know if I can just use a loop with: for k=1:length(FNM) or k=1:length(fullfilename) or maybe something entirely different to do this. And how can I ensure that this loop will analyze all files instead of writing over the previous one?

Any help is greatly appreciated..

[Stephen23]

"I don't know if I can just use a loop..."

It is not required to know everything in advance: read the documentation and experiment with small examples, then you can figure out what you need to do. Looping over the elements of fullfilename makes sense to me. Why not try that?

"how can I ensure that this loop will analyze all files instead of writing over the previous one"

In exactly the same way that you will access the elements of fullfilename: by using indexing. Have a look at the examples in the documentation:

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

The first example loads the data into a cell array named mydata, and uses indexing so that each loaded array is store in its own cell of mydata. You need to do that.

[dpb]

filepattern = fullfile(foldername, Pattern); %all files in folder with ID*sec.csv* pattern
filesfolder = dir(filepattern); %files with the pattern
handles.filesfolder = filesfolder;
filenames = struct2cell(filesfolder); 
filenames = filenames(1,1:end)'; %filenames
Filestringcheck = strfind(filenames,'csv'); %# of strings in filename without prefix
Filestringlogic = cellfun(@isempty,Filestringcheck); 
filesfolder = filesfolder(~[filesfolder.isdir]); %remove folders 
NoD= cellfun(@nnz, isstrprop(filenames,'digit')); %count number of digits in filename
Dig = str2double(get(handles.digitsfile, 'String')); %gets desired number of digits from user
Matches = NoD == Dig;
FNM = string(filenames(Matches))%string array

You're making this a lot more complicated that it is it seems...

filepattern = fullfile(foldername, Pattern); %all files in folder with ID*sec.csv* pattern

OK, presuming the above comment is true, then

 filesfolder = dir(filepattern); %files with the pattern

you now have all files matching the pattern which includes the .csv extension because you passed it in the pattern (unless your comment is wrong)

handles.filesfolder = filesfolder;
filenames = struct2cell(filesfolder); 
filenames = filenames(1,1:end)'; %filenames

I see no purpose whatsoever for the above three lines; you already have the variable filesfolder that contains the names; if you want to use the name filenames going forward then just write

filenames=dir(filepattern);

from the git-go and go on from there.

If you want to do a subselection from the overall and the result of that is is the logic variable Matches, then I'd write

filenames=filenames(Matches);

also and select in place instead of making up other variables. Then, as noted, just loop over filenames array as outlined above.

Also, if you passed the pattern as something like '*ID*.csv' to dir, then you're going to only get files returned that match that pattern including the extension and therefore won't be getting any directory entries with high probability. Doesn't hurt to be extra cautious, but it would be highly unlikely for a user to create a directory with a .csv extension--not impossible, but not likely.

[Debbie Oomen]

I changed the code the way you've mentioned. The thing is, the filenames are identified by the user by entering the prefix for the wanted files and the number of digits behind the prefix. Without using those three lines of code I get an error. I have attached the code for everything total. So far, the user can input files in three different ways: using an individual file, selecting files based on the identity or using all files in a folder. The loop would be for the specific files callback that allows the user to enter the prefix and digits of the filenames.

[dpb]

Well, as Stephen says, learn to use the debugger and step through and see what breaks and then research why.

Since you don't even tell us what the error is, and the crystal ball is in the shop (yet again :( ); not going to try to guess but I'll put odds on that it has to do with the difference between how the filenames array is being stored; you're recasting from its form as an a dir() structure array to a cell array...I've really no idea what that would look like in the end but it can't be as simple as simply filenames(i).name for the ith filename in the dir() array. ...

ADDENDUM/CORRECTUM I kept being puzzled by what was going on here so had a few minutes and played around some...there were too many similar variable names; my eyes got crossed earlier so there's one less cast than had thought earlier--you didn't convert the handles struct, only the DIR() struct. I then finally realized why you did what you did; but it's not simplest way to go at it: to return the names alone as cellstr array, just write

filenames = {filesfolder.name}.';

which will leave a column vector of the name field from the DIR() call for those files matching the pattern.

Earlier comments about searches for the same text string as was incorporated in the DIR() pattern search seems still apropos, however.