textscan loop works on windows but not on macOS - why?

Hi all,

I have a list of files that i need to open, schematically the script looks like:

for i=1:length(files)
    x=fopen(files{i});
    data=textscan(x, -all-the-formatspecs-);
    fclose(x)
end

The script works perfectly fine on windows and goes through all the files in the list. When the script is run on macOS, though, an 'invalid file identifier' error message pops up after it's gone through the loop couple of times. Also, there is no consistency on -when- it crashes. sometimes after it's gone through 10 files, sometimes - through 2... If i try to open the file it crashed on, individually using same commands - it works... What can be the problem?

[Jos (10584)]

Do the filenames contain special characters or perhaps path separators (different on Mac OS en windows!)?

[Ekaterina Avershina]

the filename is specified as './folder_name/filename.txt' - should it be specified differently on mac?

[Guillaume]

Two points to make the question clearer:

• The code does not "crash". It errors with invalid file indentifier.
• The error may be reported on the textscan line but has nothing to do with textscan. The error is because fopen failed to open the file.

The most common reason why fopen fails to open a file is because the file path is incorrect.

So, how is the cell array files generated? Are you sure that files do exist?

[Jos (10584)]

Looks fine. What is the error message you get when it crashes?

[Ekaterina Avershina]

Jos, thanks! As Guillaume pointed out, it sends an error of 'invalid file identifier'

Answer 1

[Guillaume]

To find out why fopen failed to open a file, change your script to:

for i=1:length(files)
   [fid, errmsg]=fopen(files{i});
   assert(fid > 0, 'Failed to open "%s" because %s', files{i}, errmsg);
   data = textscan(fid, -all-the-formatspecs-);
   fclose(fid)
end

This should give you a better description of the problem.

Answer 2

[Ekaterina Avershina]

Guillaume, thank you for the corrections and the suggestion. (a) yes, i do know that the file exist. When i open the file individually, it works (x gets assigned to 3 or larger). and it goes through to the data; it's only when i use the loop, that the problem occurs. (b) i will try your approach and send a feedback. (c) the filename is given as follows:

files=cellstr(ls(['./' dirname]));
for i=1:length(files)
    if contains(files{i},'.txt') 
        x=fopen(['./' dirname '/' strrep(files{i},' ','')]);
        data=textscan(x, -all-the-formatspecs-);
        fclose(x)
    end
end

[Guillaume]

The body of the loop is altogether different from what you posted in the question. In particular, any path that contains spaces, you try to open the same path without spaces. I don't know much about macOS but I would assume the two are not equivalent.

Any reasons why you're not using matlab dir?

files = dir(fullfile('.', dirname, '*.txt'));
for file = files'
   [fid, errmgs] = fopen(fullfile('.', dirname, file.name));
   assert(fid > 0, 'Failed to open "%s" because %s', files{i}, errmsg);
   data = textscan(fid, -all-the-formatspecs-);
   fclose(fid);
end

[I-Lin Wang]

What's dirname?

[Walter Roberson]

@I-Lin Wang

dirname is a character vector that is being provided by the user; @Ekaterina Avershina coded it at https://www.mathworks.com/matlabcentral/answers/370404-textscan-loop-works-on-windows-but-not-on-macos-why#answer_294131

Answer 3

[Ekaterina Avershina]

Thank you for the hint, Guillaume! Without it i would have never guessed what is causing the problem! So fopen fails due to 'too long file name', and that happens because cellstr(ls(dir)) command saves filenames as (num_files*1) cellmatrix on Windows and as 1*1 cell with a 1-row string of all the concatenated filenames on mac!. I don't have a direct access to the mac (it's in another city, and the user sends me print screens of the errors), so each time i assumed it was going through number of files and the last one - is the one it failed on; and it never even slightly occurred to me that the whole list is the 'name' of the file... Thanks a lot!!!

[Stephen23]

Using ls and cellstr to get filenames

files=cellstr(ls(['./' dirname]));

is quite unusual. Standard MATLAB practice is to use dir, and loop over the structure that it returns: this gives the same behavior on all OS's. See the MATLAB documentation:

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

or FAQs:

http://matlab.wikia.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F

https://www.mathworks.com/matlabcentral/answers/57446-faq-how-can-i-process-a-sequence-of-files

Answer 4

[Ekaterina Avershina]

Guillaume and Stephen, the only reason i used 'ls' and not 'dir' was that this was the command i found when looking in help for listing the files... Thank you! I will change it to the 'dir' now. As for removing the spaces, yes, i did it deliberately since the 'ls' command returns the list as a char and if the name of one file is shorter than of the other, the spaces are added to it so that all rows (i.e. filenames) have the same length (that is, on windows at least).

[Ekaterina Avershina]

...and you were right! use of dir saved it and now it works on both! Thanks a lot for all the help! :)