How to skip (iteration) empty folders and empty text files

Hi,

I have main folder, and many sub-folders. Inside each sub-folders, there are many sub-folders(sub-sub-folders). The sub-sub-folders contain many files(mainly text). I want to only read text files. But unfortunately some sub-folders are empty or some sub-sub-folders contain other files (non-text files). So, I wish the following:

1. Skip if the sub-folder is empty
2. Skip if the text file is empty
My meaning: skip means skip the iteration. I am reading the text files as following:
by each sub-folder and by each sub-sub-folders (the text files)    
Please some one help me how to do this.

Many many thanks in advance,

[KSSV]

doc isempty

Answer 1

[Mostafa]

%Get list of all files in current directory and beneath
[~,list] = system('dir /S *.txt');
result = textscan(list, '%s', 'delimiter', '\n');
fileList = result{1};
%Remove empty lines
fileList(cellfun(@isempty,fileList)) = []; 
%Get full paths of .txt files with size > 0
filesDir = [];
for i = 3:length(fileList)-3
    strData = strsplit(fileList{i});
    %Get directory name
    if (strcmp(strData{1}, 'Directory'))
        currDir = strData{3};
    elseif (strcmp(strData{2}, 'File(s)'))
        %Do nothing
    %Save names of files with size > 0
    elseif ~(strcmp(strData{3}, '0'))
        filesDir = [filesDir; currDir strData{4}];
    end
end

[Jan]

Calling "system('dir ..." takes much more time than using Matlab's dir() function directly.

[Mekala balaji]

Hi,

Thanks a lot for your above suggestion. May I ask something like: Can it be possible to just add "while loop" that if (1) execute only when the folder is not empty, (2) execute the loop only when the text file is not empty.

[Mostafa]

Jan: Yes, it is. However calling Matlab dir function outputs data in the command window - which can be accessed using a diary file and formatted, but it's an overhead in my opinion. But most importantly it doesn't get the size of the files, so I must get all the files, open them individually, check if they're empty or not to determine if they have data. In my approach I didn't open any file and parsed the data from system dir itself, which I believe is better.

In any case, Matlab Profiler can be used to determine which approach is better. I was only concerned with figuring out a feasible solution (which took 0.065662 seconds using tic-toc, so quasi sufficient). Cheers.

[EDIT: I didn't know there was a recursion function for dir till I saw your solution, so I've only considered dir * * /*.txt in my answer.]

[Mostafa]

Mekala: I'm sorry I can't understand your question? the result of the written code is the variable filesDir which contains the full paths of all files in the directory which have size > 0, which in this case are:

filesDir =
C:\TestFolder\MainFolder\SubFolder1\Folder1RainFallReport1.txt
C:\TestFolder\MainFolder\SubFolder1\Folder1RainFallReport2.txt
C:\TestFolder\MainFolder\SubFolder1\Folder2RainFallReport3.txt
C:\TestFolder\MainFolder\SubFolder1\Folder2RainFallReport4.txt
C:\TestFolder\MainFolder\SubFolder1\Folder2RainFallReport5.txt
C:\TestFolder\MainFolder\SubFolder1\Folder3RainFallReport6.txt
C:\TestFolder\MainFolder\SubFolder3\Folder1RainFallReport1.txt
C:\TestFolder\MainFolder\SubFolder3\Folder1RainFallReport2.txt
C:\TestFolder\MainFolder\SubFolder3\Folder2RainFallReport3.txt
C:\TestFolder\MainFolder\SubFolder3\Folder2RainFallReport4.txt
C:\TestFolder\MainFolder\SubFolder3\Folder2RainFallReport5.txt
C:\TestFolder\MainFolder\SubFolder3\Folder3RainFallReport6.txt

System dir already skips the empty subfolders, and I created a small check to skip empty files in order to create this list. You can do whatever you want using filesDir variable, parse the data, print a list, whatever you want.

Answer 2

[Jan]

USe one of the many recursive dir commands from the FileExchange: http://de.mathworks.com/matlabcentral/fileexchange/?utf8=%E2%9C%93&term=dir+recursive, e.g. http://de.mathworks.com/matlabcentral/fileexchange/15505-recursive-dir:

FileList = dirrec(MainFolder, '*.txt');
for k = 1:numel(FileList)
  FileInfo = dir(FileList{k});
  if FileInfo.bytes > 0
    ... import the data ...
  end
end

Some of the recursive dir versions reply a struc like Matlab's dir(), then:

DirList = <the_dir_replacement>(MainFolder, '*.txt');
DirList = DirList([DirList.bytes] ~= 0);
for k = 1:numel(FileList)
  FileName = DirList(k).name;
  ... import the data...
end