fread() optimization binary file

1 view (last 30 days)
Marc
Marc on 26 Jan 2013
I am reading a 40 MB file with three different ways. But the first one is way faster than the 2 others. Do you guys have an idea why ? I would rather implement condition in loops or whiles to separate data than load everything with the first quick method and separate them then - memory saving -
LL=10000000;
fseek(fid,startbytes, 'bof');
%%Read all at once %%%%%%%%%%%%%%%%%%%%%%
tic
AA(:,1)=int32(fread(fid,LL,'int32'));
toc
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
fseek(fid,startbytes,'bof');
%%Read all using WHILE loop %%%%%%%%%%%%%
tic
i=0;
AA2=int32(zeros(LL,1));
while i<LL
i=i+1;
AA2(i,1)=fread(fid,1,'int32');
end
toc
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
fseek(fid,startbytes,'bof');
%%Read all using FOR loop %%%%%%%%%%%%%%%
tic
AA3=int32(zeros(LL,1));
for i=1:LL
AA3(i,1)=fread(fid,1,'int32');
end
toc
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Elapsed time is 0.312916 seconds.
Elapsed time is 138.811520 seconds.
Elapsed time is 116.799286 seconds.

Answers (1)

Walter Roberson
Walter Roberson on 26 Jan 2013
Consider reading data in chunks (e.g., 4 Kb): that should give you better speed than reading single int32 at a time, but would also reduce the amount of memory needed.

Categories

Find more on Programming in Help Center and File Exchange

Tags

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!