Fgetl, Fscanf, Fgets, Fprintf Execution Time

Hi, I have created a serial communication for send and receive data. However, these function allows sending data to the serial port and retrieving the data from it. I understood it need some times (few seconds) for the function to execute. However, when i loop it. The time will be increase exponentially. I believe it cause hanging to my code. Can someone have better suggestion to replace fprintf for sending and fgetl for receiving binary bits to allow faster execution time? Many thanks in advance

Here is the coding for send and receive: For Send:

%Open the file

fid=fopen('XXX.mp3','rb'); % Open a file inside MATLAB any music file.

    A=fread(fid,'uint16');      % Display the data in uint16
    B=dec2bin(A);    % Change decimal (0-65535) into 16 binary bits and store in matrix (Mx1) matrix
    F=B';      % Transpose it so it will not in matrix format 

    [M N]=size(A);              %Determine the size
    looptimes=fix(M/5000);
    last=rem(M, 5000);

send_count=1;

loop_count=0;

C=[];

 for i=1+5000*(send_count-1):5000*send_count    %To send data 5kB
     C=[C F((-15+16*(i)):16*(i))];              %16bits everytime loop and send
end
fprintf(s,C)                                    %Send to the serial port
send_count=1+send_count;

For receive:

file=[];

received_count=1;

    loop_count=0;

    while strcmp(s.PinStatus.ClearToSend,'on')
        s.RequestToSend='on';
        received=fgetl(s);                      %To receive bits from serial port
        file=[file received];       %Concatenate back

        received_count=received_count+1;
size_byte=fix(size(file)/32);                   % The size of the file
        M=size_byte(2);
        Video_bin=zeros(M,1);

for j=1:M % To arrange back the file

    b=16*(j);

    a=-15+b;
             Video_bin= [Video_bin;file(a:b)];
         end
              Video_dec=bin2dec(Video_bin);     % From binary to decimal
              fid1=fopen (Save_file, 'wb');     % Form back the original file
              count=fwrite(fid1, Video_dec, 'uint16');

[Oleg Komarov]

Post some code

Answer 1

[Oleg Komarov]

Use the profiler to identify the bottlenecks and did you preallocate (if possible) when importing?

EDIT

About preallocation and taking as an example the code you posted below:

% Example inputs
A1 = [];
M = 6000;
B = randi([0,1],1,16*M);

% Manual reshape WITHOUT preallocation
tic
for j = 1:M
    b = 16*(j);
    a = -15+b; 
    A1 =  [A1; B(a:b)];
end
toc

% Manual reshape WITH preallocation
tic
A2 = zeros(M,16);
for j = 1:M
    b = 16*(j);
    a = -15+b; 
    A2(j,:) =  B(a:b);
end
toc

% Reshape
tic
A3 = reshape(B,16,M).';
toc

A1: Elapsed time is 1.708695 seconds.
A2: Elapsed time is 0.014439 seconds.
A3: Elapsed time is 0.001066 seconds.

[MrPuzzled Marc]

Yes, I did try to test using the profiler and it stated the self time for the receive callback is very long. Preallocate what do you mean?

[Oleg Komarov]

Well, you should post at least simplified code. For preallocation guide: http://www.mathworks.com/help/techdoc/matlab_prog/f8-784135.html

[MrPuzzled Marc]

Thanks mr oleg. I shall try it in my code and see the results. Thanks alot.

[MrPuzzled Marc]

Thanks mr oleg. I shall try it in my code and see the results. Thanks alot.

[MrPuzzled Marc]

Hi, may i know what is preallocation for C=[C F(a:b)]? Previously got a ; in the middle but this do not have so the bits will be concatenate together in a large stream.

[Oleg Komarov]

Calculate the final length of your C = zeros(1, final length);

Then in the loop:

C(a:b) = ...

[MrPuzzled Marc]

Does the final length refers to size byte?

[Matt Fig]

If the code you are referring to here is the same code you posted in your other question, then this would work:

Video = zeros(16,M); % Fill each column

for ii = 1:M

Video(:,ii) = file(16*(ii-1)+1:16*ii);

end

Video = reshape(Video,1,16*M);

[MrPuzzled Marc]

I think the zeros should be zeros (1,M) because i only required the first column of the matrix. How about the preallocation for this?

C=[];

for i=1+5000*(send_count-1):5000*send_count

b=16*(i);

a=-15+b;

C=[C F(a:b)];

end

To this?

C=zeros (1,M);

for i=1+5000*(send_count-1):5000*send_count

b=16*(i):

a=-15+b;

C(a:b)=F(a:b);

?

[MrPuzzled Marc]

Is the C should be C(i,:) but this will arrange it in matrix. May i know how to arrange it in a long bits? eg: 101010010101010010.....

[Matt Fig]

Did you see that at the end I reshaped the array to be a row vector (the last line)? That takes care of your needs...

[Oleg Komarov]

C(1,size(1+5000*(send_count-1):5000*send_count,2))

[MrPuzzled Marc]

Matt: I will try it and let u know the result shortly.

Oleg: This code generate error.

I typed:

C=zeros(1,M);

for i=size(1+5000*(send_count-1):5000*send_count

b=16*(i);

a=-15+b;

C(1,size(1+5000*(send_count-1):5000*send_count,2))=F(a:b);

To replace this code:

for i=1+5000*(send_count-1):5000*send_count

C=[C F((-15+16*(i)):16*(i))];

end

However this generates an error to it.

[MrPuzzled Marc]

The generate error is possibly unbalanced (,{, or [.

[MrPuzzled Marc]

C(size(1+5000*(send_count-1):5000*send_count))=F(a:b);

This generates no error

[MrPuzzled Marc]

One last question would be :

received=[];

received=fgetl(s);

file=[file received];

Can i preallocate this file? This is because received is obtained from the data obtained from the serial port, hence it should be no size byte yet. Hence is there possible to preallocate it?

[Walter Roberson]

You can preallocate to your maximum expected size, or you can dynamically allocate in chunks (e.g., 16 Kb) so as to reduce the number of allocations you need to do. To allocate in chunks, you would determine whether the new data you have would fit within the existing buffer, and if not then extend the buffer by a number of elements and then write the new data in to it. for example, file(end+16384)=0 .

[MrPuzzled Marc]

Walter: Ok will try first and let you know as soon as possible

Oleg: Hi, you gave me the piece of code C(size(1+5000*(send_count-1):5000*send_count)) it still generates error to the loop. Any better solution? At first u give that A2(j,:) = B(a:b); This is to set the stream of bits into matrix format. I will to see the reverse process.

C=[];

for i=1+5000*(send_count-1):5000*send_count

b=16*(i);

a=-15+b;

C=[C F(a:b)];

end ?

[MrPuzzled Marc]

Hi, I m really getting sick with my pieces of code. It yields more error which i do not know how to resolve. Could you all help me on this?I had been debugging few days non stop for this piece hoping for it to execute faster for my demostration on this thrusday. Hope you all can really give me some help. Thanks for you all kindness to help me. Maybe you can gave me some hint on how to write the code correctly?My sender gave error when i try to use the C(1,size(1+5000*(send_count-1):5000*send_count,2)) while receiver error for fgetl which i did not modify anything on that. Kindly hope you all can really lead some help. I m totally weak in matlab.

%Sending code

send_count=1;

loop_count=0;

while send_count < looptimes+2;

if strcmp(s.PinStatus.ClearToSend, 'on') && send_count <looptimes+1

C=[];

for i=1+5000*(send_count-1):5000*send_count

C=[C F((-15+16*(i)):16*(i))];

Bytes=s.BytesAvailable;

set(handles.EditSentBits,'Value',Bytes);

end

fprintf(s,C)

send_count=1+send_count;

set(handles.EditStatusBar,'String',send_count)

else

pause(0.01);

end

if strcmp(s.PinStatus.ClearToSend, 'on') && send_count>looptimes

C=[];

for i=1+5000*looptimes: last+5000*looptimes

C=[C F((-15+16*(i)):16*(i))];

Bytes=s.BytesAvailable;

set(handles.EditSentBits,'Value',Bytes);

end

fprintf(s,C)

send_count=1+send_count;

set(handles.EditStatusBar,'String',send_count)

else

pause(0.01);

end

if strcmp(s.PinStatus.ClearToSend, 'off')

loop_count=loop_count+1;

set(handles.EditStatusBar,'String', loop_count)

pause(1);

end

if strcmp(s.PinStatus.ClearToSend, 'off') && loop_count>10

set(handles.EditStatusBar, 'String','Sending Failed!')

break

end

end

end

s.RequestToSend='off';

guidata(hObject,handles);

%Receive code

% --- Executes on button press in Receive.

function Receive_Callback(hObject, eventdata, handles)

% hObject handle to Receive (see GCBO)

% eventdata reserved - to be defined in a future version of MATLAB

% handles structure with handles and user data (see GUIDATA)

Save_file=get(handles.EditSaveAs,'String');

if isempty (Save_file)

msgbox('No File Selected!','Error','error');

else

s=handles.serConn;

baud_list=get(handles.EditBaudRate,'String');

baud_value=get(handles.EditBaudRate,'Value');

baud_rate=str2double(baud_list{baud_value});

s.BaudRate=baud_rate;

time_out=get(handles.EditTimeOut, 'String');

s.TimeOut=str2double(time_out);

s.RequestToSend='off';

s.RequestToSend='on';

received_count=1;

loop_count=0;

file=zeros(1,10000000);

while strcmp(s.PinStatus.ClearToSend,'on')

s.RequestToSend='on';

received=fgetl(s);

file=[file received];

received_count=received_count+1;

set(handles.EditStatusBar, 'String', 'Receiving....');

pause (0.2);

loop_count=0;

if strcmp(s.PinStatus.ClearToSend,'off')

loop_count= loop_count+1;

set(handles.EditStatusBar, 'String', loop_count)

pause(0.5);

end

if strcmp(s.PinStatus.ClearToSend,'off') && loop_count>20

set(handles.EditStatusBar,'String','Received Failed!')

break

end

end

if received_count==1

set(handles.EditStatusBar, 'String', 'Nothing To Receive....')

end

s.RequestToSend='off';

size_byte=fix(size(file)/32);

M=size_byte(2);

Video_bin=zeros(M,1);

for j=1:M

b=16*(j);

a=-15+b;

Video_bin(j,:)= file(a:b);

end

Video_bin=reshape(Video_bin,1,M*1);

Video_dec=bin2dec(Video_bin);

fid1=fopen (Save_file, 'wb');

count=fwrite(fid1, Video_dec, 'uint16');

handles.savefile=Save_file;

h=handles.activex3;

h.URL=Save_file;

end

[Walter Roberson]

If you want to increase the speed, do not use dec2bin and bin2dec: just send the data itself using fwrite() and fread() .

[MrPuzzled Marc]

That one doesnt consume alot of time. Consumed most time is the fgetl and also the for loop to divide the bits and form back.

[Oleg Komarov]

This is getting messy. Please edit your original question posting the formatted codeas you're using it now.

[MrPuzzled Marc]

Hi, I had edited my question as above. You may try to replace XXX with any file inside your MATLAB folder to run this. It gives you better situation. I had tried out the code you first wrote to me. However, this gives incorrect to the bits.

Answer 2

[Walter Roberson]

Serial port delays are usually in the fraction of seconds rather than in seconds, unless you are talking about the time to transmit a large batch of information.

I can think of at least two different meanings for talking about "looping" the program, and neither of them would result in an exponential increase in the execution time.

I have the suspicion that you should be switching to use the BytesAvailableFcn serial callback, and possibly switching terminator mode and possibly terminator character.

Does data get lost when you allow the transmission to proceed at full speed? If it does then you should probably be using hardware flow control.

[MrPuzzled Marc]

Sorry my explanations is not clear. Actually I test out send and receive binary data via serial port. I test the function by typing fprintf(s, '101010') on sender and fscanf (s) to receive the bits. Fprintf can send the data fast( click and send) however fscanf needs around 1 or 2 seconds (estimation) to receive the data. Hence I think naturally when transfer 100kB it needs to fscanf many times and cause the program to run slow.

Here is the codes for my program:

To receive the binary bits:

received=fgetl(s); %Received from serial port

file=[file received]; %Align it in a stream of bits

To form back the original bits in matrix form in order to form back original data:

for j=1:M

b=16*(j);%Arithmetic sequence to arrange the bits in 16 bits

a=-15+b; %Arithmetic sequence

Video_bin= [Video_bin;file(a:b)]; %Arrange the bits from stream of bits into 16 bits per columns.

end

I have checked using the profiler in MATLAB. It stated the receive (which include these 2) have huge amount of self time. So i suspect is these 2 which causes the problem. Do you have any better suggestion to replace the code with some functions or anything to reduce the time?

The file i m intended to send is 100kB. However the rate of success for completion transfer is very minimal and time to transmit is around 10 mins. I believe this is not a large batch of information.

I did not use the BytesAvailablFcn. Yes, I suspect the is terminator that cause problem. Because fgetl reads and terminate when only it reaches the terminator but not the last data it receives. I believe it needs to wait for every transmission and naturally add up will cause the delay in the code.

The data did not get lost. However, where can I know is the transmission is at full speed? I m using flow control to none and I control the send and receive by writing a condition to control the RTS/CTS pins.