For loop slows down on row 25 of 3D matrix and does not advance and original speed. What may be the issue?

Hi everyone,

I am using the following code to calculate a given parameter. I have included a progress bar for testing purposes and I noted that it always sticks at 19%. Upon further snooping around, I realize the for loop does not stop but greatly slows down after 19% even though the progress bar says only 2 min 33 secs left.

I am not running out of memory (checking with top command in the terminal) and there seems to be nothing odd about the data/anything different from surrounding data.

testcase1 = single(zeros(length(lon),length(lat),length(time)));
testcase2 = single(zeros(length(lon),length(lat),length(time)));
progressbar

for i = 1:length(lon);
    for j = 1:length(lat);
        for t = 1:length(time);

              if C(i,j,t) >= -500 && C(i,j,t) <= -12
                  testcase1(i,j,t) = A(i,j,t)*(0.35*((-(B(i,j,t)/(K*C(i,j,t))))^(2/3)) + (2 -(10/B(i,j,t))))^(1/2);       
              else              
                  testcase2(i,j,t) = 2*A(i,j,t)*((1 -(10/B(i,j,t)))^(1/2));

              end
          end
      end
      progressbar(i/121) % Update progress bar
  end

Any ideas why this would stick/slow down here?

Answer 1

[Guillaume]

If that's the above code that you're running, I don't see any reason for it to hang.

However, there are many ways to speed it up. For a start, passing the output class (single) to zeros instead of generating a double matrix and then converting it to single would be faster:

testcase1 = zeros(numel(lon), numel(lat), numel(time), 'single');
testcase2 = zeros(numel(lon), numel(lat), numel(time), 'single');

Secondly, the loop is completely unnecessary (and is a performance killer). Use vectorised operations:

tf = C >= -500 & C <= -12;
testcase1(tf) = A(tf).*(0.35*((-(B(tf)./(K*C(tf)))).^(2/3)) + (2 -(10./B(tf)))).^(1/2);
testcase2(~tf) = 2*A(~tf).*((1 -(10./B(~tf))).^(1/2));

Just three lines!

[mashtine]

Hi Guillaume,

Thanks for the help! Unfortunately though, your indexing version is a lot more memory intensive and actually is a lot slower and freezes my computer (lower RAM than desired).

[mashtine]

I have also edited my question above. Seems that the for loop isn't hanging at the index I mentioned before but just greatly slows down after row 25! No idea why ... My CPU% does reach to 100 however but I have seen this before without getting this problem. Perhaps it is this?

[Guillaume]

I don't understand how you can say the vectorised version is a lot slower when you're saying that your for loop does not even finishes. Whole matrix operations in matlab are always several order of magnitude faster than the same calculations element by element.

As for the memory, the only extra memory required compared to your code is for the logical array and possibly some temporaries. You can't do away with the logical array, but possibly this would require less memory:

%no need to predeclare. Assuming that at lease one of A or B is of type single
testcase1 = A .* sqrt(0.35*(-B./(K*C)).^(2/3) + 2 - 10./B); %what's with all the unnecessary brackets?
testcase2 = 2*A .* sqrt(1-10./B);
testcase1(C < -500 | C > -12) = 0;
testcase2(C >= 500 & C <= 12) = 0;

[mashtine]

The vectorized version quickly increased the memory being used and after killing the script after a controlled time, a lot less was written to the output variables. Not sure why. I did not know that the for loop slowed down versus stopping at the time of my comment so forgive the confusion there! Nonetheless, your last suggestion (thanks a million) does the trick. My memory surges but if produces an output in a fraction of the time!

The superfluous brackets were just an editing issues sorry!

Thanks again.

[Guillaume]

The vectorised version will either produces the whole output or nothing at all if it runs out of memory. There's no in between.

[mashtine]

Ah ok, makes sense.