How do i replace this code to fast up?

Variable poshidstates have 4 possible values(0,1,2,3). Everytime, poshidstates has to select 1 value out of this four.In this below code, i=100,j=500. Because of below mentioned code part , my program run more than 8 hour instead of just 1 hour(I have to call this code for 600 batches and each batch for 50 times). how can i replace this?

   val_vect=[0 1 2 3];
  for i=1:numcases;
      for j=1:numhid;
          prob=[poshidprobs0(i,j),poshidprobs1(i,j),poshidprobs2(i,j),poshidprobs3(i,j)];
          K=find(mnrnd(1,prob)==1);
          poshidstate(i,j)=val_vect(K);
      end
  end

[Geoff Hayes]

Is it always guaranteed that prob has an element that is equal to one? And that it has at most one element like this?

As for speeding it up, have you pre-allocated memory to poshidstate? Just prior to entering the for loop do

 poshidstate = zeros(numcases,numhid);

Though, to be honest, I don't think that this pre-allocation will reduce the running time by all that much.

[subha]

yes. I have done this already. With that only , it took 8hrs to run. Yes, when add all those four probs it gaurantees to give 1. The idea behind this code is, i have defined already probability of getting those 4 values and named as poshidprobs 0,1,2,3. I want poshidstates should select atleast one value out of those four. So, i preferred categorical distribution.

[subha]

@ salaheddin k gets only one value every time. out of the four probs value, only one can be selected everytime

[Geoff Hayes]

Subha - where in the code are you spending the eight hours? I don't have the Statistics Toolbox so can't observe how expensive mrnd is, but running the above with your dimensions of i==100,j=500, takes around a second.

So out of curiosity if you replace

 K=find(mnrnd(1,prob)==1);

with

 K=1;

how long does it take to run the code?

And as Salaheddin commented, have you verified that the result returned by

 K=find(mnrnd(1,prob)==1);

is in fact a 1x1 scalar, and not a vector of multiple indices?

[Iain]

The only two things I can think of that would make this code so slow is poshidprobs0,1 2 & 3 being anonymous functions, or you're running out of RAM.

[Geoff Hayes]

@Iain - that is a good point. I had assumed that the poshidprobs were matrices.

@Subha - are the poshidprobsX matrices or functions?

[subha]

My code runs this part of code to 600 batches(600 example or 600 times) and each batch will be called for 50 epochs(50 times). When i remove this part from my code and run with some approximation. It could finish it within less than one hour. But when i include this part, it tooks approximately 8 hrs. poshidprobs is matrix.100*500 matrix.

[subha]

@lain- what you meant by anonymous function

[subha]

@geoff- yes k is 1*1 scalar. Not a vector of mutiple indices

[Geoff Hayes]

Subha - when you say When i remove this part from my code and run with some approximation, what part of the code have you removed? Please describe the line.

And, what are poshidprobs0, poshidprobs1, poshidprobs2, and poshidprobs4? Are they matrices or function calls?

[subha]

what i guess is, prob is taken elementwise. So it tooks so much time as i have too many batches and too many epochs. If somehow it could be calculated in the form of matrix, time could be reduced. Any suggestion in this angle?

[Geoff Hayes]

Subha - I missed from your previous comment that the poshidprobsX are matrices.

But when you say When i remove this part from my code and run with some approximation, what part of the code have you removed? Please describe the line, and the approximation.

Given how you have mentioned batches and epochs, I get the feeling that we are not seeing all of your code, only a piece of it...

[subha]

@geoff- AS you asked,i have replaced k=1 and run it. it takes approximately 58 min. poshidprobs 0,1,2,3 are matrices with size(100*500)

[subha]

@Geoff. I removed whole lines whatever i posted here. Instead that i used, poshidstate = poshidprobs1+ poshidprobs2+poshidprobs3 . During that time, it took around 40 min only. It was working fine. But it is an approximation.I want to use the one,posted here. numbatches=600.

yes this is part of my program. As my code is around 100 lines long, i couldnot post everything over here. But in short, my program looks likes this.

for epoch=1:50;
   for batch= 1:numbatches;
       data = batchdata(:,:,batch)
          % calculations related to poshidprobs 0, 1, 2, 3 goes  here
           % then the posted part comes here
   end
end

[Iain]

Ok, now I see why it takes so long. You're repeating that calculation 30,000 times.

But seeing what you're doing, this might be faster:

 temp = rand(100, 500);
 poshidstate = (temp > poshidprobs0) + (temp > (poshidprobs0 + poshidprobs1)) + (temp > (poshidprobs0 + poshidprobs1 + poshidprobs2));

[subha]

Hi lain, Thanks a lot..I am so happy. You saved lots and lots of time .Problem resolved. I am searching for this for long. Now its just take 40 minutes. Thank u so much.No words to express.So happy. Please post this as answer. So many will be looking for this. Thanks mathworks

Answer 1

[Iain]

Ok, now I see why it takes so long. You're repeating that calculation 30,000 times.

But seeing what you're doing, this might be faster:

 temp = rand(100, 500);
 poshidstate = (temp > poshidprobs0) + (temp > (poshidprobs0 + poshidprobs1)) + (temp > (poshidprobs0 + poshidprobs1 + poshidprobs2));