Increasing speed of indexing

Hello community. I have a whole bunch of data (real data is being acquired and processed while running). In this example file, I just have fake numbers.

Previously, I had this code running fast enough by manually declaring dozens of variables, and sorting the data explicitly for each variable, with at most 4 indices. This is not fun, or clean, and if I need to change the math on the data, I need to change it in dozens of places. So I modified the code to do everything in a couple of loops. I also really need it this way, as in my real data I want to change what is coming in, it will only take modifying a user selectable value in this type of code to change data analysis. It also makes it easier when I save the data, as I do not need dozens of manually explicit variables declared, and such. So now everything is dynamic in that the user can specify how many data output parameters, etc. It is sweet this way! And will make it easier for other people to modify data processing later.

Only, now that I am using more indices to keep track of things, it is to slow! It has to be fast enough to process the data, otherwise the cards the data is being acquired on fill up and it stops acquisition.

Any ideas on how to speed up the code? Attached is a simplified example with fake data.

clear

%%Setup some variables
numOfPol =2;
impulsePerPixel = 8;
numOfChannels = 4;
flapJackAvg = zeros(1,512*512*numOfChannels*2);
flapJackAvg = reshape(flapJackAvg, [1,512,512,numOfChannels,2]);
flapJackAvg = single(flapJackAvg);

flapJackCount = zeros(1,512*512*4*2);
flapJackCount = reshape(flapJackCount, [1,512,512,numOfChannels,2]);
flapJackCount = single(flapJackCount);

%%Create Data

testData = 1:impulsePerPixel*512*512*2;
testData = uint16(testData);

%%Process Data
tic
for i = 1:2
    boardId = i;  
    for j = 1:2
        channelId = j;
        channelSelect = boardId + channelId;
        if channelId == 1, channelSelect = channelSelect -1; end %subtract 1 under certian conditions to set channelSelect range from 1-4

              RawData = single(reshape(testData, [impulsePerPixel,512,512,2] )); % reshape testData

                     for polPosition = 1:numOfPol                             
                          flapJackAvg(:,:,:,channelSelect,polPosition) = flapJackAvg(:,:,:,channelSelect,polPosition) + mean(RawData(polPosition:numOfPol:impulsePerPixel,:,:,channelId),1);                                                                                                                        
                          flapJackCount(:,:,:,channelSelect,polPosition) = flapJackCount(:,:,:,channelSelect,polPosition) + sum(RawData(polPosition:numOfPol:impulsePerPixel,:,:,channelId) < 5,1);                                                        
                     end % end polPosition
      end  % end j loop  
  end % end i loop
  toc

[dpb]

First, are the data actually collected in the array before the processing and not during? IOW, is the sample case representative of real thing and it is only the loop structure that needs work?

Second, unless the line wrap from the excessively long variable names and comments has caused confusion, it seems the first obvious optimization is to take the line

RawData = single(reshape(testData, [impulsePerPixel,512,512,2] ));

completely out of the loops -- it's invariant to i and j

Another minor optimization is the channel addressing can avoid the if--

channelSelect = boardId + channelId;
if channelId == 1, channelSelect = channelSelect -1; end

is

chSel = bdId + 2*(chId-1);

Beyond that I'll wait to hear on whether it is really a batch processing or not...

[Muthu Annamalai]

Interesting take @dpb. You must be doing peephole and loop optimizations in your mind all the time, :-)

[dpb]

45-some years beginning w/ Philco 2000's 'll do that to ya'... :)

And, I've done quite a lot of this kind of addressing over those years so looking for the expression to generate the index is automagic reaction when see it.

Actually, the cute way here is

for bdID=1,2
  for chID=0:1
    chSel = bdId + 2*chId;

to eliminate the subtraction on the second index and the intermediary indices that aren't used except as the loop indices and the variables used are just copies of same.