Combining masks in 4D matrix

I have a 4D matrix of logical values with dimensions [trials, channels, frequencies, time] which contains about 500,000,000 elements which are mainly false. I would like to identify when there are a number of consecutive true cells in the time dimension. The number of consecutive elements are the same for each trial and channel, but will vary depending on frequency and are held in the variable nr_of_cycles.

Here is my pseudo code:

for i_freqs = foi_idx                   % foi_idx are indices of the frequencies (3rd dim) where the operation is to be done
    conseq_n = nr_of_cycles(i_freqs);   % nr_of_cycles is a simple 1d array
    
    % Below is the function i need help with. It needs to go through in the
    % 4th dimension (time) and only keep true values when they are part of
    % a group of conseq_n consecutive trues
    mask(:,:,i_freqs,:) = help_me_with_this_function(mask,conseq_n);          
end

So for example if mask(1,1,1,:) is [0 1 0 1 1 0 1 0 1 1 1 1 1 1 1 0 1]

then help_me_with_this_function(mask,5) should return mask so that mask(1,1,1,:) is [0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0]

thanks!

[Jan]

What does "500 mm elements" mean? Why do you assume loops to be slow?

Is the mask really a sparse object, or do you mean, that it contains mainly zeros? Is is a logical array?

[Jlil]

500 mm was a poor abbreviation - what i meant to write what that the matrix contains 500,000,000 values.

I wrote sparse meaning that the majority of the values are false.

I have updated the post for clarity on both these items - thanks

I am assuming that nested for loops will be a poor way of handling so many values across four dimensions but havent really tested. I was assuming that there were better ways maybe with vectorization of handling this.

[Jan]

Don'toverestimate vectorization. If large temporary arrays are created, the code can be slower than with loops.

Answer 1

[Jan]

With FEX: RunLength :

s    = size(mask);
mask = reshape(mask, [], s(4));
for k = 1:size(mask, 1)                       % [EDITED 2] Typo, data->mask
    [b, n]          = RunLength(mask(k, :));  % [EDITED] (:,k) -> (k,:)
    b(n < conseq_n) = 0;
    mask(:, k)      = RunLength(b, n);
end
mask = reshape(mask, s);                      % [EDITED 2]

Please try this. You need a C-compiler or download pre-compiled mex functions from the page mentioned in the documentation.

[Jlil]

Thanks Jan,

Clever bit of code and i think i get the gist of it. I have one problem, which is that the number of consecutive true values depend on the frequency dimension (dim #3). I think i managed to adapt your code by adding a dimension giving me the following:

s    = size(mask);
mask = reshape(mask, [], s(3),s(4));
for i_freqs = foi_idx      % This is the indices to my frequencies in dim 3
    for k = 1:size(mask, 1)
        [b, n]          = RunLength(squeeze(mask(k,i_freqs, :))); 
        b(n < nr_of_cycles(i_freqs)) = 0;                              
        mask(k,i_freqs,:)      = RunLength(b, n);
    end
end
mask = reshape(mask, s);

A few notes:

• in your 3rd line, you are referring to size(data,1) I am assuming data should be my mask variable?
• I am not sure of the difference between resize and reshape but will look into it. I think reshape does the job for me though hence why i changed it

[Jan]

@Jlil: Yes, "data" and "resize" was a typo.

Is this code fast enough?

[Jlil]

It is, takes 5-10 seconds to run which is just fine - FEX and reshape are new to me so io learned a bit as well

Many thanks for your help!