Update each element of multi dimensional array

Hello!

i am working with multi dimensional arrays, and I need to check every combination of values and update them. For this pusrpose, I am using for loops but they are super slow. I have waited hours but it was still processing. Is there any faster way?

My code for 3D array is like this

for i = 1:nrows
    for j = 1:ncols
        for k = 1:ndepths  
            AI = function(a(i), b(j), c(k));
            cspace (i,j,k) = JKM(AI);
        end
    end  
end

[madhan ravi]

upload you function and the values to test

[Stephen23]

"Is there any faster way?

Is cspace preallocated before the loops?

[bilal javed]

yes, cspace is preallocated,

a = 1:2:360;
b = 1:2:360;
c = 1:2:360;
nrows = length(a);
ncols = length(b);
ndepths = length(c);
cspace = true(nrows,ncols,ndepths) ;
for i = 1:nrows
    for j = 1:ncols
        for k = 1:ndepths  
            AI = function(a(i), b(j), c(k));
            % AI is patch structure for a(i), b(j), c(k)
            cspace (i,j,k) = JKM(AI); % either true or false
        end
    end  
end

I am working on problem where cspace might be 6 or 7 dimensional. But I am stuck with 3D case

[Stephen23]

"I am working on problem where cspace might be 6 or 7 dimensional. But I am stuck with 3D case"

Then you need to vectorize your function.

Remember that each nested loop multiplies the total number of iterations by its own number of iterations, so six or seven loops will require calling your function 360^6 or 360^7 times. Unless your function is exceptionally fast, this will take some time.

[bilal javed]

Stephen Cobeldick! you are right. But I have no idea how to vectorize my function in order to get result from all possible combinations.

can you please give some example for 3D case?

[Jos (10584)]

Why do you need all combinations? What are you going to do with them?

[Guillaume]

If this is exactly how you preallocate cspace:

cspace = true(nrows*ncols*ndepths);

Then it is not preallocated correctly. it has way too many rows and columns, and only one page, which will grow at each k step. You'll end up with a nrows*ncols*ndepth x nrows*ncols*ndepth x ndepth which will have been unnecessary copied ndepth-1 times.

Proper preallocation would be:

cspace = true(nrows, ncols, ndepths);

"But I have no idea how to vectorize my function"

How you'd vectorise the function depends entirely on what the function does. Some functions cannot be vectorised. Without more details, we can't really help you.

[madhan ravi]

If you had responded to this comment a solution would have been proposed by now.

[bilal javed]

Guillaume! You are right. I wrote cspace equation wrong.

cspace = true(nrows, ncols, ndepths);

Actually I am working on configuration space for my 3 degree of freedom manipulator. AI is patch structure for my manipulator and its shape keeps change on every angle combination. JKM funtion check if AI collided or not with obstacles speified before.

The function AI takes angles combination, do some forward kinematics, find position of each link with repect to origin, use these positions to create patch structure AI for manipulator. AI then checked by JKM function if it's colision free or not and return 1 or 0 for given angle combination i,j,k

[Guillaume]

As Madhan said, we need to see the code for that JKM function. From the sound of it, it is probably complex enough that it may not be possible to vectorise it.

[bilal javed]

Hello !

you were right. Without having a look on actual code, it's hard to solve the problem. I have been working for many days to solve this. Now I new sitiation.

Here function fv is alos using parfor loop. I am using sections to access small part of my large data. But still I am having erro OUT OF MEMORY after 1st iteration.

% Theta ranges
theta1_range = 0:2:360;
theta2_range = 0:2:360;
theta3_range = 0:2:360;

%%%%%%%%%%%%%%%%%%%%%%

nrows = length(theta1_range);
ncols = length(theta2_range);
ndepths = length(theta3_range);

[TH1, TH2, TH3] = ndgrid(theta1_range,theta2_range,theta3_range);
THETA1 = TH1(:);
THETA2 = TH2(:);
THETA3 = TH3(:);
l = length(THETA1);
clear TH1 TH2 TH3
cspace_coords = [THETA1, THETA2, THETA3];
clear THETA1 THETA2 THETA3 theta1_range theta2_range theta3_range
cspace = [];


if (rem(l, 2) == 0)
    sections = round(linspace(0,l,20)); % divide dimension l into small equal parts
else 
    sections = 0:round(l/20):l; % divide dimension l into small equal parts
    if sections(end) ~= l
        sections = round([sections, l]);
    end
end
%%
for i = 1:(length(sections)-1)
    fprintf ('%d of %d\n', i, (length(sections)-1))
    fv = ThreeLinkRobot(DH_para, cspace_coords((sections(i)+1):sections(i+1),:));
    collide = true(length(fv),1);
    parfor j = 1:length(fv)
        collide(j) = GJK(patch(fv(i)),[patch(obstacle1);patch(obstacle2);patch(obstacle3)],10);
    end
    cspace = [cspace; collide];
    clear fv;
end
%%

[Jan]

While clear is usually less useful than the programmer thinks, letting an array grow iteratively is very inefficient. So avoid cspace = [cspace, collide], but pre-allocate.

If you write "OUT OF MEMORY after 1st iteration", which line does this mean exactly? The less the readers have to guess, the better.

[Guillaume]

I assume the clear are an attempt to save memory. Most of the variables you clear use very little memory so it's a waste of time. The largest matrix in the code you show is cspace_coords which only requires 135 MB of memory.

These matrices are not the cause for the low memory.

In any case, as Jan says we need to know the line where the error occur.

I would replace this lot:

nrows = length(theta1_range);
ncols = length(theta2_range);
ndepths = length(theta3_range);
[TH1, TH2, TH3] = ndgrid(theta1_range,theta2_range,theta3_range);
THETA1 = TH1(:);
THETA2 = TH2(:);
THETA3 = TH3(:);
l = length(THETA1);
clear TH1 TH2 TH3
cspace_coords = [THETA1, THETA2, THETA3];
clear THETA1 THETA2 THETA3 theta1_range theta2_range theta3_range

by:

cspace_coords = zeros(numel(theta1_range), numel(theta2_range), numel(theta3_range), 3);
[cspace_coords(:, :, :, 1), cspace_coords(:, :, :, 2), cspace_coords(:, :, :, 3)] = ndgrid(theta1_range, theta2_range, theta3_range)
cspace_coords = reshape(cspace_coords, [], 3);

[Bilal Javed]

I am using matlab 2018b. wndows 10, intel corei7, 16 gb ram, 250 gb ssd, 1 tb hdd, nvidia 4gb graphic card

[Walter Roberson]

[patch(obstacle1), patch(obstacle2),patch(obstacle3)] look like array indexing that could be done ahead of time.

[per isakson]

Have you tried Profile execution time for functions on your problem?

As Jan said, avoid cspace = [cspace; collide];

[Guillaume]

The error message tells you the out of memory occurs in the parfor. So, it all depends on that GJK function for which you haven't shown the code.

What's the output of memory before you execute the parfor loop? By far, the biggest variable is your fv which uses ~4.5 GB of memory.

[Walter Roberson]

I just noticed that your fv is a struct with fields face and vertices. That suggests that you are creating four patch objects for each of your 296487 struct array elements, and that those form parameters to whatever GJK does. That is a lot of graphics objects, over 72 million verticles for the fv alone, and unknown numbers of graphics elements triggered by the obstacle patches.

Note that graphics objects created on parallel workers cannot change the display: parallel workers are different processes that work independently without connection to the graphics display thread.