Use multiple GPUs for functions

Dear All, I had my previous question, because it was not clear, now I tried to simplify it. At my disposal there are two GPU devices (GeForce GTX 1070 Ti and GeForce GTX 1060 6GB). I would like to parallelize my calculations on both GPUs. Lets say I have 3D gpuArray and I would like to pass this data in chunks to both GPUs, just in my code function is more difficult, this is an example of what I am trying to achieve, and yes it does not work.

clear;
delete(gcp('nocreate'));
nGPUs = gpuDeviceCount();
parpool('local', nGPUs);
d1=rand(10,10,10);
d=gpuArray(d1);
parfor i =1:nGPUs
c1 = zeros(10,10,10);
c=gpuArray(c1);
  for j=1:10
     c(:,:,j)=d(:,:,j)*2;
  end
end
der=c;

It gives temporary variable error.

Answer 1

[Matt J]

Is the question then, why do you get the temporary variable error? The reason is because the variable 'c' is created inside the parfor-loop. It is therefore a temporary variable, meaning that it has no life after the parfor loop. It is both forbidden and illogical to use a temporary variable after the for-loop as you have done at the line,

der=c;

This is because the parfor loop maintains several parallel versions of c. Every parallel worker has its own version of c which might end up carrying a different value at the end of the loop, depending on the parallel operations done to it. So, which of these versions would be assigned to der?

[Matt J]

Maybe this is what you were trying to do,

 clear;

parpool('local', nGPUs);
d1=rand(10,10,10);

c1(1:nGPUs) = {zeros(10,10,10)};

parfor i = 1:nGPUs

    gpuDevice(i-1);

    c=gpuArray(c1{i});
    d=gpuArray(d1);

    for j=1:10
       c(:,:,j)=d(:,:,j)*2;
    end

    c1{i}=gather(c);

end

   der=c1;

[Mantas Vaitonis]

Yes, your way does work, jus I had to change gpuDevice(i-1) to gpuDevice(i), because it did bring me error otherwise. In this case for loop is done on both GPUs and results in c1 array with two same cells. Is there a way to dived loop for both GPUs, that one part of for loop is done by one GPU and other by second GPU simultaneously? Maybe something like this, just not sure if this is a right way to go:

clear;
delete(gcp('nocreate'));
nGPUs = gpuDeviceCount();
parpool('local', nGPUs ); 
d1=rand(10,10,10 ); 
c1(1:nGPUs) = {zeros(10,10,10)};
parfor i = 1:nGPUs
gpuDevice(i);
c=gpuArray(c1{i }); 
d=gpuArray(d1);
if i==1
for j=1:5 
c(:,:,j)=d(:,:,j)*2 ; 
end
end
if i==2
  for j=6:10 
c(:,:,j)=d(:,:,j)*2 ; 
  end
end
c1{i}=gather(c ); 
end
der=c1;

[Matt J]

Just add some more interesting (i,j)-dependence to the operations inside the loop:

 parfor i = 1:nGPUs

    gpuDevice(i);
    c=gpuArray(c1{i});
    d=gpuArray(d1);

    for j=1:10
       c(:,:,j)=somefunction( d(:,:,j) i,j);
    end

    c1{i}=gather(c);

 end

[Mantas Vaitonis]

Could you explain a bit more? Because if I use function somefunction( d(:,:,j), i,j); i and j become input arguments for that function. If I use c(:,:,j,i) still it pass all data to both GPUs, not dividing it to both.

[Matt J]

All of these examples are sending separate copies of c to the different GPUs, and all of them are collecting the results from all GPUs back to c1{}. Because you have been doing identical computation in each iteration of the parfor loop, however, you are getting the same result back from each GPU, making it appear to you that the work is not being split.

I suggest you modify the example so that you can see that the result sent to c1{i} indeed depends on the GPU index, i.

parfor i = 1:nGPUs

    gpuDevice(i);
    c=gpuArray(c1{i});
    d=gpuArray(d1);

    for j=1:10
       c(:,:,j)=d(:,:,j)+i-j; %A fake i-dependent operation.
    end

    c1{i}=gather(c);

 end

[Mantas Vaitonis]

Yes I do understand that these calculations are done in parallel and that same for loop for j=1:10 is processed on both GPUs. But what would be the way if my variable d1=rand(1e8,1e8,1e8); and for j=1:1e8, but divide this for loop for both GPUs, that one is from j=1:5e7 and other GPU j=5e7:1e8, or this is not suitable for GPU? I am able to pass all data to one GPU, but if I pass it to two GPU it should result in data process speedup.

[Matt J]

One way,

 d1Cell={d1(:,:,1:5e7), d1(:,:,5e7+1:end)}; 

 parfor i = 1:nGPUs

      gpuDevice(i);

      c=gpuArray(c1{i});    
      d=gpuArray(d1Cell{i});

      for j=1:size(d,3)
         c(:,:,j)=d(:,:,j)+i-j; %A fake i-dependent operation.
      end
      c1{i}=gather(c);
   end