How to transfer date from gpu to workplace faster?

Hi everyone,

There is bottleneck in my codes.

I use CUDA to accelerate some parts of my algorithm, and the results should be returned workplace for following calculation.

I use gather(), firstly, to transfer data from gpu to workplace. However, the profiler shows that execution time of gather() (1000s) account for almost 1/3 of total time (3000s). My question is any stratgies can help to improve data transfer process?

ALso, I posted a similar question https://www.mathworks.com/matlabcentral/answers/703592-why-does-the-gather-function-only-take-around-0-001-seconds-at-command-window-while-1s-in-a-loo . But I still confuse about how gputimeit() or wait(gpuDevice) work?

Thank you in advance!

[Walter Roberson]

What evidence do you have that it is the data transfer operation that is taking so much time, as compared to time spend waiting for the GPU to finish ?

[Han F]

Hi Walter, thank you for your comment. Following profiler is the latest result. Line 649 and 651 speed 650 seconds and total time is 2900s. [dk1,...,dk10] is a 1000*9 matrix.

Do you mean there is no way to reduce the time spend waiting for the GPU to finish ?

[Joss Knight]

Can you explain how you expect to reduce the time spent waiting for the GPU to finish? The GPU has to finish computing a result before it can return a result! The only way to make that go faster is to get a faster GPU - or maybe write your own CUDA code, if you think you can implement a faster algorithm.

Answer 1

[Walter Roberson]

These are the strategies to reduce the time spent in gather():

• do less complex operations on the GPU so that the GPU finishes the task faster
• Use smaller output matrices -- ideally only a scalar
• Use a faster GPU with faster memory and faster I/O bus
• Use faster memory on you CPU
• don't use the GPU for operations that would complete about as quickly on the CPU. For example, even though addition is faster on the GPU, if you use large arrays, most of the time is spent transfering the data to the GPU and the results back from the GPU

There is no

gpuprefs(gpudevice, 'gather_rate', 'ludicrous_speed')

You have to wait for the GPU to finish executing the task given to it, and the driver has to do a DMA operation to get the data back.

Oh yes, there is one more strategy to reduce the time spent in gather():

• use Linux instead of Windows. The Windows drivers are less efficient (for reasons having to do with the architecture requirements that Windows places.)

[Han F]

Thank you so much! It seems clear now. Thank you for your time. Appreciate.

[Han F]

By the way, what's the meaning of gpuprefs(), I did not find it in help

[Walter Roberson]

https://youtu.be/NAWL8ejf2nM