gpuDevice: Available memory greater than Total Memory -- What might be the cause?

Hello, I'm using mexcuda compiler to compile a mex function(ending in .cu) which uses cufft toolkits to accelerate fft.

Both the input and output are 3D single gpuArrays, I'm using matlab2021a, compiler is 'mexcuda -R2018a' and my graphic card is 'NVIDIA GeForce RTX 3090'.

I'm running into this problem: The result of the function turns out correct, but all following matlab functions can't be performed as long as they involve computing on gpu. So I'm supposing that there's someting wrong with my gpuDevice.

A simple gpuDevice check after performing my function gives:

  CUDADevice with properties:

                      Name: 'NVIDIA GeForce RTX 3090'
                     Index: 1
         ComputeCapability: '8.6'
            SupportsDouble: 1
             DriverVersion: 11.4000
            ToolkitVersion: 11
        MaxThreadsPerBlock: 1024
          MaxShmemPerBlock: 49152
        MaxThreadBlockSize: [1024 1024 64]
               MaxGridSize: [2.1475e+09 65535 65535]
                 SIMDWidth: 32
               TotalMemory: 2.5770e+10
           AvailableMemory: 4.9904e+10
       MultiprocessorCount: 82
              ClockRateKHz: 1695000
               ComputeMode: 'Default'
      GPUOverlapsTransfers: 1
    KernelExecutionTimeout: 1
          CanMapHostMemory: 1
           DeviceSupported: 1
           DeviceAvailable: 1
            DeviceSelected: 1

I found that the AvailbleMemory is greater than TotalMemory after function execution, and I think here is the problem.

Any attempt to run other functions would arise CUDA_ERROR_ILLEGAL_ADDRESS error.

Memory manipulation (or maybe memory related) in my .cu file are listed as follows:

            mxGPUCreateFromMxArray
            mxGPUCreateMxArrayOnGPU
            cudaMalloc
            cudaMemcpy
            cudaFree (in pair with cudaMalloc)
            mxGPUDestroyGPUArray (in pair with mxGPUCreateFromMxArray)
            mxGPUDestroyGPUArray (in pair with mxGPUCreateMxArrayOnGPU)

I'm wondering what might be the cause of problem. Any help would be appreciated!

[Walter Roberson]

Is this Windows or Linux? The memory interface is different for the two.

How much RAM do you have? Would it happen to be 16 gigabytes?

[Joss Knight]

AvailableMemory is computed from the hidden property FreeMemory, which is a direct result of querying the driver. To produce a result like this it would probably need to be returning an invalid result. If you've done something bad (like write into illegal memory) you could disrupt the driver and essentially break it. So:

What does this return:

gpu = gpuDevice;
gpu.FreeMemory
!nvidia-smi

It's plausible this has something to do with allocating Unified Memory, but unlikely.

General rule for MEXCUDA (which it looks like you're following):

• Call cudaFree for any memory you cudaMalloc (you must not cudaFree GPU memory you retrieved from a mxGPUArray).
• Call mxGPUDestroy for any mxGPUArray you created using mxGPUCreate.

[Yuxin Yang]

Thanks for your advice! I solved it...Turns out that I'm using two different graphic cards in my .cu file and matlab terminal. I'm switching gpuDevice in the .cu file, that's why the memory changed to a larger one.

The gpu.Freememory check was a very good startoff for my debugging, the system check is also a good hint. As you can see, I'm really a greenhand at cuda programming. So thanks so much for your help!

[Joss Knight]

So are you saying that you are calling cudaSetDevice in your code and not switching back to the currently selected device at the end? So MATLAB's pooled memory on Device 1 is being added to the FreeMemory of Device 2.

This is a flag to me; I feel that we should look into checking that the right device is still selected after calling a MEX function. Thanks for the heads-up.

[Yuxin Yang]

Yeah, that's what I mean.

Glad to be helpful.

Answer 1

[Yuxin Yang]

Turns out that I'm using two different graphic cards in my .cu file and matlab terminal, which has memory of 25 GiB and 50GiB respectively. At first,I was using the 25GiB one. When I call the function, I'm switching gpu to the 50 GiB one, that's why the memory changed to a larger one.

Note: Index of GPU starts from 1 in matlab, but 0 in cuda programming. Hope it will helpful to anyone!