GPU Array Max Dimensions/Size (i.e., int32 of ~2e9) Not Same as Other CUDA-Enabled Languages (i.e., Python 3.9 - Tensorflow 2.11)

I would like to create Matlab GPU arrays that have a number of elements exceeding the max value of int32 (~2e9). In Matlab, when I try to do this, I get a "Max variable size exceeded" error. Other answers on this forum point to this limitation in GPU array size being caused by limitations in the CUDA API. However, I don't believe this is accurate. Other languages, like Python 3.9 using theTensorflow 2.11 library for example, that are built on CUDA/CUBLAS allow for arrays that have a number of elements that exceed the MATLAB limitation.

Why is this the case, and why does Matlab seem limited where other languages are not?

The version of Matlab that I am using is 2024a, tested on both Win11 and Ubuntu 20.04LTS with a Nvidia Ada A6000 GPU (VRAM 48GB).

Matlab Code to reproduce:

g = gpuDevice();
arr = ones(1e4, 1e4, 60, 'single');
gpu_arr = gpuArray(arr);

Matlab ERROR:

Error using gpuArray

Maximum variable size allowed on the device is exceeded.

Python Code to reproduce:

import tensorflow as tf
arr = tf.ones([10000, 10000, 60])

Answer 1

[Joss Knight]

In CUDA 12 NVIDIA introduced a limited set of 64-bit integer APIs in cublas and other libraries, allowing arrays with more elements to be passed to the library routines which MATLAB depends on for basic operations such as matrix multiplication. As stated in the CUDA documentation, this support is piecemeal.

MATLAB only just adopted CUDA 12 in the latest R2024a release. Adopting these new APIs in a way that provides a consistent high quality user experience may take a little while.