Libpointer creating multiple copies of data with C++ code.

I am attempting to use libpointer to pass array data to a C++ DLL. Note that this part has so far been successful - the DLL is working as expected, and data can be passed from Matlab to the DLL.

We can test this basic functionality by trying the following function in C++:

void SetArrayValue(double* array, int index, double value)
{
    array[index] = value; //%Set the value of array, at the index specified in the argument
}

When we call this in Matlab via:

y = [2, 4, 6];                                          % Create a small array to test
p_y = libpointer('doublePtr', y);                       % Create pointer to this array
calllib('DLL_Library','SetArrayValue', p_y, 0, -1000)   % Modify the 1st value to "-1000"
get(p_y, 'Value')                                       % Get the value at the pointer (expect [-1000, 4, 6])
y                                                       % Get the value of y (expect [2, 4, 6])

We get the expected values

ans =                                      % Value of data pointed to by p_y
       -1000           4           6       % This was modified successfully by the C++ code
y =                                        % Value of original variable
     2     4     6                         % Not modified

However, I would like to do something a little more complex in my C++ functions. Specifically, I'd like to create a static pointer to my array, so that I do not need to pass it into C++ with each call. I am trying this with the following tester functions

static double* testvar = NULL;   //%Static pointer to a double array

void SetTestVar(double* val)     //%Set the pointer to the address of my array
{
    testvar = val;
}

double GetTestVar(int idx)       //%Get a value from the array
{
    return testvar[idx];
}

void ModifyTestVar(int idx, double val)    //%Change a value in the array
{
    testvar[idx] = val;
}

I can test this in Matlab with the following code:

x = [1, 2, 3];                                    % Create the array for testing
p_x = libpointer('doublePtr',x);                  % Create pointer to this array
calllib('DLL_Library','SetTestVar', p_x);         % Set the pointer "testvar" in the C++ code to correspond to the array at p_x

res = [calllib('DLL_Library','GetTestVar', 0),... % Get the value from the C++ code
    calllib('DLL_Library','GetTestVar', 1),...    % we expect [1, 2, 3] here
    calllib('DLL_Library','GetTestVar', 2)]       % and that's what we get

set(p_x, 'Value', [999, 999, 999])                % Use Matlab to set the value at the pointer to something else
get(p_x)                                          % Get the value from Matlab: we get [999, 999, 999]

calllib('DLL_Library','ModifyTestVar', 2, -1000); % Now try modifying the array using the C++ code - change the 
                                                  % 3rd element to -1000

get(p_x)                                          % Get value at p_x - we get [999, 999, 999]

res = [calllib('DLL_Library','GetTestVar', 0),... % Get the value from the C++ code
    calllib('DLL_Library','GetTestVar', 1),...    % we get [1, 2, -1000]
    calllib('DLL_Library','GetTestVar', 2)]       

x                                                 % Get value from x
                                                  % we don't expect this to change. It's still [1, 2, 3]    

In the 2nd example, I can count 3 copies of x:

1. Original x from Matlab: x = [1, 2, 3]
2. Value of p_x, with final value [999, 999, 999]
3. Value of *testvar in C++ code with value [1, 2, -1000]

If testvar is not pointing to the same memory as p_x, then where is it pointing?

This is important to me, because I anticipate my arrays being ~1 GB in size, with several arrays in use. It is not efficient to have 3 copies. Am I going about this in entirely the wrong way?

To avoid having an "x - y" problem here, I'll describe my problem a little more deeply. I need to pass several arrays into a C++ code, have this code run several functions on the arrays, and be able to access values from within the arrays at intermediate points between running functions.

Answer 1

[Hornett]

Your issue stems from how MATLAB handles data passed to functions like libpointer and how it interacts with external libraries like your C++ DLL. When you pass MATLAB data to a C++ DLL using libpointer, MATLAB does not necessarily pass the pointer to the original data array. Instead, MATLAB can create a copy of the data and pass a pointer to this copy, especially if MATLAB determines that the data needs to be marshaled to match the expected format of the external code. This behavior is why you're observing what seems like multiple copies of your array.

When you first set testvar in your C++ code to point to the array passed from MATLAB, testvar is indeed pointing to the memory location of the data MATLAB prepared for the DLL call. However, when you later modify the array in MATLAB using set(p_x, 'Value', [999, 999, 999]), MATLAB updates the memory associated with p_x, but this does not automatically propagate changes to the memory location that testvar is pointing to, because testvar is not aware of MATLAB's memory management mechanisms.

You can check the following documentation : https://www.mathworks.com/help/matlab/ref/calllib.html