How does one have arrays in a Matlab share the same memory

Greetings, Is there a simple means in Matlab for two arrays to share the same data space? In other words letting A be a N by M array of type int32 and B being a NxMX4 type int8. In C I would do this with pointers. Thank you

Answer 1

[Stephen23]

It seems to be possible to do this with memmapfile:

myData = int8(1:10)';
fileID = fopen('records.dat','w');
fwrite(fileID, myData,'int8');
fclose(fileID);

m1 = memmapfile('records.dat','Writable',true,'Format','int8');
m2 = memmapfile('records.dat','Writable',true,'Format','int32');

and checking the original data:

>> m1.Data
ans =
    1
    2
    3
    4
    5
    6
    7
    8
    9
   10
>> m2.Data
ans =
    67305985
   134678021

Now we change one of the int32 values:

 >> m2.Data(2) = m2.Data(2) + 1;

and check what happened to the int8 data:

 >> m1.Data
 ans =
    1
    2
    3
    4
    6
    6
    7
    8
    9
   10

So, the 5 has changed into a 6, proving that memmapfile can be used to achieve your goal. You will need to play around with byte order, dimensions, and whatnot, but it seems to be possible to make this happen.

[Guillaume]

"it seems to be possible to make this happen". Whether it's a good idea is another matter.

It may make more sense to just keep one array and typecast it to the other type as needed.

[Patrick Ford]

That may be a clever means, as long as the file is maintained in ram and there is no writing to the disk to update. The whole point of this is to be able to modify data without the overhead of copying. This can be done in Fortran with the equivalence, C with pointers or Pascal with data casting type.

Is there a means of preventing the updating of the file on the HD until it is closed?

[Stephen23]

The easiest solution may be to write a mex function to do this.

[James Tursa]

As Stephen points out, this could be done with a mex routine but with a couple of caveats:

1) You would need to make the shared data copy of a different class using a mex routine that hacks into the variable mxArray structure since there are no existing official MATLAB or mex API functions that will do this. E.g., my typecastx function from the FEX can do this. MATLAB's typecast function can't be used for this since it produces a deep data copy.

2) ALL of the data element changes/updates would have to be done at the mex level so that you could modify values in-place without causing an unsharing data copy to take place. I.e., any data element change that you would make at the m-file level will see the shared variable status and invoke the copy-on-write mechanism and first unshare the variables ... which is precisely what you are trying to avoid.

Answer 2

[Thorsten]

You cannot do that in Matlab.

[Patrick Ford]

I was considering trying libpointer without calling any external C routine. It seems that may work, but it depends on how Matlab manages its memory. Could I depend on the pointer remaining valid or would I need to keep updating it each time I accessed it?

Answer 3

[Philip Borghesani]

A libpointer can be (mis)used to do this but I am not sure it will help in the long run due to the overall performance of and copying out (accessing ptr.value) needed with libpointers.

 pi8=libpointer('int8Ptr',1:16)
 pi16=libpointer('voidPtr',pi8)
 pi16.setdatatype('int16Ptr',1,8) 
 pi16.value(3)=100;
 pi8.value
ans =
  1×16 int8 row vector
     1     2     3     4   100     0     7     8     9    10    11    12    13    14    15    16

[Patrick Ford]

Thank you for the answer. The overhead issue will be if using pointers or the ram disk overhead is significantly less than copying arrays. Since the arrays have the potential the potential of multiple megabytes or higher, I suspect the frequent updating would be a bottleneck. I did something similar when I was using IDL and C. The address to the arrays were not consistently set in memory, so I had to allocate the pointers each time before accessing the array then deallocate, which was considerably faster than copying, and used less ram too.

Answer 4

[Patrick Ford]

Well, I ran into another issue. What I want to do is have an array in MatLab, say a 3D of size (A,B,C) of type int16 or int32 and have basically a pointer to the first address so I can access the entire array as a string of bytes or int8 with a size of A*B*C. If I can set this up, would need to just know the addressing scheme so I would know the where each AxB matrices starts and ends.

So I can either reference as a 3D int32 or 1D int8 (which would be size of A*B*C*4 bytes)

The libpointer allows me to address the same space as int32 or int8. But I cannot create a pointer to an array defined in MatLab.

Example:

a = zeros(10,10) px = libpointer('voidPtr', a)

Altering 'a' does not change px.value, although px.value is the same type and size as a.

[James Tursa]

In your libpointer example, the value that libpointer creates is a deep data copy of "a". There would be no way to use this libpointer, even in a mex routine, to alter any values in "a". E.g.,

>> a = zeros(1,5)
a =

Structure address = 72ca8b0 
m = 1
n = 5
pr = 24545770 
pi = 0
     0     0     0     0     0
>> px = libpointer('voidPtr', a)
px =
libpointer
>> px.value
ans =

Structure address = 72ca178 
m = 1
n = 5
pr = 206aee80 
pi = 0
     0     0     0     0     0

You can see that the pr data pointers for a and px.value are different, which means that they are not shared data copies of each other.

To have variables of different classes sharing memory and being able to alter one without automatic unsharing, see my mex comment above. You would probably need to employ something like my typecastx FEX submission to accomplish it. Or maybe just pass in the int16 or int32 and manipulate the data elements in-place in the mex routine as int8 without ever creating a shared int8 version of the variable.