MCR 7.17 memory leak when used from Java

We recently shifted from MCR 7.10 to MCR 7.17 which has caused my Java program to die after being running for around 24 hours because all memory has been used.

After running a log of tests on both 7.10 and 7.17 I can see that especially when using advanced data structures, it seems that memory is not released when using disposeArray.

I made the following test program:

package com.test.matlab;

import com.mathworks.toolbox.javabuilder.MWArray;
import com.mathworks.toolbox.javabuilder.MWNumericArray;

public class MatlabMemoryLeakClient {

  /**
   * @param args
   */
  public static void main(String[] args) {
    try {
      // Just to load javabuilder.jar so we know how much memory it uses
      MWNumericArray init = new MWNumericArray(0);
      MWArray.disposeArray(init);
      System.out.println("Press ENTER when ready to go...");
      
      // Wait for user to press enter
      System.in.read();
      
      for (int i=0; i < 5000; i++) {
        final String[] FIELDS = new String[]{"Field1", "Field2", "Field3", "Field4"};
        final int KMAX = 200;
        MWStructArray data5 = new MWStructArray(1, KMAX, FIELDS);
        for (int k=0; k < KMAX; k++) {
          data5.set(FIELDS[0], k+1, new MWNumericArray(k*1.13));
          data5.set(FIELDS[1], k+1, new MWNumericArray(k*9.48));
          data5.set(FIELDS[2], k+1, new MWCharArray("TEST1" + k*9.48));
          data5.set(FIELDS[3], k+1, new MWCharArray("TEST2" + k*9.48));
        }
        MWArray.disposeArray(data5);
        System.out.println("MWStructArray. Run=<" + i + ">");
      }
      
      Thread.sleep(10 * 60 * 1000);
    } catch (Throwable e) {
      System.out.println("Error in matlab testing. Exception=<" + e);
    }
  }
}

I then ran it against MCR 7.10 and MCR 7.17

MCR 7.10 - When started and waiting for user to hit ENTER it used 15.708K memory. After hitting ENTER the memory continously grew until around 1.500 loops. Then it stayed on 31.984K for the following 3.500 loops.

MCR 7.17 - When started and waiting for user to hit ENTER it used 31.224K memory. After hitting ENTER the memory continued to grow, program crashed at 2525 loops (out of 5.000) with a memory usage at 527.304K.

WIth both MCR 7.10 and MCR 7.17 it seems that memory is not relased correctly. But the problem is much more severe with MCR 7.17.

Any ideas to what the problem might be, do I release the memory in a wrong way?

PS: I tried using the dispose command on the object itself also, but that makes no difference.

[Walter Roberson]

This previous discussion might be relevant:

http://www.mathworks.com/matlabcentral/answers/39858-clearing-handle-subclasses-with-timer-objects

[John Datson]

Thanks for the reply. I checked it out, but I don't think this is the same issues. As I understand the issue in the link the problem is that an object is never de-referenced, thus the memory is not released.

In my case I first of all call disposeArray (which I assume releases the memory in the Matlab C structures) and also the Java objects are released and eventually freed by the garbage collector.

Answer 1

[Martijn]

The problem here is with the anonymous MWArrays which you are creating to fill the struct array with. I will try to explain this graphically, but for this I slightly simplify your example: I use only one field: "Field1" and I set KMAX = 1.

You start with creating the MWStructArray:

 MWStructArray data5 = new MWStructArray(1, KMAX, FIELDS);

You then call:

 data5.set(FIELDS[0], k + 1, new MWNumericArray(k * 1.13));

Where the new MWNumericArray(k * 1.13) part creates a new anonymous MWNumericArray with an underlying native MATLAB array with the actual data:

And the set operation makes, in accordance to the documentation "If element is of type MWArray, the cell at index set to a shared copy of the underlying MATLAB array", Field1 point to the underlying native MATLAB array:

Then when your code continues, the anonymous array goes out of scope. However in Java going out of scope means that it gets marked for being disposed, it does not necessarily mean that the object is really deleted at that time; Java itself determines whether and when to run the object's finalizer.

It is important to note here that the Java Garbage Collector is not aware of the size of the Native MATLAB array to which the anonymous MWNumericArray is keeping a reference; it only sees the relatively small Java part of the MWNumericArray. This could lead to the Garbage Collector deciding to never release the anonymous MWNumericArray at all (i.e. it does not think it is worth the CPU cycles to clear the array which is using only a little amount of memory anyway).

So you will have a situation:

Then when you do specifically dispose of the MWStructArray with MWarray.disposeArray, data5 is really deleted and not only marked for deletion. This will also delete Field1 and its reference to the Native MATLAB Array, leaving:

But as you can see the anonymous MWNumericArray may still exist and if so, it does still have a reference to the Native MATLAB Array and as long as this is true, the Native MATLAB Array may not be deleted. This may lead to something which looks like a memory leak, but in fact it is not, it is how finalizers and garbage collection work in Java in combination with how MWArrays may have large underlying native data.

Now how to prevent this from happening?

There are two options:

• Instead of using an anonymous array you can use a temporary named variable which you can specifically dispose:

 MWNumericArray temp = new MWNumericArray(k * 1.13);
 data5.set(FIELDS[0], k + 1, temp);
 MWArray.disposeArray(temp);

The following illustrations show what happens in this situation:

 MWNumericArray temp = new MWNumericArray(k * 1.13);

 data5.set(FIELDS[0], k + 1, temp);

 MWArray.disposeArray(temp);

If you then also call MWArray.disposeArray(data5); all references to the native MATLAB array are removed and then the native array will also be cleared immediately.

• Or even easier: the set method directly accepts native Java datatypes:

 data5.set(FIELDS[0], k + 1, k*1.13);

Which would basically directly create:

Where again MWArray.disposeArray(data5); would remove all references to the native array and clear it.

[Titus Edelhofer]

Thanks for the detailed explanation. Maybe it's worth stating, that for passing parameters from MATLAB back to Java the same principle holds true. When you extract data from a MWStructArray using getField, you should use a variable, i.e.

MWStructArray aField = (MWStructArray) aResult.getField(aName, 1);
// use this aField to create some java result
// mark the variable aField for deletion
MWArray.disposeArray(aField);

Titus