Memory problems with Cell Array mex file

I got a memory issue with the following piece of mex-file codes. The codes do yield correct results and without crash in Debug mode in visual studio 2022, but I got a crash when running them in Release mode with errors about heap memory issue related to the variable vOut. Here vOut is of type std::vector<double> and is the return value of a function. I have tried different ways to detect the problem, but could not solve it. I greatly appreciate if anyone can help me with the issue.

Below is the extracted codes that cause the problem:

       std::vector<double> vOut;//Input argument of the function and is passed by reference. It's used to store returning result.
 
      //Input
        mmxArray* mxToolID = mxCreateDoubleMatrix(1, 1, mxREAL);
        *(double *)mxGetPr(mxToolID) = (double)nToolID;// nToolID is an input of type int
        
        //Ouputs
        int nargout = 1;
        mxArray *mxNP = mxCreateCellMatrix(1, nargout);

        //this function is from an external dll file generated from matlab
        bool retval = mlfSimulator(1, &mxNP, mxToolID);

        //Decode the ouput mxNP which contains a cell array with each cell being a 2D matrix
        
        mxArray* mxOut = mxGetCell(mxNP, 0);
     
        if (mxOut != nullptr)
        {
            size_t n_cell = mxGetNumberOfElements(mxOut);
            for (size_t k = 0; k < n_cell; k++) 
            {
                mxArray* cell = mxGetCell(mxOut, k);// cell is a 2D matrix
                double* ptr = mxGetPr(cell);
                
                if (ptr != nullptr)
                {
                    size_t rowNum = mxGetM(cell);
                    size_t columnNum = mxGetN(cell);
                    
                    for (size_t i = 0; i < rowNum; i++)
                    {
                        for (size_t j = 0; j < columnNum; j++)
                        {
                             size_t loc = j * rowNum + i; 
                             vOut.push_back(ptr[loc]);
                         }
                     }
                 }
              }
          }
         
         if (mxToolID != nullptr) mxDestroyArray(mxToolID);
         if (mxNP != nullptr)  mxDestroyArray(mxNP);

[Truyen]

I did some testing to isolate the issue and it appears that at least a part of the problem is related to the statement vOut.push_back(ptr[loc]). I'm still not sure if the problem is due to my use of the pointer ptr or due to the use of the push_back operation for std::vector.

[Bruno Luong]

I'm not sure but I have some doubt

mxArray* cell = mxGetCell(mxOut, k);// cell is a 2D matrix
double* ptr = mxGetPr(cell);

cell can be NULL, in this case it seems mxGetPr(NULL) migh not be a valid call. Similar for mxGetM(cell); mxGetN(cell);

[Truyen]

Thank you Bruno for spotting that. It's a legit point and I get that fixed. However, the cause of the heap memory problem seems to be elsewhere. After some more testing, I see that the push_back operation to vOut is related to the issue. If I replace the statement vOut.push_back(ptr[loc]) by some calculations with value ptr[loc] (and do not push_back to vOut, but push_back to some local vector), then the problem disappears. This is strange as I do use push_back to a similar argument vector passed by reference at another part of the program and it works fine. BTW, my codes run fine in the Debug mode. It only crashs when I run it in the Release mode.

[James Tursa]

Bruno has likely pointed out the issue (+1), and I will double down on that. These lines are certainly suspect:

mxArray *mxNP = mxCreateCellMatrix(1, nargout); // All elements of mxNP are NULL
bool retval = mlfSimulator(1, &mxNP, mxToolID); // Elements of mxNP filled in here?
mxArray* mxOut = mxGetCell(mxNP, 0);
if (mxOut != nullptr)
{
        size_t n_cell = mxGetNumberOfElements(mxOut);
        for (size_t k = 0; k < n_cell; k++) 
        {
            mxArray* cell = mxGetCell(mxOut, k);// cell is a 2D matrix
            // Maybe you should check cell to make sure it is not NULL here???
            double* ptr = mxGetPr(cell);

You as the programmer must fill the cell array elements with valid mxArray pointers before you can use them and dereference them. You haven't shown us enough code to definitively track down the problem since you haven't shown us the code that fills in the mxNP elements. What is going on in mlfSimulator( )? Until you show us this code we can't track down your problem, but as a guess it sure looks like maybe not all of the mxNP elements got filled with valid mxArray pointers. And there are lots of other checks you could be doing to make your code more robust, like ensuring that cell is a real double full 2D matrix, etc.

I would note that the following code does not store the elements in the same memory order. I.e., the memory order of the data in the 2D MATLAB matrices is different from the memory order of the data in your vOut variable. Maybe this doesn't make a difference to you, or it is intentional, but I thought I would point it out:

            for (size_t i = 0; i < rowNum; i++)
            {
                for (size_t j = 0; j < columnNum; j++)
                {
                    size_t loc = j * rowNum + i; 
                    vOut.push_back(ptr[loc]);
                }
            }

As an aside, this:

            mmxArray* mxToolID = mxCreateDoubleMatrix(1, 1, mxREAL);
            *(double *)mxGetPr(mxToolID) = (double)nToolID;// nToolID is an input of type int

is a very convoluted way of just doing this:

            mmxArray* mxToolID = mxCreateDoubleScalar(nToolID);

[Truyen]

Thank you James for insightful comments and suggestions. There are a lot of matlab codes used to generate the function mlfSimulator( ) via matlab compiler, and I intentionally do not want to focus on that. The part of codes around the line "size_t loc = j * rowNum + i" is also intentional as I want to collect information in the row-wise order instead of column-wise order as the memory order. I really like your rewritten codes for defining mxToolID. I'll learn about this and your other suggestions to improve my codes.

I think I now know what causes the heap memory problem in my program. This involves two simple lines of codes for the function argument input variable vOut that I did not show here, namely

vOut.clear();

vOut.reserve(N);

I did some testing and things are working fine by removing these two lines. I guess mex files might have issue with allocating memory, so I now try to avoid asking them to handle any of that if possible. Thank you and Bruno for your kind help. I greatly appreciate it.

Answer 1

[Bruno Luong]

Another potential issue is your cell does not contain double as assumed.

In this case

double* ptr = mxGetPr(cell);

and later accesiing

ptr[loc]

is wrong.

If you have recent MATLAB and compile the mex with -R2018a option, it is safer (and faster) to call

double* ptr = mxGetDoubles(cell);

or you should check the class of te cell before doing such pointer manipulation.

[James Tursa]

@Bruno Luong I don't understand your "safer (and faster)" comment. If you compile with -R2018a, then mxGetDoubles( ) or (double *)mxGetData( ) are the only options to get the pointer to double data. mxGetPr( ) isn't even allowed in this case. For speed comparisons, there is a performance hit if you use complex variables and don't use the -R2018a option because in that case you get a deep copy-in-copy-out behavior. Is that what you are referring to? The act of just getting the data pointer would be pretty much the same in both cases.

[Bruno Luong]

mxGetPr is allowed when compiled with -R2018a, acoording to my test.

mxGetDoubles is safer since it returns NULL when using with non-double variable, so OP code catch it later in skip processing the data.

Yes you are right, it is faster only when applied on complex, which doesn't seem to be the case with OP. My remark in general remark of advantage using mxGetDoubles vs mxGetPr.

[James Tursa]

"... mxGetPr is allowed when compiled with -R2018a, acoording to my test. ..."

Hmmm ... guess I am remembering this incorrectly. I thought it was disallowed, but maybe it is just deprecated with a note saying that in the future it will be disallowed. I don't have access to all the older MATLAB versions that I used to, so can no longer test stuff like this for version differrences. When I read the current doc it simply states that mxGetPr( ) is not recommended, and will produce a runtime error if used on a complex variable with the -R2018a option. But I distinctly remember getting compile errors when I tried this back in 2018 ... or so I thought. I will try to find the source of my comment about this ...