How to use tall to solve out of memory problem

Hello

When I run a ConvertTDMS-code (from File Exchange) in matlab to convert .tdms files to .mat files I get the error "out of memory". I have tried changing the workspace preference and the virtual memory, but it does not seem to solve the problem. Therefore I am now trying to use the tall function as I have read that can do the trick.

I just have to face that I am not enough of a programming-person to understand this more advanced code in the ConvertTDMS file and I can´t figure out how to implement the tall function without changing a lot in the code.

The error seems to come when creating lists of zeros:

%Assign the generic field name

obname=ObjNameList(NameIndex).FieldName;

%Create the 'index' structure

if (~isfield(index,obname))

index.(obname).name=obname;

index.(obname).long_name=long_obname;

index.(obname).rawdatacount=0;

index.(obname).datastartindex=zeros(NumOfSeg,1);

index.(obname).arrayDim=zeros(NumOfSeg,1);

index.(obname).nValues=zeros(NumOfSeg,1);

index.(obname).byteSize=zeros(NumOfSeg,1);

index.(obname).index=zeros(NumOfSeg,1);

index.(obname).rawdataoffset=zeros(NumOfSeg,1);

index.(obname).multiplier=ones(NumOfSeg,1);

index.(obname).skip=zeros(NumOfSeg,1);

end

Please let me know if I need to provide more information in order to get the proper help to solve this.

Best regards,

Henriette

[Henriette Larsen]

Ekstra:

If I take tall(zeros(NumOfSeg,1)) I get a tall array out, but that is not readable to the rest of the code, and then I am stuck with a new error... what do I need to do more to give it the right "format" for the rest of the code to function normally with the tall-array implemented? I don´t understand the examples in the documentation, so that is not so usefull. Any help with this would be much appreciated.

Isn´t there anyone who knows how to use the tall function? please :)

- Henriette

[Mario Malic]

Hi,

unfortunately, I don't know so much about tall but you can check the following video on YouTube, there's a tiny tiny bit on the tall in it starting from 39:10.

MATLAB for Analyzing and Visualizing Geospatial Data | Master Class with Loren Shure

[Javier Ignacio Camacho Hernandez]

Hi Henriette

I have the same problem. Have you solved?

Answer 1

[Harsh Mahalwar]

Hi Henriette,

I can recognise that you’re trying to use the tall arrays to tackle the out of memory error.

After going through your code, I can confirm that the out of memory error is most probably due to the ones and zeros arrays that are being created, as in these operations MATLAB will be trying to acquire a huge chunk of memory (depending upon the value of NumOfSeg variable).

So instead of changing the entire code, we only need to change these lines of code,

%Assign the generic field name 
obname=ObjNameList(NameIndex).FieldName; 

%Create the 'index' structure 
if (~isfield(index,obname)) 
    index.(obname).name=obname; 
    index.(obname).long_name=long_obname; 
    index.(obname).rawdatacount=0; 
    index.(obname).datastartindex=tall(zeros(NumOfSeg,1)); 
    index.(obname).arrayDim=tall(zeros(NumOfSeg,1)); 
    index.(obname).nValues=tall(zeros(NumOfSeg,1)); 
    index.(obname).byteSize=tall(zeros(NumOfSeg,1)); 
    index.(obname).index=tall(zeros(NumOfSeg,1)); 
    index.(obname).rawdataoffset=tall(zeros(NumOfSeg,1)); 
    index.(obname).multiplier=tall(ones(NumOfSeg,1)); 
    index.(obname).skip=tall(zeros(NumOfSeg,1)); 
end 

You can learn more about the tall arrays and how you can use them to avoid out of memory error from these documentation pages, here:

1. Tall Arrays - MATLAB & Simulink - MathWorks

2. Tall Arrays for Out-of-Memory Data - MATLAB & Simulink - MathWorks

I hope this helps, thanks!