Singular value decomposition, Cant get the final results

Im trying to Perform SVD as the following, My laptop RAM is 64GB

The XTrain size is 120*120*1*39000

I tried to reduce the number of matrices to 120*120*1*10000, but still getting the same message

But the MATLAB stucks in this message and the evaluation percentage doesnt increase, any suggestion please?

N=14;

X=reshape(XTrain,[], size(XTrain,4)).';

size(X)
vn=max([vecnorm(X,inf,1),1],1);


%Process as tall
T=tall([X,  ones(height(X),1)]./vn);

clear X

[~,~,V]=svd( T , "econ"); 

clear T

Abt=gather( head( V(:,end+1-N:end))  ) ./vn';
A=Abt(1:end-1,:)';
b=Abt(end,:)';
s=vecnorm(A,2,2);
[A,b]=deal(A./s, b./s);

Message

Evaluating tall expression using the Parallel Pool 'Processes':
- Pass 1 of 2: Completed in 7.1 sec
Evaluation 45% complete

[Torsten]

Why do you ask such complicated technical questions here in the forum ? Contact Technical Support.

[M]

@Torsten how to contact Technical Support.?

Also I asked it here beacuse it may others faced the same problem, or may be there is a problem in the code.

I want to check the opinions about the code

[Torsten]

https://uk.mathworks.com/support/contact_us.html

If you get an answer from there, please post it here so that others can profit from it.

I cannot help you with your question, but the first thing I would try is to apply the code to a smaller-sized problem and see if it works.

[Torsten]

As said, I have no experience with parallel computing and tall arrays, but since the problem with your code seems to be of technical nature, the Technical Support seems to be the correct address in my opinion.

Maybe you could build the svd's of the 39000 matrices one by one and classify afterwards with the svd's so obtained ?

[M]

@Torsten but what do you think about the analysis code for getting the affine subspace and the grassman manifold, is it correct?

Regardless the tall arrays

[M]

@Torsten Maybe you could build the svd's of the 39000 matrices one by one and classify afterwards with the svd's so obtained ?

Please explain more, these 39000 matrices for training and belongs to the same class, and how can I do the test then?

At the end I want the Grassman manifold for each class

[Torsten]

Your question was

But the MATLAB stucks in this message and the evaluation percentage doesnt increase, any suggestion please?

My advice was: Contact Technical Support.

If your question was: Do you know if I do something wrong ? How can I get the affine subspace and the Grassman manifold ? - then I'm not able to contribute something.

[Matt J]

Have you tried leaving it to run overnight? Maybe it just takes a while.

Answer 1

[Matt J]

Since you are now on a more powerful computer with more RAM, perhaps you should also try doing the SVD without tall arrays.

Regardless, you should probably clear XTrain from memory as soon as it is no longer needed,

X=reshape(XTrain,[], size(XTrain,4)).'; clear XTrain

so as to maximize available RAM.

You might also consider using single floats to reduce RAM consumption as well,

 X=reshape(single(XTrain),[], size(XTrain,4)).'; clear XTrain

although I am not sure what impact the reduced precision will have on the SVD results.

[M]

@Matt J perhaps you should also try doing the SVD without tall arrays.

Thanks Matt, This worked. But I am wondered how it's working and the tall array which is supposed to work in less memory is not working.

[Matt J]

Thanks Matt, This worked.

I'm glad, but please Accept-click the answer to indicate so.

But I am wondered how it's working and the tall array which is supposed to work in less memory is not working.

We don't know that it is "not working" with tall arrays. It's possible you didn't wait long enough.