Fasten the speed of assigning values to a large tensor
3 views (last 30 days)
Show older comments
Hi! Now I have a for-loop assigning values to some entries of a large tensor at every iteration. But it is too slow since the tensor is large. Now I am thinking if there is any way to fasten this speed. One simple example is shown below. Here I used 'ndSparse' function to create a four dimensional tensor instead of 'zeros' function. The link of 'ndSparse' is https://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays. ndSparse generalizes 'sparse' function from sparse matrix to high dimensional sparse tensor.
% Simple example
M = 500;
A = ndSparse.build([M,M,M,M]);
n = 10;
for i=1:n
disp(i)
idx1 = M/n*(i-1)+1:M/n*i;
idx2 = M/n*(i-1)+1:M/n*i;
idx3 = M/n*(i-1)+1:M/n*i;
idx4 = M/n*(i-1)+1:M/n*i;
tmp = randn([M/n,M/n,M/n,M/n]);
A(idx1,idx2,idx3,idx4) = tmp;
end
Is there any way to make it faster?
Thank you very much!
5 Comments
Answers (2)
Jan
on 28 Apr 2021
Edited: Jan
on 28 Apr 2021
Avoid creating index vectors explicitly:
idx1 = a1:b1;
A(idx1, :) = tmp; % Slow
A(a1:b1, :) = tmp; % Faster
In the first case, each element of idx1 is tested, if it is a valid index:
% For each index something like this is performed:
if idx(1) == flor(idx(1)) && idx(1) > 0 && idx(1) < size(A, 1)
With a1:b1 Matlab checks the validity of the first and last element only.
Please post a minimal working example. Before the readers can test, if their suggestion is faster, they currently have to implement your code based on guesses, what
A = create a sparse all zero 6-D tensor of size (M,M,M,M,M,M);
or
idx1 = a1:b1; (b1-a1=m)
explicitly means.
2 Comments
Jan
on 3 May 2021
@Jason Yang: You are right, ndSparse is infact an exception, because this user defined class is not supoorted by Matlab's JIT acceleration.
Matt J
on 3 May 2021
Edited: Matt J
on 3 May 2021
You can save some time if you pre-allocate the array:
% Simple example
M = 400;
n = 10;
tic
A = ndSparse.spalloc([M,M,M,M],(M/n)^4*n); %Pre-allocate
%A=ndSparse.build([M,M,M,M]);
for i=1:n
idx1 = M/n*(i-1)+1:M/n*i;
idx2 = M/n*(i-1)+1:M/n*i;
idx3 = M/n*(i-1)+1:M/n*i;
idx4 = M/n*(i-1)+1:M/n*i;
tmp = randn([M/n,M/n,M/n,M/n]);
A(idx1,idx2,idx3,idx4) = tmp;
end
toc%Elapsed time is 1.282638 seconds.
I don't think there are great gains to be made, however. Most of the computation is being spent just generating the entries for the array:
clear B
tic;
[B{1:4}]=ndgrid(1:M/n);
B = reshape( cat(5,B{:}) ,[],4);
[C,vals]=deal(cell(n,1));
for i=1:n
C{i}=B+M/n*(i-1);
vals{i} = randn(prod([M/n,M/n,M/n,M/n]),1);
end
C=cell2mat(C); vals=cell2mat(vals);
I=sub2ind([M,M,M,M],C(:,1),C(:,2),C(:,3));
J=C(:,4);
toc; %Elapsed time is 1.499094 seconds.
tic
S=sparse(I,J,vals, M^3,M, (M/n)^4*n);
toc; %Elapsed time is 0.344042 seconds.
tic;
A=ndSparse(S,[M,M,M,M]);
toc;%Elapsed time is 0.016531 seconds.
0 Comments
See Also
Categories
Find more on Data Type Conversion in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!