Alternating least squares for Tucker model
The function tucker_als computes the best rank(R1,R2,..,Rn) approximation of tensor X, according to the specified dimensions in vector R. The input X can be a tensor, sptensor, ktensor, or ttensor. The result returned in T is a ttensor.
The method is originally from Tucker (1966) and later revisited in De Lathauwer et al. (2000).
- Tucker, L. R. Some mathematical notes on three-mode factor analysis. Psychometrika, 1966, 31, 279-311.
- De Lathauwer, L.; De Moor, B. & Vandewalle, J. On the best rank-1 and rank-(R_1, R_2, R_N) approximation of higher-order tensors. SIAM Journal on Matrix Analysis and Applications, 2000, 21, 1324-1342.
Note: Oftentimes it's better to use hosvd instead.
Contents
Create a data tensor of size [5 4 3]
rng('default'); rng(0); %<-- Set seed for reproducibility X = sptenrand([5 4 3], 10)
X is a sparse tensor of size 5 x 4 x 3 with 10 nonzeros (1,2,3) 0.0759 (1,3,2) 0.0540 (2,2,2) 0.5308 (2,2,3) 0.7792 (3,1,3) 0.9340 (3,4,2) 0.1299 (4,1,2) 0.5688 (4,4,2) 0.4694 (5,2,1) 0.0119 (5,4,3) 0.3371
Create a [2 2 2] approximation
T = tucker_als(X,2) %<-- best rank(2,2,2) approximation
Tucker Alternating Least-Squares: Iter 1: fit = 3.266855e-01 fitdelta = 3.3e-01 Iter 2: fit = 4.285677e-01 fitdelta = 1.0e-01 Iter 3: fit = 4.707375e-01 fitdelta = 4.2e-02 Iter 4: fit = 4.728036e-01 fitdelta = 2.1e-03 Iter 5: fit = 4.728492e-01 fitdelta = 4.6e-05 T is a ttensor of size 5 x 4 x 3 T.core is a tensor of size 2 x 2 x 2 T.core(:,:,1) = 0.9045 0.0007 -0.0007 0.8920 T.core(:,:,2) = 0.2732 0.0006 0.0006 -0.2771 T.U{1} = 0.0666 0.0001 0.9978 0.0008 -0.0008 1.0000 -0.0001 0.0007 -0.0001 0.0018 T.U{2} = -0.0015 1.0000 1.0000 0.0015 0.0021 0.0000 -0.0001 0.0007 T.U{3} = -0.0000 -0.0000 0.2971 0.9548 0.9548 -0.2971
Create a [2 2 1] approximation
T = tucker_als(X,[2 2 1]) %<-- best rank(2,2,1) approximation
Tucker Alternating Least-Squares: Iter 1: fit = 2.363442e-01 fitdelta = 2.4e-01 Iter 2: fit = 3.907381e-01 fitdelta = 1.5e-01 Iter 3: fit = 4.304797e-01 fitdelta = 4.0e-02 Iter 4: fit = 4.328533e-01 fitdelta = 2.4e-03 Iter 5: fit = 4.331455e-01 fitdelta = 2.9e-04 Iter 6: fit = 4.331975e-01 fitdelta = 5.2e-05 T is a ttensor of size 5 x 4 x 3 T.core is a tensor of size 2 x 2 x 1 T.core(:,:,1) = 0.9283 0.0000 -0.0000 0.8930 T.U{1} = 0.0753 -0.0000 0.9972 -0.0000 0.0000 0.9613 0.0001 0.2723 0.0001 0.0414 T.U{2} = -0.0000 0.9921 1.0000 -0.0000 0.0017 -0.0000 0.0001 0.1252 T.U{3} = 0.0000 0.3959 0.9183
Use a different ordering of the dimensions
T = tucker_als(X,2,struct('dimorder',[3 2 1]))
Tucker Alternating Least-Squares: Iter 1: fit = 3.954350e-01 fitdelta = 4.0e-01 Iter 2: fit = 4.650831e-01 fitdelta = 7.0e-02 Iter 3: fit = 4.724949e-01 fitdelta = 7.4e-03 Iter 4: fit = 4.728343e-01 fitdelta = 3.4e-04 Iter 5: fit = 4.728495e-01 fitdelta = 1.5e-05 T is a ttensor of size 5 x 4 x 3 T.core is a tensor of size 2 x 2 x 2 T.core(:,:,1) = 0.9036 -0.0394 0.0389 0.8910 T.core(:,:,2) = 0.2730 -0.0146 -0.0149 -0.2769 T.U{1} = 0.0665 -0.0004 0.9978 -0.0055 0.0055 1.0000 0.0007 0.0000 0.0005 0.0000 T.U{2} = 0.0491 0.9988 0.9988 -0.0491 0.0021 -0.0001 0.0016 0.0000 T.U{3} = 0.0000 0.0000 0.2970 0.9549 0.9549 -0.2970
Use the n-vecs initialization method
This initialization is more expensive but generally works very well.
T = tucker_als(X,2,struct('dimorder',[3 2 1],'init','eigs'))
Computing 2 leading e-vectors for factor 2. Computing 2 leading e-vectors for factor 1. Tucker Alternating Least-Squares: Iter 1: fit = 4.726805e-01 fitdelta = 4.7e-01 Iter 2: fit = 4.728466e-01 fitdelta = 1.7e-04 Iter 3: fit = 4.728501e-01 fitdelta = 3.5e-06 T is a ttensor of size 5 x 4 x 3 T.core is a tensor of size 2 x 2 x 2 T.core(:,:,1) = 0.9045 0.0000 -0.0000 0.8918 T.core(:,:,2) = 0.2731 0.0000 0.0000 -0.2775 T.U{1} = 0.0666 -0.0000 0.9978 -0.0000 0.0000 1.0000 0.0000 0.0001 -0.0000 0.0002 T.U{2} = -0.0000 1.0000 1.0000 0.0000 0.0021 -0.0000 0.0000 0.0005 T.U{3} = 0.0000 0.0000 0.2973 0.9548 0.9548 -0.2973
Specify the initial guess manually
U0 = {rand(5,2),rand(4,2),[]}; %<-- Initial guess for factors of T T = tucker_als(X,2,struct('dimorder',[3 2 1],'init',{U0}))
Tucker Alternating Least-Squares: Iter 1: fit = 3.733166e-01 fitdelta = 3.7e-01 Iter 2: fit = 4.397339e-01 fitdelta = 6.6e-02 Iter 3: fit = 4.717403e-01 fitdelta = 3.2e-02 Iter 4: fit = 4.728257e-01 fitdelta = 1.1e-03 Iter 5: fit = 4.728497e-01 fitdelta = 2.4e-05 T is a ttensor of size 5 x 4 x 3 T.core is a tensor of size 2 x 2 x 2 T.core(:,:,1) = 0.9047 0.0100 -0.0099 0.8916 T.core(:,:,2) = 0.2725 0.0037 0.0038 -0.2779 T.U{1} = 0.0666 0.0001 0.9978 0.0014 -0.0014 1.0000 -0.0002 0.0002 -0.0001 0.0005 T.U{2} = -0.0125 0.9999 0.9999 0.0125 0.0021 0.0000 -0.0004 0.0013 T.U{3} = -0.0000 -0.0000 0.2978 0.9546 0.9546 -0.2978