Practical Leverage-Based Sampling for Low-Rank Tensor Decomposition

Abstract

The low-rank canonical polyadic tensor decomposition is useful in data analysis and can be computed by solving a sequence of overdetermined least squares subproblems. Motivated by consideration of sparse tensors, we propose sketching each subproblem using leverage scores to select a subset of the rows, with probabilistic guarantees of the solution accuracy. We randomly sample rows proportional to leverage score upper bounds that can be efficiently computed using the special Khatri-Rao subproblem structure inherent in tensor decomposition. Crucially, the number of rows in the sketched system is independent of the number of nonzeros in the full tensor and the number of rows in the full system. Along the way, we provide a practical solution to the generic matrix sketching problem of sampling overabundance for high-leverage-score rows, proposing to include such rows deterministically and combine repeated samples in the sketched system; we conjecture that this can lead to improved theoretical bounds. Numerical results on real-world large-scale tensors show the method is significantly faster than deterministic methods without sacrificing accuracy.

@article{LaKo22,  
author = {Brett W. Larsen and Tamara G. Kolda}, 
title = {Practical Leverage-Based Sampling for Low-Rank Tensor Decomposition}, 
journal = {SIAM J. Matrix Analysis and Applications}, 
volume = {43}, 
number = {3}, 
pages = {1488--1517}, 
month = {August}, 
year = {2022},
doi = {10.1137/21M1441754},
eprint = {2006.16438},
}