Minimizing low-rank models of high-order tensors: Hardness, span, tight relaxation, and applications

• URL: http://arxiv.org/abs/2210.11413v3
• Date: Thu, 21 Dec 2023 19:41:37 GMT
• Title: Minimizing low-rank models of high-order tensors: Hardness, span, tight relaxation, and applications
• Authors: Nicholas D. Sidiropoulos, Paris Karakasis, and Aritra Konar
• Abstract summary: We show that this fundamental tensor problem is NP-hard for any tensor rank higher than one. For low-enough ranks, the proposed continuous reformulation is amenable to low-complexity gradient-based optimization. We demonstrate promising results on a number of hard problems, including low density parity check codes and general decoding parity check codes.
• Score: 26.602371644194143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of finding the smallest or largest entry of a tensor of order N that is specified via its rank decomposition. Stated in a different way, we are given N sets of R-dimensional vectors and we wish to select one vector from each set such that the sum of the Hadamard product of the selected vectors is minimized or maximized. We show that this fundamental tensor problem is NP-hard for any tensor rank higher than one, and polynomial-time solvable in the rank-one case. We also propose a continuous relaxation and prove that it is tight for any rank. For low-enough ranks, the proposed continuous reformulation is amenable to low-complexity gradient-based optimization, and we propose a suite of gradient-based optimization algorithms drawing from projected gradient descent, Frank-Wolfe, or explicit parametrization of the relaxed constraints. We also show that our core results remain valid no matter what kind of polyadic tensor model is used to represent the tensor of interest, including Tucker, HOSVD/MLSVD, tensor train, or tensor ring. Next, we consider the class of problems that can be posed as special instances of the problem of interest. We show that this class includes the partition problem (and thus all NP-complete problems via polynomial-time transformation), integer least squares, integer linear programming, integer quadratic programming, sign retrieval (a special kind of mixed integer programming / restricted version of phase retrieval), and maximum likelihood decoding of parity check codes. We demonstrate promising experimental results on a number of hard problems, including state-of-art performance in decoding low density parity check codes and general parity check codes.

Related papers

• Tensor cumulants for statistical inference on invariant distributions [49.80012009682584]
  We show that PCA becomes computationally hard at a critical value of the signal's magnitude. We define a new set of objects, which provide an explicit, near-orthogonal basis for invariants of a given degree. It also lets us analyze a new problem of distinguishing between different ensembles.
  arXiv  Detail & Related papers  (2024-04-29T14:33:24Z)
• Decomposition of linear tensor transformations [0.0]
  The aim of this paper is to develop a mathematical framework for exact tensor decomposition. In the paper three different problems will be carried out to derive.
  arXiv  Detail & Related papers  (2023-09-14T16:14:38Z)
• Distributed Extra-gradient with Optimal Complexity and Communication Guarantees [60.571030754252824]
  We consider monotone variational inequality (VI) problems in multi-GPU settings where multiple processors/workers/clients have access to local dual vectors. Extra-gradient, which is a de facto algorithm for monotone VI problems, has not been designed to be communication-efficient. We propose a quantized generalized extra-gradient (Q-GenX), which is an unbiased and adaptive compression method tailored to solve VIs.
  arXiv  Detail & Related papers  (2023-08-17T21:15:04Z)
• Efficiently Learning One-Hidden-Layer ReLU Networks via Schur Polynomials [50.90125395570797]
  We study the problem of PAC learning a linear combination of $k$ ReLU activations under the standard Gaussian distribution on $mathbbRd$ with respect to the square loss. Our main result is an efficient algorithm for this learning task with sample and computational complexity $(dk/epsilon)O(k)$, whereepsilon>0$ is the target accuracy.
  arXiv  Detail & Related papers  (2023-07-24T14:37:22Z)
• Nonnegative Low-Rank Tensor Completion via Dual Formulation with Applications to Image and Video Completion [2.1227526213206542]
  Recent approaches to the tensor completion problem have often overlooked the nonnegative structure of the data. We consider the problem of learning a nonnegative low-rank tensor, and using duality theory, we propose a novel factorization of such tensors. We test the proposed algorithm across various tasks such as colour image inpainting, video completion, and hyperspectral image completion.
  arXiv  Detail & Related papers  (2023-05-13T17:51:00Z)
• Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomials [93.59919600451487]
  "Statistical-computational gaps" occur in many statistical inference tasks. We consider a model for random order-3 decomposition where one component is slightly larger in norm than the rest. We show that tensor entries can accurately estimate the largest component when $ll n3/2$ but fail to do so when $rgg n3/2$.
  arXiv  Detail & Related papers  (2022-11-10T00:40:37Z)
• Smooth tensor estimation with unknown permutations [14.391648046717073]
  We develop a family of smooth tensor models up to arbitrary index permutations. We show that a constrained least-squares estimator in the block-wise family achieves the minimax error bound. We also provide an efficient-time Borda count algorithm that achieves provably optimal rate.
  arXiv  Detail & Related papers  (2021-11-08T17:53:48Z)
• Nonnegative Tensor Completion via Integer Optimization [5.932152752097064]
  We prove that our algorithm converges in a linear (in numerical tolerance) number of oracle steps, while achieving the information-theoretic rate. Because the norm is defined using a 0-1 polytope, this means we can use integer linear programming to solve linear separation problems over the polytope.
  arXiv  Detail & Related papers  (2021-11-08T15:43:19Z)
• Gradient Free Minimax Optimization: Variance Reduction and Faster Convergence [120.9336529957224]
  In this paper, we denote the non-strongly setting on the magnitude of a gradient-free minimax optimization problem. We show that a novel zeroth-order variance reduced descent algorithm achieves the best known query complexity.
  arXiv  Detail & Related papers  (2020-06-16T17:55:46Z)
• Optimal Randomized First-Order Methods for Least-Squares Problems [56.05635751529922]
  This class of algorithms encompasses several randomized methods among the fastest solvers for least-squares problems. We focus on two classical embeddings, namely, Gaussian projections and subsampled Hadamard transforms. Our resulting algorithm yields the best complexity known for solving least-squares problems with no condition number dependence.
  arXiv  Detail & Related papers  (2020-02-21T17:45:32Z)
• Grassmannian Optimization for Online Tensor Completion and Tracking with the t-SVD [10.137631021498109]
  We show the t-SVD is a specialization of the well-studied block-term decomposition for third-order tensors. We present an algorithm under this model that can track changing free submodules from incomplete streaming 2-D data. Our results are competitive in accuracy but much faster in compute time than state-of-the-art tensor completion algorithms on real applications.
  arXiv  Detail & Related papers  (2020-01-30T15:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.