Symmetry & Critical Points for Symmetric Tensor Decomposition Problems

• URL: http://arxiv.org/abs/2306.07886v3
• Date: Mon, 7 Aug 2023 10:01:49 GMT
• Title: Symmetry & Critical Points for Symmetric Tensor Decomposition Problems
• Authors: Yossi Arjevani, Gal Vinograd
• Abstract summary: We consider the non optimization problem associated with the decomposition of a real symmetric tensor into a sum of rank one terms. Use is made of a rich symmetry structure to construct infinite families of critical points represented by Puiseux series in the problem dimension. Adesirable phenomenon, occurring for all critical points considered, concerns the number of negative Hessian eigenvalues increasing with the value of the objective function.
• Score: 6.650108973968032
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the nonconvex optimization problem associated with the decomposition of a real symmetric tensor into a sum of rank one terms. Use is made of the rich symmetry structure to construct infinite families of critical points represented by Puiseux series in the problem dimension, and so obtain precise analytic estimates on the value of the objective function and the Hessian spectrum. The results allow an analytic characterization of various obstructions to using local optimization methods, revealing in particular a complex array of saddles and minima differing by their symmetry, structure and analytic properties. A~desirable phenomenon, occurring for all critical points considered, concerns the number of negative Hessian eigenvalues increasing with the value of the objective function. Our approach makes use of Newton polyhedra as well as results from real algebraic geometry, notably the Curve Selection Lemma, to determine the extremal character of degenerate critical points, establishing in particular the existence of infinite families of third-order saddles which can significantly slow down the optimization process.

Related papers

• Topological complexity of spiked random polynomials and finite-rank spherical integrals [2.1756081703276]
  In particular, we establish variational formulas for the exponentials of the average number of total critical points and the determinants of local parameters of a finite-rank spiked Gaussian Wigner matrix. The analysis is based on recent advances on finite-rank spherical integrals by [Guionnet, Husson] to study the large deviations of multi-rank spiked Gaussian Wigner matrices. There is an exact threshold for the external parameters such that, once exceeded, the complexity function vanishes into new regions in which the critical points are close to the given vectors.
  arXiv  Detail & Related papers  (2023-12-19T16:52:01Z)
• The Tempered Hilbert Simplex Distance and Its Application To Non-linear Embeddings of TEMs [36.135201624191026]
  We introduce three different parameterizations of finite discrete TEMs via Legendre functions of the negative tempered entropy function. Similar to the Hilbert geometry, the tempered Hilbert distance is characterized as a $t$-symmetrization of the oriented tempered Funk distance.
  arXiv  Detail & Related papers  (2023-11-22T15:24:29Z)
• Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
  This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold. It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
  arXiv  Detail & Related papers  (2023-11-08T15:17:13Z)
• Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds [77.4346324549323]
  We show that a step size agnostic to the curvature of the manifold achieves a curvature-independent and linear last-iterate convergence rate. To the best of our knowledge, the possibility of curvature-independent rates and/or last-iterate convergence has not been considered before.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems [98.34292831923335]
  Motivated by the problem of online correlation analysis, we propose the emphStochastic Scaled-Gradient Descent (SSD) algorithm. We bring these ideas together in an application to online correlation analysis, deriving for the first time an optimal one-time-scale algorithm with an explicit rate of local convergence to normality.
  arXiv  Detail & Related papers  (2021-12-29T18:46:52Z)
• Spectral clustering under degree heterogeneity: a case for the random walk Laplacian [83.79286663107845]
  This paper shows that graph spectral embedding using the random walk Laplacian produces vector representations which are completely corrected for node degree. In the special case of a degree-corrected block model, the embedding concentrates about K distinct points, representing communities.
  arXiv  Detail & Related papers  (2021-05-03T16:36:27Z)
• Symmetry Breaking in Symmetric Tensor Decomposition [44.181747424363245]
  We consider the nonsymmetry problem associated with computing the points rank decomposition of symmetric tensors. We show that critical points the loss function is detected by standard methods.
  arXiv  Detail & Related papers  (2021-03-10T18:11:22Z)
• Classical symmetries and QAOA [3.2880869992413246]
  We study the relationship between the Quantum Approximate Optimization Algorithm (QAOA) and the underlying symmetries of the objective function to be optimized. Our approach formalizes the connection between quantum symmetry properties of the QAOA dynamics and the group of classical symmetries of the objective function.
  arXiv  Detail & Related papers  (2020-12-08T20:02:09Z)
• Nonconvex Matrix Completion with Linearly Parameterized Factors [10.163102766021373]
  Parametric Factorization holds for important examples including subspace and completion simulations. The effectiveness of our unified nonconstrained matrix optimization method is also illustrated.
  arXiv  Detail & Related papers  (2020-03-29T22:40:47Z)
• Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees [106.91654068632882]
  We consider the bipartite graph and formalize its representation learning problem as a statistical estimation problem of parameters in a semiparametric exponential family distribution. We show that the proposed objective is strongly convex in a neighborhood around the ground truth, so that a gradient descent-based method achieves linear convergence rate. Our estimator is robust to any model misspecification within the exponential family, which is validated in extensive experiments.
  arXiv  Detail & Related papers  (2020-03-02T16:40:36Z)
• Convex Geometry and Duality of Over-parameterized Neural Networks [70.15611146583068]
  We develop a convex analytic approach to analyze finite width two-layer ReLU networks. We show that an optimal solution to the regularized training problem can be characterized as extreme points of a convex set. In higher dimensions, we show that the training problem can be cast as a finite dimensional convex problem with infinitely many constraints.
  arXiv  Detail & Related papers  (2020-02-25T23:05:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.