Symmetry Breaking in Symmetric Tensor Decomposition
- URL: http://arxiv.org/abs/2103.06234v2
- Date: Thu, 28 Dec 2023 16:50:25 GMT
- Title: Symmetry Breaking in Symmetric Tensor Decomposition
- Authors: Yossi Arjevani, Joan Bruna, Michael Field, Joe Kileel, Matthew Trager,
Francis Williams
- Abstract summary: We consider the nonsymmetry problem associated with computing the points rank decomposition of symmetric tensors.
We show that critical points the loss function is detected by standard methods.
- Score: 44.181747424363245
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this note, we consider the highly nonconvex optimization problem
associated with computing the rank decomposition of symmetric tensors. We
formulate the invariance properties of the loss function and show that critical
points detected by standard gradient based methods are \emph{symmetry breaking}
with respect to the target tensor. The phenomena, seen for different choices of
target tensors and norms, make possible the use of recently developed analytic
and algebraic tools for studying nonconvex optimization landscapes exhibiting
symmetry breaking phenomena of similar nature.
Related papers
- Symmetry Breaking and Equivariant Neural Networks [17.740760773905986]
We introduce a novel notion of'relaxed equiinjection'
We show how to incorporate this relaxation into equivariant multilayer perceptronrons (E-MLPs)
The relevance of symmetry breaking is then discussed in various application domains.
arXiv Detail & Related papers (2023-12-14T15:06:48Z) - Learning Layer-wise Equivariances Automatically using Gradients [66.81218780702125]
Convolutions encode equivariance symmetries into neural networks leading to better generalisation performance.
symmetries provide fixed hard constraints on the functions a network can represent, need to be specified in advance, and can not be adapted.
Our goal is to allow flexible symmetry constraints that can automatically be learned from data using gradients.
arXiv Detail & Related papers (2023-10-09T20:22:43Z) - Symmetry Induces Structure and Constraint of Learning [0.0]
We unveil the importance of the loss function symmetries in affecting, if not deciding, the learning behavior of machine learning models.
Common instances of mirror symmetries in deep learning include rescaling, rotation, and permutation symmetry.
We show that the theoretical framework can explain intriguing phenomena, such as the loss of plasticity and various collapse phenomena in neural networks.
arXiv Detail & Related papers (2023-09-29T02:21:31Z) - Symmetry & Critical Points for Symmetric Tensor Decomposition Problems [6.650108973968032]
We consider the non optimization problem associated with the decomposition of a real symmetric tensor into a sum of rank one terms.
Use is made of a rich symmetry structure to construct infinite families of critical points represented by Puiseux series in the problem dimension.
Adesirable phenomenon, occurring for all critical points considered, concerns the number of negative Hessian eigenvalues increasing with the value of the objective function.
arXiv Detail & Related papers (2023-06-13T16:25:30Z) - Suppression of one-dimensional weak localization by band asymmetry [0.0]
We investigate disorder-induced localization in metals that break time-reversal and inversion symmetries through their energy dispersion.
In the perturbative regime of disorder, we show that weak localization is suppressed due to a mismatch of the Fermi velocities of left and right movers.
arXiv Detail & Related papers (2023-02-27T14:25:40Z) - The Surprising Effectiveness of Equivariant Models in Domains with
Latent Symmetry [6.716931832076628]
We show that imposing symmetry constraints that do not exactly match the domain symmetry is very helpful in learning the true symmetry in the environment.
We demonstrate that an equivariant model can significantly outperform non-equivariant methods on domains with latent symmetries both in supervised learning and in reinforcement learning for robotic manipulation and control problems.
arXiv Detail & Related papers (2022-11-16T21:51:55Z) - Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector
Problems [98.34292831923335]
Motivated by the problem of online correlation analysis, we propose the emphStochastic Scaled-Gradient Descent (SSD) algorithm.
We bring these ideas together in an application to online correlation analysis, deriving for the first time an optimal one-time-scale algorithm with an explicit rate of local convergence to normality.
arXiv Detail & Related papers (2021-12-29T18:46:52Z) - When Random Tensors meet Random Matrices [50.568841545067144]
This paper studies asymmetric order-$d$ spiked tensor models with Gaussian noise.
We show that the analysis of the considered model boils down to the analysis of an equivalent spiked symmetric textitblock-wise random matrix.
arXiv Detail & Related papers (2021-12-23T04:05:01Z) - On Convergence of Training Loss Without Reaching Stationary Points [62.41370821014218]
We show that Neural Network weight variables do not converge to stationary points where the gradient the loss function vanishes.
We propose a new perspective based on ergodic theory dynamical systems.
arXiv Detail & Related papers (2021-10-12T18:12:23Z) - Understanding Implicit Regularization in Over-Parameterized Single Index
Model [55.41685740015095]
We design regularization-free algorithms for the high-dimensional single index model.
We provide theoretical guarantees for the induced implicit regularization phenomenon.
arXiv Detail & Related papers (2020-07-16T13:27:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.