Regularizing Towards Soft Equivariance Under Mixed Symmetries
- URL: http://arxiv.org/abs/2306.00356v1
- Date: Thu, 1 Jun 2023 05:33:41 GMT
- Title: Regularizing Towards Soft Equivariance Under Mixed Symmetries
- Authors: Hyunsu Kim, Hyungi Lee, Hongseok Yang, and Juho Lee
- Abstract summary: We present a regularizer-based method for building a model for a dataset with mixed approximate symmetries.
We show that our method achieves better accuracy than prior approaches while discovering the approximate symmetry levels correctly.
- Score: 23.603875905608565
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Datasets often have their intrinsic symmetries, and particular deep-learning
models called equivariant or invariant models have been developed to exploit
these symmetries. However, if some or all of these symmetries are only
approximate, which frequently happens in practice, these models may be
suboptimal due to the architectural restrictions imposed on them. We tackle
this issue of approximate symmetries in a setup where symmetries are mixed,
i.e., they are symmetries of not single but multiple different types and the
degree of approximation varies across these types. Instead of proposing a new
architectural restriction as in most of the previous approaches, we present a
regularizer-based method for building a model for a dataset with mixed
approximate symmetries. The key component of our method is what we call
equivariance regularizer for a given type of symmetries, which measures how
much a model is equivariant with respect to the symmetries of the type. Our
method is trained with these regularizers, one per each symmetry type, and the
strength of the regularizers is automatically tuned during training, leading to
the discovery of the approximation levels of some candidate symmetry types
without explicit supervision. Using synthetic function approximation and motion
forecasting tasks, we demonstrate that our method achieves better accuracy than
prior approaches while discovering the approximate symmetry levels correctly.
Related papers
- A Generative Model of Symmetry Transformations [44.87295754993983]
We build a generative model that explicitly aims to capture the data's approximate symmetries.
We empirically demonstrate its ability to capture symmetries under affine and color transformations.
arXiv Detail & Related papers (2024-03-04T11:32:18Z) - Self-Supervised Detection of Perfect and Partial Input-Dependent Symmetries [11.54837584979607]
Group equivariance can overly constrain models if the symmetries in the group differ from those observed in data.
We propose a method able to detect the level of symmetry of each input without the need for labels.
Our framework is general enough to accommodate different families of both continuous and discrete symmetry distributions.
arXiv Detail & Related papers (2023-12-19T15:11:46Z) - Asymmetric Norms to Approximate the Minimum Action Distance [9.040428950629153]
This paper presents a state representation for reward-free Markov decision processes.
We show how this representation can be leveraged to learn goal-conditioned policies.
arXiv Detail & Related papers (2023-12-16T00:50:17Z) - Symmetry Breaking and Equivariant Neural Networks [17.740760773905986]
We introduce a novel notion of'relaxed equiinjection'
We show how to incorporate this relaxation into equivariant multilayer perceptronrons (E-MLPs)
The relevance of symmetry breaking is then discussed in various application domains.
arXiv Detail & Related papers (2023-12-14T15:06:48Z) - Learning Layer-wise Equivariances Automatically using Gradients [66.81218780702125]
Convolutions encode equivariance symmetries into neural networks leading to better generalisation performance.
symmetries provide fixed hard constraints on the functions a network can represent, need to be specified in advance, and can not be adapted.
Our goal is to allow flexible symmetry constraints that can automatically be learned from data using gradients.
arXiv Detail & Related papers (2023-10-09T20:22:43Z) - Oracle-Preserving Latent Flows [58.720142291102135]
We develop a methodology for the simultaneous discovery of multiple nontrivial continuous symmetries across an entire labelled dataset.
The symmetry transformations and the corresponding generators are modeled with fully connected neural networks trained with a specially constructed loss function.
The two new elements in this work are the use of a reduced-dimensionality latent space and the generalization to transformations invariant with respect to high-dimensional oracles.
arXiv Detail & Related papers (2023-02-02T00:13:32Z) - Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras
from First Principles [55.41644538483948]
We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset.
We use fully connected neural networks to model the transformations symmetry and the corresponding generators.
Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.
arXiv Detail & Related papers (2023-01-13T16:25:25Z) - The Surprising Effectiveness of Equivariant Models in Domains with
Latent Symmetry [6.716931832076628]
We show that imposing symmetry constraints that do not exactly match the domain symmetry is very helpful in learning the true symmetry in the environment.
We demonstrate that an equivariant model can significantly outperform non-equivariant methods on domains with latent symmetries both in supervised learning and in reinforcement learning for robotic manipulation and control problems.
arXiv Detail & Related papers (2022-11-16T21:51:55Z) - Meta-Learning Symmetries by Reparameterization [63.85144439337671]
We present a method for learning and encoding equivariances into networks by learning corresponding parameter sharing patterns from data.
Our experiments suggest that it can automatically learn to encode equivariances to common transformations used in image processing tasks.
arXiv Detail & Related papers (2020-07-06T17:59:54Z) - Inverse Learning of Symmetries [71.62109774068064]
We learn the symmetry transformation with a model consisting of two latent subspaces.
Our approach is based on the deep information bottleneck in combination with a continuous mutual information regulariser.
Our model outperforms state-of-the-art methods on artificial and molecular datasets.
arXiv Detail & Related papers (2020-02-07T13:48:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.