A Generative Model of Symmetry Transformations
- URL: http://arxiv.org/abs/2403.01946v2
- Date: Thu, 20 Jun 2024 21:56:54 GMT
- Title: A Generative Model of Symmetry Transformations
- Authors: James Urquhart Allingham, Bruno Kacper Mlodozeniec, Shreyas Padhy, Javier Antorán, David Krueger, Richard E. Turner, Eric Nalisnick, José Miguel Hernández-Lobato,
- Abstract summary: We build a generative model that explicitly aims to capture the data's approximate symmetries.
We empirically demonstrate its ability to capture symmetries under affine and color transformations.
- Score: 44.87295754993983
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Correctly capturing the symmetry transformations of data can lead to efficient models with strong generalization capabilities, though methods incorporating symmetries often require prior knowledge. While recent advancements have been made in learning those symmetries directly from the dataset, most of this work has focused on the discriminative setting. In this paper, we take inspiration from group theoretic ideas to construct a generative model that explicitly aims to capture the data's approximate symmetries. This results in a model that, given a prespecified broad set of possible symmetries, learns to what extent, if at all, those symmetries are actually present. Our model can be seen as a generative process for data augmentation. We provide a simple algorithm for learning our generative model and empirically demonstrate its ability to capture symmetries under affine and color transformations, in an interpretable way. Combining our symmetry model with standard generative models results in higher marginal test-log-likelihoods and improved data efficiency.
Related papers
- Learning Infinitesimal Generators of Continuous Symmetries from Data [15.42275880523356]
We propose a novel symmetry learning algorithm based on transformations defined with one- parameter groups.
Our method is built upon minimal inductive biases, encompassing not only commonly utilized symmetries rooted in Lie groups but also extending to symmetries derived from nonlinear generators.
arXiv Detail & Related papers (2024-10-29T08:28:23Z) - SymmetryLens: A new candidate paradigm for unsupervised symmetry learning via locality and equivariance [0.0]
We develop a new, unsupervised symmetry learning method that starts with raw data.
We demonstrate that this coupling between symmetry and locality, together with a special optimization technique developed for entropy estimation, results in a highly stable system.
The symmetry actions we consider are group representations, however, we believe the approach has the potential to be generalized to more general, nonlinear actions of non-commutative Lie groups.
arXiv Detail & Related papers (2024-10-07T17:40:51Z) - Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation.
In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model.
We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
arXiv Detail & Related papers (2024-04-11T09:23:36Z) - Geometric Neural Diffusion Processes [55.891428654434634]
We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling.
We show that with these conditions, the generative functional model admits the same symmetry.
arXiv Detail & Related papers (2023-07-11T16:51:38Z) - Regularizing Towards Soft Equivariance Under Mixed Symmetries [23.603875905608565]
We present a regularizer-based method for building a model for a dataset with mixed approximate symmetries.
We show that our method achieves better accuracy than prior approaches while discovering the approximate symmetry levels correctly.
arXiv Detail & Related papers (2023-06-01T05:33:41Z) - Approximation-Generalization Trade-offs under (Approximate) Group
Equivariance [3.0458514384586395]
Group equivariant neural networks have demonstrated impressive performance across various domains and applications such as protein and drug design.
We show how models capturing task-specific symmetries lead to improved generalization.
We examine the more general question of model mis-specification when the model symmetries don't align with the data symmetries.
arXiv Detail & Related papers (2023-05-27T22:53:37Z) - FAENet: Frame Averaging Equivariant GNN for Materials Modeling [123.19473575281357]
We introduce a flexible framework relying on frameaveraging (SFA) to make any model E(3)-equivariant or invariant through data transformations.
We prove the validity of our method theoretically and empirically demonstrate its superior accuracy and computational scalability in materials modeling.
arXiv Detail & Related papers (2023-04-28T21:48:31Z) - Oracle-Preserving Latent Flows [58.720142291102135]
We develop a methodology for the simultaneous discovery of multiple nontrivial continuous symmetries across an entire labelled dataset.
The symmetry transformations and the corresponding generators are modeled with fully connected neural networks trained with a specially constructed loss function.
The two new elements in this work are the use of a reduced-dimensionality latent space and the generalization to transformations invariant with respect to high-dimensional oracles.
arXiv Detail & Related papers (2023-02-02T00:13:32Z) - Autoregressive Asymmetric Linear Gaussian Hidden Markov Models [1.332091725929965]
Asymmetric hidden Markov models provide a framework where the trend of the process can be expressed as a latent variable.
We show how inference, hidden states decoding and parameter learning must be adapted to fit the proposed model.
arXiv Detail & Related papers (2020-10-27T08:58:46Z) - Inverse Learning of Symmetries [71.62109774068064]
We learn the symmetry transformation with a model consisting of two latent subspaces.
Our approach is based on the deep information bottleneck in combination with a continuous mutual information regulariser.
Our model outperforms state-of-the-art methods on artificial and molecular datasets.
arXiv Detail & Related papers (2020-02-07T13:48:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.