A Unified Framework to Enforce, Discover, and Promote Symmetry in
Machine Learning
- URL: http://arxiv.org/abs/2311.00212v1
- Date: Wed, 1 Nov 2023 01:19:54 GMT
- Title: A Unified Framework to Enforce, Discover, and Promote Symmetry in
Machine Learning
- Authors: Samuel E. Otto, Nicholas Zolman, J. Nathan Kutz, Steven L. Brunton
- Abstract summary: We provide a unifying theoretical and methodological framework for incorporating symmetry into machine learning models.
We show that enforcing and discovering symmetry are linear-algebraic tasks that are dual with respect to the bilinear structure of the Lie derivative.
We explain how these ideas can be applied to a wide range of machine learning models including basis function regression, dynamical systems discovery, multilayer perceptrons, and neural networks acting on spatial fields such as images.
- Score: 5.582881461692378
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Symmetry is present throughout nature and continues to play an increasingly
central role in physics and machine learning. Fundamental symmetries, such as
Poincar\'{e} invariance, allow physical laws discovered in laboratories on
Earth to be extrapolated to the farthest reaches of the universe. Symmetry is
essential to achieving this extrapolatory power in machine learning
applications. For example, translation invariance in image classification
allows models with fewer parameters, such as convolutional neural networks, to
be trained on smaller data sets and achieve state-of-the-art performance. In
this paper, we provide a unifying theoretical and methodological framework for
incorporating symmetry into machine learning models in three ways: 1. enforcing
known symmetry when training a model; 2. discovering unknown symmetries of a
given model or data set; and 3. promoting symmetry during training by learning
a model that breaks symmetries within a user-specified group of candidates when
there is sufficient evidence in the data. We show that these tasks can be cast
within a common mathematical framework whose central object is the Lie
derivative associated with fiber-linear Lie group actions on vector bundles. We
extend and unify several existing results by showing that enforcing and
discovering symmetry are linear-algebraic tasks that are dual with respect to
the bilinear structure of the Lie derivative. We also propose a novel way to
promote symmetry by introducing a class of convex regularization functions
based on the Lie derivative and nuclear norm relaxation to penalize symmetry
breaking during training of machine learning models. We explain how these ideas
can be applied to a wide range of machine learning models including basis
function regression, dynamical systems discovery, multilayer perceptrons, and
neural networks acting on spatial fields such as images.
Related papers
- Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs)
Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators.
Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
arXiv Detail & Related papers (2024-05-22T17:23:15Z) - Exploiting Symmetry in Dynamics for Model-Based Reinforcement Learning with Asymmetric Rewards [0.6612847014373572]
We introduce a technique for learning dynamics which, by construction, exhibit specified symmetries.
We demonstrate through numerical experiments that the proposed method learns a more accurate dynamical model.
arXiv Detail & Related papers (2024-03-27T21:31:46Z) - A Generative Model of Symmetry Transformations [44.87295754993983]
We build a generative model that explicitly aims to capture the data's approximate symmetries.
We empirically demonstrate its ability to capture symmetries under affine and color transformations.
arXiv Detail & Related papers (2024-03-04T11:32:18Z) - ${\rm E}(3)$-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning [7.712824077083934]
We focus on exploiting Euclidean symmetries inherent in certain cooperative multi-agent reinforcement learning problems.
We design neural network architectures with symmetric constraints embedded as an inductive bias for multi-agent actor-critic methods.
arXiv Detail & Related papers (2023-08-23T00:18:17Z) - Geometric Neural Diffusion Processes [55.891428654434634]
We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling.
We show that with these conditions, the generative functional model admits the same symmetry.
arXiv Detail & Related papers (2023-07-11T16:51:38Z) - FAENet: Frame Averaging Equivariant GNN for Materials Modeling [123.19473575281357]
We introduce a flexible framework relying on frameaveraging (SFA) to make any model E(3)-equivariant or invariant through data transformations.
We prove the validity of our method theoretically and empirically demonstrate its superior accuracy and computational scalability in materials modeling.
arXiv Detail & Related papers (2023-04-28T21:48:31Z) - Capturing dynamical correlations using implicit neural representations [85.66456606776552]
We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data.
In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
arXiv Detail & Related papers (2023-04-08T07:55:36Z) - Generative Adversarial Symmetry Discovery [19.098785309131458]
LieGAN represents symmetry as interpretable Lie algebra basis and can discover various symmetries.
The learned symmetry can also be readily used in several existing equivariant neural networks to improve accuracy and generalization in prediction.
arXiv Detail & Related papers (2023-02-01T04:28:36Z) - Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras
from First Principles [55.41644538483948]
We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset.
We use fully connected neural networks to model the transformations symmetry and the corresponding generators.
Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.
arXiv Detail & Related papers (2023-01-13T16:25:25Z) - Learning Physical Dynamics with Subequivariant Graph Neural Networks [99.41677381754678]
Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics.
Physical laws abide by symmetry, which is a vital inductive bias accounting for model generalization.
Our model achieves on average over 3% enhancement in contact prediction accuracy across 8 scenarios on Physion and 2X lower rollout MSE on RigidFall.
arXiv Detail & Related papers (2022-10-13T10:00:30Z) - On the Symmetries of Deep Learning Models and their Internal
Representations [1.418465438044804]
We seek to connect the symmetries arising from the architecture of a family of models with the symmetries of that family's internal representation of data.
Our work suggests that the symmetries of a network are propagated into the symmetries in that network's representation of data.
arXiv Detail & Related papers (2022-05-27T22:29:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.