Equivariant Manifold Neural ODEs and Differential Invariants
- URL: http://arxiv.org/abs/2401.14131v1
- Date: Thu, 25 Jan 2024 12:23:22 GMT
- Title: Equivariant Manifold Neural ODEs and Differential Invariants
- Authors: Emma Andersdotter, Fredrik Ohlsson
- Abstract summary: We develop a manifestly geometric framework for equivariant manifold neural ordinary differential equations (NODEs)
We use it to analyse their modelling capabilities for symmetric data.
- Score: 0.7770029179741429
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper we develop a manifestly geometric framework for equivariant
manifold neural ordinary differential equations (NODEs), and use it to analyse
their modelling capabilities for symmetric data. First, we consider the action
of a Lie group $G$ on a smooth manifold $M$ and establish the equivalence
between equivariance of vector fields, symmetries of the corresponding Cauchy
problems, and equivariance of the associated NODEs. We also propose a novel
formulation of the equivariant NODEs in terms of the differential invariants of
the action of $G$ on $M$, based on Lie theory for symmetries of differential
equations, which provides an efficient parameterisation of the space of
equivariant vector fields in a way that is agnostic to both the manifold $M$
and the symmetry group $G$. Second, we construct augmented manifold NODEs,
through embeddings into equivariant flows, and show that they are universal
approximators of equivariant diffeomorphisms on any path-connected $M$.
Furthermore, we show that the augmented NODEs can be incorporated in the
geometric framework and parameterised using higher order differential
invariants. Finally, we consider the induced action of $G$ on different fields
on $M$ and show how it can be used to generalise previous work, on, e.g.,
continuous normalizing flows, to equivariant models in any geometry.
Related papers
- Geometric Generative Models based on Morphological Equivariant PDEs and GANs [3.6498648388765513]
We propose a geometric generative model based on an equivariant partial differential equation (PDE) for group convolution neural networks (G-CNNs)
The proposed geometric morphological GAN (GM-GAN) is obtained by using the proposed morphological equivariant convolutions in PDE-G-CNNs.
Preliminary results show that GM-GAN model outperforms classical GAN.
arXiv Detail & Related papers (2024-03-22T01:02:09Z) - Geometry of Linear Neural Networks: Equivariance and Invariance under
Permutation Groups [0.0]
We investigate the subvariety of functions that are equivariant or invariant under the action of a permutation group.
We draw conclusions for the parameterization and the design of equivariant and invariant linear networks.
arXiv Detail & Related papers (2023-09-24T19:40:15Z) - Geometric Neural Diffusion Processes [55.891428654434634]
We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling.
We show that with these conditions, the generative functional model admits the same symmetry.
arXiv Detail & Related papers (2023-07-11T16:51:38Z) - Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras
from First Principles [55.41644538483948]
We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset.
We use fully connected neural networks to model the transformations symmetry and the corresponding generators.
Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.
arXiv Detail & Related papers (2023-01-13T16:25:25Z) - Equivalence Between SE(3) Equivariant Networks via Steerable Kernels and
Group Convolution [90.67482899242093]
A wide range of techniques have been proposed in recent years for designing neural networks for 3D data that are equivariant under rotation and translation of the input.
We provide an in-depth analysis of both methods and their equivalence and relate the two constructions to multiview convolutional networks.
We also derive new TFN non-linearities from our equivalence principle and test them on practical benchmark datasets.
arXiv Detail & Related papers (2022-11-29T03:42:11Z) - Measuring dissimilarity with diffeomorphism invariance [94.02751799024684]
We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces.
We prove that DID enjoys properties which make it relevant for theoretical study and practical use.
arXiv Detail & Related papers (2022-02-11T13:51:30Z) - Equivariant Discrete Normalizing Flows [10.867162810786361]
We focus on building equivariant normalizing flows using discrete layers.
We introduce two new equivariant flows: $G$-coupling Flows and $G$-Residual Flows.
Our construction of $G$-Residual Flows are also universal, in the sense that we prove an $G$-equivariant diffeomorphism can be exactly mapped by a $G$-residual flow.
arXiv Detail & Related papers (2021-10-16T20:16:00Z) - Equivariant bifurcation, quadratic equivariants, and symmetry breaking
for the standard representation of $S_n$ [15.711517003382484]
Motivated by questions originating from the study of a class of shallow student-teacher neural networks, methods are developed for the analysis of spurious minima in classes of equivariant dynamics related to neural nets.
It is shown that spurious minima do not arise from spontaneous symmetry breaking but rather through a complex deformation of the landscape geometry that can be encoded by a generic $S_n$-equivariant bifurcation.
Results on generic bifurcation when there are quadratic equivariants are also proved; this work extends and clarifies results of Ihrig & Golubitsky and Chossat, Lauterback &
arXiv Detail & Related papers (2021-07-06T06:43:06Z) - Learning Equivariant Energy Based Models with Equivariant Stein
Variational Gradient Descent [80.73580820014242]
We focus on the problem of efficient sampling and learning of probability densities by incorporating symmetries in probabilistic models.
We first introduce Equivariant Stein Variational Gradient Descent algorithm -- an equivariant sampling method based on Stein's identity for sampling from densities with symmetries.
We propose new ways of improving and scaling up training of energy based models.
arXiv Detail & Related papers (2021-06-15T01:35:17Z) - A Differential Geometry Perspective on Orthogonal Recurrent Models [56.09491978954866]
We employ tools and insights from differential geometry to offer a novel perspective on orthogonal RNNs.
We show that orthogonal RNNs may be viewed as optimizing in the space of divergence-free vector fields.
Motivated by this observation, we study a new recurrent model, which spans the entire space of vector fields.
arXiv Detail & Related papers (2021-02-18T19:39:22Z) - Universal Approximation Theorem for Equivariant Maps by Group CNNs [14.810452619505137]
This paper provides a unified method to obtain universal approximation theorems for equivariant maps by CNNs.
As its significant advantage, we can handle non-linear equivariant maps between infinite-dimensional spaces for non-compact groups.
arXiv Detail & Related papers (2020-12-27T07:09:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.