Universal Approximation Theorem for Equivariant Maps by Group CNNs
- URL: http://arxiv.org/abs/2012.13882v1
- Date: Sun, 27 Dec 2020 07:09:06 GMT
- Title: Universal Approximation Theorem for Equivariant Maps by Group CNNs
- Authors: Wataru Kumagai, Akiyoshi Sannai
- Abstract summary: This paper provides a unified method to obtain universal approximation theorems for equivariant maps by CNNs.
As its significant advantage, we can handle non-linear equivariant maps between infinite-dimensional spaces for non-compact groups.
- Score: 14.810452619505137
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Group symmetry is inherent in a wide variety of data distributions. Data
processing that preserves symmetry is described as an equivariant map and often
effective in achieving high performance. Convolutional neural networks (CNNs)
have been known as models with equivariance and shown to approximate
equivariant maps for some specific groups. However, universal approximation
theorems for CNNs have been separately derived with individual techniques
according to each group and setting. This paper provides a unified method to
obtain universal approximation theorems for equivariant maps by CNNs in various
settings. As its significant advantage, we can handle non-linear equivariant
maps between infinite-dimensional spaces for non-compact groups.
Related papers
- A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs [5.141137421503899]
Steerable convolutional neural networks (SCNNs) enhance task performance by modelling geometric symmetries.
Yet, unknown or varying symmetries can lead to overconstrained weights and decreased performance.
This paper introduces a probabilistic method to learn the degree of equivariance in SCNNs.
arXiv Detail & Related papers (2024-06-06T10:45:19Z) - Almost Equivariance via Lie Algebra Convolutions [0.0]
We provide a definition of almost equivariance and give a practical method for encoding it in models.
Specifically, we define Lie algebra convolutions and demonstrate that they offer several benefits over Lie group convolutions.
We prove two existence theorems, one showing the existence of almost isometries within bounded distance of isometries of a manifold, and another showing the converse for Hilbert spaces.
arXiv Detail & Related papers (2023-10-19T21:31:11Z) - A tradeoff between universality of equivariant models and learnability
of symmetries [0.0]
We prove that it is impossible to simultaneously learn symmetries and functions equivariant under certain conditions.
We analyze certain families of neural networks for whether they satisfy the conditions of the impossibility result.
On the practical side, our analysis of group-convolutional neural networks allows us generalize the well-known convolution is all you need'' to non-homogeneous spaces.
arXiv Detail & Related papers (2022-10-17T21:23:22Z) - Architectural Optimization over Subgroups for Equivariant Neural
Networks [0.0]
We propose equivariance relaxation morphism and $[G]$-mixed equivariant layer to operate with equivariance constraints on a subgroup.
We present evolutionary and differentiable neural architecture search (NAS) algorithms that utilize these mechanisms respectively for equivariance-aware architectural optimization.
arXiv Detail & Related papers (2022-10-11T14:37:29Z) - Frame Averaging for Invariant and Equivariant Network Design [50.87023773850824]
We introduce Frame Averaging (FA), a framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types.
We show that FA-based models have maximal expressive power in a broad setting.
We propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs.
arXiv Detail & Related papers (2021-10-07T11:05:23Z) - Coordinate Independent Convolutional Networks -- Isometry and Gauge
Equivariant Convolutions on Riemannian Manifolds [70.32518963244466]
A major complication in comparison to flat spaces is that it is unclear in which alignment a convolution kernel should be applied on a manifold.
We argue that the particular choice of coordinatization should not affect a network's inference -- it should be coordinate independent.
A simultaneous demand for coordinate independence and weight sharing is shown to result in a requirement on the network to be equivariant.
arXiv Detail & Related papers (2021-06-10T19:54:19Z) - Group Equivariant Subsampling [60.53371517247382]
Subsampling is used in convolutional neural networks (CNNs) in the form of pooling or strided convolutions.
We first introduce translation equivariant subsampling/upsampling layers that can be used to construct exact translation equivariant CNNs.
We then generalise these layers beyond translations to general groups, thus proposing group equivariant subsampling/upsampling.
arXiv Detail & Related papers (2021-06-10T16:14:00Z) - Group Equivariant Conditional Neural Processes [30.134634059773703]
We present the group equivariant conditional neural process (EquivCNP)
We show that EquivCNP achieves comparable performance to conventional conditional neural processes in a 1D regression task.
arXiv Detail & Related papers (2021-02-17T13:50:07Z) - LieTransformer: Equivariant self-attention for Lie Groups [49.9625160479096]
Group equivariant neural networks are used as building blocks of group invariant neural networks.
We extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models.
We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
arXiv Detail & Related papers (2020-12-20T11:02:49Z) - Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric
graphs [81.12344211998635]
A common approach to define convolutions on meshes is to interpret them as a graph and apply graph convolutional networks (GCNs)
We propose Gauge Equivariant Mesh CNNs which generalize GCNs to apply anisotropic gauge equivariant kernels.
Our experiments validate the significantly improved expressivity of the proposed model over conventional GCNs and other methods.
arXiv Detail & Related papers (2020-03-11T17:21:15Z) - Generalizing Convolutional Neural Networks for Equivariance to Lie
Groups on Arbitrary Continuous Data [52.78581260260455]
We propose a general method to construct a convolutional layer that is equivariant to transformations from any specified Lie group.
We apply the same model architecture to images, ball-and-stick molecular data, and Hamiltonian dynamical systems.
arXiv Detail & Related papers (2020-02-25T17:40:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.