Connecting Permutation Equivariant Neural Networks and Partition
Diagrams
- URL: http://arxiv.org/abs/2212.08648v1
- Date: Fri, 16 Dec 2022 18:48:54 GMT
- Title: Connecting Permutation Equivariant Neural Networks and Partition
Diagrams
- Authors: Edward Pearce-Crump
- Abstract summary: We show how the Schur-Weyl duality that exists between the partition algebra and the symmetric group results in a stronger theoretical foundation for characterising possible permutation equivariant neural networks.
In particular, we find a basis of matrices for the learnable, linear, permutation equivariant layer functions between such tensor power spaces in the standard basis of $M_n$.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We show how the Schur-Weyl duality that exists between the partition algebra
and the symmetric group results in a stronger theoretical foundation for
characterising all of the possible permutation equivariant neural networks
whose layers are some tensor power of the permutation representation $M_n$ of
the symmetric group $S_n$. In doing so, we unify two separate bodies of
literature, and we correct some of the major results that are now widely quoted
by the machine learning community. In particular, we find a basis of matrices
for the learnable, linear, permutation equivariant layer functions between such
tensor power spaces in the standard basis of $M_n$ by using an elegant
graphical representation of a basis of set partitions for the partition algebra
and its related vector spaces. Also, we show how we can calculate the number of
weights that must appear in these layer functions by looking at certain paths
through the McKay quiver for $M_n$. Finally, we describe how our approach
generalises to the construction of neural networks that are equivariant to
local symmetries.
Related papers
- Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry [63.694184882697435]
Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations.
arXiv Detail & Related papers (2024-07-15T07:11:44Z) - Fast computation of permutation equivariant layers with the partition
algebra [0.0]
Linear neural network layers that are either equivariant or invariant to permutations of their inputs form core building blocks of modern deep learning architectures.
Examples include the layers of DeepSets, as well as linear layers occurring in attention blocks of transformers and some graph neural networks.
arXiv Detail & Related papers (2023-03-10T21:13:12Z) - Permutation Equivariant Neural Functionals [92.0667671999604]
This work studies the design of neural networks that can process the weights or gradients of other neural networks.
We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order.
In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
arXiv Detail & Related papers (2023-02-27T18:52:38Z) - How Jellyfish Characterise Alternating Group Equivariant Neural Networks [0.0]
We find a basis for the learnable, linear, $A_n$-equivariant layer functions between such tensor power spaces in the standard basis of $mathbbRn$.
We also describe how our approach generalises to the construction of neural networks that are equivariant to local symmetries.
arXiv Detail & Related papers (2023-01-24T17:39:10Z) - Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras
from First Principles [55.41644538483948]
We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset.
We use fully connected neural networks to model the transformations symmetry and the corresponding generators.
Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.
arXiv Detail & Related papers (2023-01-13T16:25:25Z) - Equivalence Between SE(3) Equivariant Networks via Steerable Kernels and
Group Convolution [90.67482899242093]
A wide range of techniques have been proposed in recent years for designing neural networks for 3D data that are equivariant under rotation and translation of the input.
We provide an in-depth analysis of both methods and their equivalence and relate the two constructions to multiview convolutional networks.
We also derive new TFN non-linearities from our equivalence principle and test them on practical benchmark datasets.
arXiv Detail & Related papers (2022-11-29T03:42:11Z) - Geometric Deep Learning and Equivariant Neural Networks [0.9381376621526817]
We survey the mathematical foundations of geometric deep learning, focusing on group equivariant and gauge equivariant neural networks.
We develop gauge equivariant convolutional neural networks on arbitrary manifold $mathcalM$ using principal bundles with structure group $K$ and equivariant maps between sections of associated vector bundles.
We analyze several applications of this formalism, including semantic segmentation and object detection networks.
arXiv Detail & Related papers (2021-05-28T15:41:52Z) - A Practical Method for Constructing Equivariant Multilayer Perceptrons
for Arbitrary Matrix Groups [115.58550697886987]
We provide a completely general algorithm for solving for the equivariant layers of matrix groups.
In addition to recovering solutions from other works as special cases, we construct multilayer perceptrons equivariant to multiple groups that have never been tackled before.
Our approach outperforms non-equivariant baselines, with applications to particle physics and dynamical systems.
arXiv Detail & Related papers (2021-04-19T17:21:54Z) - LieTransformer: Equivariant self-attention for Lie Groups [49.9625160479096]
Group equivariant neural networks are used as building blocks of group invariant neural networks.
We extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models.
We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
arXiv Detail & Related papers (2020-12-20T11:02:49Z) - Applying Lie Groups Approaches for Rigid Registration of Point Clouds [3.308743964406687]
We use Lie groups and Lie algebras to find the rigid transformation that best register two surfaces represented by point clouds.
The so called pairwise rigid registration can be formulated by comparing intrinsic second-order orientation tensors.
We show promising results when embedding orientation tensor fields in Lie algebras.
arXiv Detail & Related papers (2020-06-23T21:26:57Z) - The general theory of permutation equivarant neural networks and higher
order graph variational encoders [6.117371161379209]
We derive formulae for general permutation equivariant layers, including the case where the layer acts on matrices by permuting their rows and columns simultaneously.
This case arises naturally in graph learning and relation learning applications.
We present a second order graph variational encoder, and show that the latent distribution of equivariant generative models must be exchangeable.
arXiv Detail & Related papers (2020-04-08T13:29:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.