Unifying O(3) Equivariant Neural Networks Design with Tensor-Network Formalism

• URL: http://arxiv.org/abs/2211.07482v3
• Date: Wed, 22 May 2024 03:40:46 GMT
• Title: Unifying O(3) Equivariant Neural Networks Design with Tensor-Network Formalism
• Authors: Zimu Li, Zihan Pengmei, Han Zheng, Erik Thiede, Junyu Liu, Risi Kondor,
• Abstract summary: We propose using fusion diagrams, a technique widely employed in simulating SU($2$)-symmetric quantum many-body problems, to design new equivariant components for equivariant neural networks. When applied to particles within a given local neighborhood, the resulting components, which we term "fusion blocks," serve as universal approximators of any continuous equivariant function. Our approach, which combines tensor networks with equivariant neural networks, suggests a potentially fruitful direction for designing more expressive equivariant neural networks.
• Score: 12.008737454250463
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many learning tasks, including learning potential energy surfaces from ab initio calculations, involve global spatial symmetries and permutational symmetry between atoms or general particles. Equivariant graph neural networks are a standard approach to such problems, with one of the most successful methods employing tensor products between various tensors that transform under the spatial group. However, as the number of different tensors and the complexity of relationships between them increase, maintaining parsimony and equivariance becomes increasingly challenging. In this paper, we propose using fusion diagrams, a technique widely employed in simulating SU($2$)-symmetric quantum many-body problems, to design new equivariant components for equivariant neural networks. This results in a diagrammatic approach to constructing novel neural network architectures. When applied to particles within a given local neighborhood, the resulting components, which we term "fusion blocks," serve as universal approximators of any continuous equivariant function defined in the neighborhood. We incorporate a fusion block into pre-existing equivariant architectures (Cormorant and MACE), leading to improved performance with fewer parameters on a range of challenging chemical problems. Furthermore, we apply group-equivariant neural networks to study non-adiabatic molecular dynamics of stilbene cis-trans isomerization. Our approach, which combines tensor networks with equivariant neural networks, suggests a potentially fruitful direction for designing more expressive equivariant neural networks.

Related papers

• Relaxing Continuous Constraints of Equivariant Graph Neural Networks for Physical Dynamics Learning [39.25135680793105]
  We propose a general Discrete Equivariant Graph Neural Network (DEGNN) that guarantees equivariance to a given discrete point group. Specifically, we show that such discrete equivariant message passing could be constructed by transforming geometric features into permutation-invariant embeddings. We show that DEGNN is data efficient, learning with less data, and can generalize across scenarios such as unobserved orientation.
  arXiv  Detail & Related papers  (2024-06-24T03:37:51Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Permutation Equivariant Neural Functionals [92.0667671999604]
  This work studies the design of neural networks that can process the weights or gradients of other neural networks. We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order. In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-02-27T18:52:38Z)
• Spatial Attention Kinetic Networks with E(n)-Equivariance [0.951828574518325]
  Neural networks that are equivariant to rotations, translations, reflections, and permutations on n-dimensional geometric space have shown promise in physical modeling. We propose a simple alternative functional form that uses neurally parametrized linear combinations of edge vectors to achieve equivariance. We design spatial attention kinetic networks with E(n)-equivariance, or SAKE, which are competitive in many-body system modeling tasks while being significantly faster.
  arXiv  Detail & Related papers  (2023-01-21T05:14:29Z)
• Interrelation of equivariant Gaussian processes and convolutional neural networks [77.34726150561087]
  Currently there exists rather promising new trend in machine leaning (ML) based on the relationship between neural networks (NN) and Gaussian processes (GP) In this work we establish a relationship between the many-channel limit for CNNs equivariant with respect to two-dimensional Euclidean group with vector-valued neuron activations and the corresponding independently introduced equivariant Gaussian processes (GP)
  arXiv  Detail & Related papers  (2022-09-17T17:02:35Z)
• Generalization capabilities of neural networks in lattice applications [0.0]
  We investigate the advantages of adopting translationally equivariant neural networks in favor of non-equivariant ones. We show that our best equivariant architectures can perform and generalize significantly better than their non-equivariant counterparts.
  arXiv  Detail & Related papers  (2021-12-23T11:48:06Z)
• Revisiting Transformation Invariant Geometric Deep Learning: Are Initial Representations All You Need? [80.86819657126041]
  We show that transformation-invariant and distance-preserving initial representations are sufficient to achieve transformation invariance. Specifically, we realize transformation-invariant and distance-preserving initial point representations by modifying multi-dimensional scaling. We prove that TinvNN can strictly guarantee transformation invariance, being general and flexible enough to be combined with the existing neural networks.
  arXiv  Detail & Related papers  (2021-12-23T03:52:33Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• UNiTE: Unitary N-body Tensor Equivariant Network with Applications to Quantum Chemistry [33.067344811580604]
  We propose unitary $N$-body tensor equivariant neural network (UNiTE) for general class of symmetric tensors. UNiTE is equivariant with respect to the actions of a unitary group, such as the group of 3D rotations. When applied to quantum chemistry, UNiTE outperforms all state-of-the-art machine learning methods.
  arXiv  Detail & Related papers  (2021-05-31T00:48:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.