Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformers
- URL: http://arxiv.org/abs/2311.04744v2
- Date: Thu, 14 Mar 2024 10:55:46 GMT
- Title: Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformers
- Authors: Pim de Haan, Taco Cohen, Johann Brehmer,
- Abstract summary: We study versions of this architecture for Euclidean, projective, and conformal algebras.
The simplest Euclidean architecture is computationally cheap, but has a smaller symmetry group and is not as sample-efficient.
Both the conformal algebra and an improved version of the projective algebra define powerful, performant architectures.
- Score: 16.656636729960727
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The Geometric Algebra Transformer (GATr) is a versatile architecture for geometric deep learning based on projective geometric algebra. We generalize this architecture into a blueprint that allows one to construct a scalable transformer architecture given any geometric (or Clifford) algebra. We study versions of this architecture for Euclidean, projective, and conformal algebras, all of which are suited to represent 3D data, and evaluate them in theory and practice. The simplest Euclidean architecture is computationally cheap, but has a smaller symmetry group and is not as sample-efficient, while the projective model is not sufficiently expressive. Both the conformal algebra and an improved version of the projective algebra define powerful, performant architectures.
Related papers
- A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems [87.30652640973317]
Recent advances in computational modelling of atomic systems represent them as geometric graphs with atoms embedded as nodes in 3D Euclidean space.
Geometric Graph Neural Networks have emerged as the preferred machine learning architecture powering applications ranging from protein structure prediction to molecular simulations and material generation.
This paper provides a comprehensive and self-contained overview of the field of Geometric GNNs for 3D atomic systems.
arXiv Detail & Related papers (2023-12-12T18:44:19Z) - Geometric Algebra Transformer [16.656636729960727]
Problems involving geometric data arise in physics, chemistry, robotics, computer vision, and many other fields.
There is no single architecture that can be applied to such a wide variety of geometric types while respecting their symmetries.
We introduce the Geometric Algebra Transformer (GATr), a general-purpose architecture for geometric data.
arXiv Detail & Related papers (2023-05-28T18:48:50Z) - GeometricImageNet: Extending convolutional neural networks to vector and
tensor images [6.540771405203322]
GeometricImageNet is a generalization of convolution with outer products, tensor index contractions, and tensor index permutations.
In numerical experiments, we find that GeometricImageNet has good generalization for a small simulated physics system.
We expect this tool will be valuable for scientific and engineering machine learning, for example in cosmology or ocean dynamics.
arXiv Detail & Related papers (2023-05-21T22:44:18Z) - Exploring Data Geometry for Continual Learning [64.4358878435983]
We study continual learning from a novel perspective by exploring data geometry for the non-stationary stream of data.
Our method dynamically expands the geometry of the underlying space to match growing geometric structures induced by new data.
Experiments show that our method achieves better performance than baseline methods designed in Euclidean space.
arXiv Detail & Related papers (2023-04-08T06:35:25Z) - Geometric Clifford Algebra Networks [53.456211342585824]
We propose Geometric Clifford Algebra Networks (GCANs) for modeling dynamical systems.
GCANs are based on symmetry group transformations using geometric (Clifford) algebras.
arXiv Detail & Related papers (2023-02-13T18:48:33Z) - Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras
from First Principles [55.41644538483948]
We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset.
We use fully connected neural networks to model the transformations symmetry and the corresponding generators.
Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.
arXiv Detail & Related papers (2023-01-13T16:25:25Z) - Frame Averaging for Equivariant Shape Space Learning [85.42901997467754]
A natural way to incorporate symmetries in shape space learning is to ask that the mapping to the shape space (encoder) and mapping from the shape space (decoder) are equivariant to the relevant symmetries.
We present a framework for incorporating equivariance in encoders and decoders by introducing two contributions.
arXiv Detail & Related papers (2021-12-03T06:41:19Z) - Geometric Algebra Attention Networks for Small Point Clouds [0.0]
Problems in the physical sciences deal with relatively small sets of points in two- or three-dimensional space.
We present rotation- and permutation-equivariant architectures for deep learning on these small point clouds.
We demonstrate the usefulness of these architectures by training models to solve sample problems relevant to physics, chemistry, and biology.
arXiv Detail & Related papers (2021-10-05T22:52:12Z) - Learning from Protein Structure with Geometric Vector Perceptrons [6.5360079597553025]
We introduce geometric vector perceptrons, which extend standard dense layers to operate on collections of Euclidean vectors.
We demonstrate our approach on two important problems in learning from protein structure: model quality assessment and computational protein design.
arXiv Detail & Related papers (2020-09-03T01:54:25Z) - DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape
Generation [98.96086261213578]
We introduce DSG-Net, a deep neural network that learns a disentangled structured and geometric mesh representation for 3D shapes.
This supports a range of novel shape generation applications with disentangled control, such as of structure (geometry) while keeping geometry (structure) unchanged.
Our method not only supports controllable generation applications but also produces high-quality synthesized shapes, outperforming state-of-the-art methods.
arXiv Detail & Related papers (2020-08-12T17:06:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.