SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
- URL: http://arxiv.org/abs/2006.10503v3
- Date: Tue, 24 Nov 2020 19:03:03 GMT
- Title: SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
- Authors: Fabian B. Fuchs, Daniel E. Worrall, Volker Fischer, Max Welling
- Abstract summary: We introduce the SE(3)-Transformer, a variant of the self-attention module for 3D point clouds and graphs, which is equivariant under continuous 3D roto-translations.
We evaluate our model on a toy N-body particle simulation dataset, showcasing the robustness of the predictions under rotations of the input.
- Score: 71.55002934935473
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We introduce the SE(3)-Transformer, a variant of the self-attention module
for 3D point clouds and graphs, which is equivariant under continuous 3D
roto-translations. Equivariance is important to ensure stable and predictable
performance in the presence of nuisance transformations of the data input. A
positive corollary of equivariance is increased weight-tying within the model.
The SE(3)-Transformer leverages the benefits of self-attention to operate on
large point clouds and graphs with varying number of points, while guaranteeing
SE(3)-equivariance for robustness. We evaluate our model on a toy N-body
particle simulation dataset, showcasing the robustness of the predictions under
rotations of the input. We further achieve competitive performance on two
real-world datasets, ScanObjectNN and QM9. In all cases, our model outperforms
a strong, non-equivariant attention baseline and an equivariant model without
attention.
Related papers
- UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues [55.69339788566899]
UPose3D is a novel approach for multi-view 3D human pose estimation.
It improves robustness and flexibility without requiring direct 3D annotations.
arXiv Detail & Related papers (2024-04-23T00:18:00Z) - Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection [37.142470149311904]
We propose atemporal equivariant learning framework by considering both spatial and temporal augmentations jointly.
We show our pre-training method for 3D object detection which outperforms existing equivariant and invariant approaches in many settings.
arXiv Detail & Related papers (2024-04-17T20:41:49Z) - DuEqNet: Dual-Equivariance Network in Outdoor 3D Object Detection for
Autonomous Driving [4.489333751818157]
We propose DuEqNet, which first introduces the concept of equivariance into 3D object detection network.
The dual-equivariant of our model can extract the equivariant features at both local and global levels.
Our model presents higher accuracy on orientation and better prediction efficiency.
arXiv Detail & Related papers (2023-02-27T08:30:02Z) - Transformers meet Stochastic Block Models: Attention with Data-Adaptive
Sparsity and Cost [53.746169882193456]
Recent works have proposed various sparse attention modules to overcome the quadratic cost of self-attention.
We propose a model that resolves both problems by endowing each attention head with a mixed-membership Block Model.
Our model outperforms previous efficient variants as well as the original Transformer with full attention.
arXiv Detail & Related papers (2022-10-27T15:30:52Z) - SVNet: Where SO(3) Equivariance Meets Binarization on Point Cloud
Representation [65.4396959244269]
The paper tackles the challenge by designing a general framework to construct 3D learning architectures.
The proposed approach can be applied to general backbones like PointNet and DGCNN.
Experiments on ModelNet40, ShapeNet, and the real-world dataset ScanObjectNN, demonstrated that the method achieves a great trade-off between efficiency, rotation, and accuracy.
arXiv Detail & Related papers (2022-09-13T12:12:19Z) - Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic
Graphs [3.0603554929274908]
3D-related inductive biases are indispensable to graph neural networks operating on 3D atomistic graphs such as molecules.
Inspired by the success of Transformers in various domains, we study how to incorporate these inductive biases into Transformers.
We present Equiformer, a graph neural network leveraging the strength of Transformer architectures.
arXiv Detail & Related papers (2022-06-23T21:40:37Z) - VN-Transformer: Rotation-Equivariant Attention for Vector Neurons [13.027392571994204]
We introduce a novel "VN-Transformer" architecture to address several shortcomings of the current VN models.
We apply our VN-Transformer to 3D shape classification and motion forecasting with compelling results.
arXiv Detail & Related papers (2022-06-08T21:48:47Z) - Geometry-Contrastive Transformer for Generalized 3D Pose Transfer [95.56457218144983]
The intuition of this work is to perceive the geometric inconsistency between the given meshes with the powerful self-attention mechanism.
We propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies.
We present a latent isometric regularization module together with a novel semi-synthesized dataset for the cross-dataset 3D pose transfer task.
arXiv Detail & Related papers (2021-12-14T13:14:24Z) - Equivariant Point Network for 3D Point Cloud Analysis [17.689949017410836]
We propose an effective and practical SE(3) (3D translation and rotation) equivariant network for point cloud analysis.
First, we present SE(3) separable point convolution, a novel framework that breaks down the 6D convolution into two separable convolutional operators.
Second, we introduce an attention layer to effectively harness the expressiveness of the equivariant features.
arXiv Detail & Related papers (2021-03-25T21:57:10Z) - Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes [86.2129580231191]
Adjoint Rigid Transform (ART) Network is a neural module which can be integrated with a variety of 3D networks.
ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks.
We will release our code and pre-trained models for further research.
arXiv Detail & Related papers (2021-02-01T20:58:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.