Smooth, exact rotational symmetrization for deep learning on point clouds

• URL: http://arxiv.org/abs/2305.19302v3
• Date: Tue, 6 Feb 2024 13:14:35 GMT
• Title: Smooth, exact rotational symmetrization for deep learning on point clouds
• Authors: Sergey N. Pozdnyakov and Michele Ceriotti
• Abstract summary: General-purpose point-cloud models are more varied but often disregard rotational symmetry. We propose a general symmetrization method that adds rotational equivariance to any given model while preserving all the other requirements. We demonstrate this idea by introducing the Point Edge Transformer (PET) architecture, which is not intrinsically equivariant but achieves state-of-the-art performance on several benchmark datasets of molecules and solids.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Point clouds are versatile representations of 3D objects and have found widespread application in science and engineering. Many successful deep-learning models have been proposed that use them as input. The domain of chemical and materials modeling is especially challenging because exact compliance with physical constraints is highly desirable for a model to be usable in practice. These constraints include smoothness and invariance with respect to translations, rotations, and permutations of identical atoms. If these requirements are not rigorously fulfilled, atomistic simulations might lead to absurd outcomes even if the model has excellent accuracy. Consequently, dedicated architectures, which achieve invariance by restricting their design space, have been developed. General-purpose point-cloud models are more varied but often disregard rotational symmetry. We propose a general symmetrization method that adds rotational equivariance to any given model while preserving all the other requirements. Our approach simplifies the development of better atomic-scale machine-learning schemes by relaxing the constraints on the design space and making it possible to incorporate ideas that proved effective in other domains. We demonstrate this idea by introducing the Point Edge Transformer (PET) architecture, which is not intrinsically equivariant but achieves state-of-the-art performance on several benchmark datasets of molecules and solids. A-posteriori application of our general protocol makes PET exactly equivariant, with minimal changes to its accuracy.

Related papers

• Large Language-Geometry Model: When LLM meets Equivariance [53.8505081745406]
  We propose EquiLLM, a novel framework for representing 3D physical systems. We show that EquiLLM delivers significant improvements over previous methods across molecular dynamics simulation, human motion simulation, and antibody design.
  arXiv  Detail & Related papers  (2025-02-16T14:50:49Z)
• A Versatile and Differentiable Hand-Object Interaction Representation [2.184775414778289]
  Coarse Hand-Object Interaction Representation (CHOIR) is versatile and differentiable for HOI modelling. ChoIR represents dense contact maps with few parameters. JointDiffusion is a diffusion model to learn a grasp distribution conditioned on noisy hand-object interactions.
  arXiv  Detail & Related papers  (2024-09-25T12:06:30Z)
• Similarity Equivariant Graph Neural Networks for Homogenization of Metamaterials [3.6443770850509423]
  Soft, porous mechanical metamaterials exhibit pattern transformations that may have important applications in soft robotics, sound reduction and biomedicine. We develop a machine learning-based approach that scales favorably to serve as a surrogate model. We show that this network is more accurate and data-efficient than graph neural networks with fewer symmetries.
  arXiv  Detail & Related papers  (2024-04-26T12:30:32Z)
• Adaptive Point Transformer [88.28498667506165]
  Adaptive Point Cloud Transformer (AdaPT) is a standard PT model augmented by an adaptive token selection mechanism. AdaPT dynamically reduces the number of tokens during inference, enabling efficient processing of large point clouds.
  arXiv  Detail & Related papers  (2024-01-26T13:24:45Z)
• Equivariant Transformer is all you need [0.0]
  We introduce symmetry equivariant attention to self-learning Monte-Carlo. We find that it overcomes poor acceptance rates for linear models and observe the scaling law of the acceptance rate as in the large language models with Transformers.
  arXiv  Detail & Related papers  (2023-10-20T01:57:03Z)
• Variational Monte Carlo on a Budget -- Fine-tuning pre-trained Neural Wavefunctions [5.145741425164946]
  Deep-learning-based Variational Monte Carlo (DL-VMC) has recently outperformed conventional approaches in terms of accuracy, but only at large computational cost. We propose a DL-VMC model which has been pre-trained using self-supervised wavefunction optimization on a large and chemically diverse set of molecules. Applying this model to new molecules without any optimization, yields wavefunctions and absolute energies that outperform established methods such as CCSD(T)-2Z.
  arXiv  Detail & Related papers  (2023-07-15T09:56:22Z)
• Equivariant geometric convolutions for emulation of dynamical systems [6.3003220645859175]
  We use geometric convolutions to enforce coordinate freedom in surrogate machine learning models. In numerical experiments emulating 2D compressible Navier-Stokes, we see better accuracy and improved stability. The ease of enforcing coordinate freedom without making major changes to the model architecture provides an exciting recipe for any CNN-based method applied to an appropriate class of problems.
  arXiv  Detail & Related papers  (2023-05-21T22:44:18Z)
• FAENet: Frame Averaging Equivariant GNN for Materials Modeling [123.19473575281357]
  We introduce a flexible framework relying on frameaveraging (SFA) to make any model E(3)-equivariant or invariant through data transformations. We prove the validity of our method theoretically and empirically demonstrate its superior accuracy and computational scalability in materials modeling.
  arXiv  Detail & Related papers  (2023-04-28T21:48:31Z)
• Learning Physical Dynamics with Subequivariant Graph Neural Networks [99.41677381754678]
  Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics. Physical laws abide by symmetry, which is a vital inductive bias accounting for model generalization. Our model achieves on average over 3% enhancement in contact prediction accuracy across 8 scenarios on Physion and 2X lower rollout MSE on RigidFall.
  arXiv  Detail & Related papers  (2022-10-13T10:00:30Z)
• REMuS-GNN: A Rotation-Equivariant Model for Simulating Continuum Dynamics [0.0]
  We introduce REMuS-GNN, a rotation-equivariant multi-scale model for simulating continuum dynamical systems. We demonstrate and evaluate this method on the incompressible flow around elliptical cylinders.
  arXiv  Detail & Related papers  (2022-05-05T16:20:37Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles [60.12186997181117]
  Prediction of a molecule's 3D conformer ensemble from the molecular graph holds a key role in areas of cheminformatics and drug discovery. Existing generative models have several drawbacks including lack of modeling important molecular geometry elements. We propose GeoMol, an end-to-end, non-autoregressive and SE(3)-invariant machine learning approach to generate 3D conformers.
  arXiv  Detail & Related papers  (2021-06-08T14:17:59Z)
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling [86.9726984929758]
  We focus on the integration of incomplete physics models into deep generative models. We propose a VAE architecture in which a part of the latent space is grounded by physics. We demonstrate generative performance improvements over a set of synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-02-25T20:28:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.