So3krates -- Self-attention for higher-order geometric interactions on arbitrary length-scales

• URL: http://arxiv.org/abs/2205.14276v1
• Date: Sat, 28 May 2022 00:01:30 GMT
• Title: So3krates -- Self-attention for higher-order geometric interactions on arbitrary length-scales
• Authors: J. Thorben Frank, Oliver T. Unke, Klaus-Robert M\"uller
• Abstract summary: Some quantum mechanical properties of molecules and materials depend on non-local electronic effects. This work proposes a modified attention mechanism adapted to the underlying physics. Our proposed model So3krates is able to describe non-local quantum mechanical effects over arbitrary length scales.
• Score: 2.1485350418225244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The application of machine learning methods in quantum chemistry has enabled the study of numerous chemical phenomena, which are computationally intractable with traditional ab-initio methods. However, some quantum mechanical properties of molecules and materials depend on non-local electronic effects, which are often neglected due to the difficulty of modeling them efficiently. This work proposes a modified attention mechanism adapted to the underlying physics, which allows to recover the relevant non-local effects. Namely, we introduce spherical harmonic coordinates (SPHCs) to reflect higher-order geometric information for each atom in a molecule, enabling a non-local formulation of attention in the SPHC space. Our proposed model So3krates -- a self-attention based message passing neural network -- uncouples geometric information from atomic features, making them independently amenable to attention mechanisms. We show that in contrast to other published methods, So3krates is able to describe non-local quantum mechanical effects over arbitrary length scales. Further, we find evidence that the inclusion of higher-order geometric correlations increases data efficiency and improves generalization. So3krates matches or exceeds state-of-the-art performance on popular benchmarks, notably, requiring a significantly lower number of parameters (0.25--0.4x) while at the same time giving a substantial speedup (6--14x for training and 2--11x for inference) compared to other models.

Related papers

• Modeling Non-Covalent Interatomic Interactions on a Photonic Quantum Computer [50.24983453990065]
  We show that the cQDO model lends itself naturally to simulation on a photonic quantum computer. We calculate the binding energy curve of diatomic systems by leveraging Xanadu's Strawberry Fields photonics library. Remarkably, we find that two coupled bosonic QDOs exhibit a stable bond.
  arXiv  Detail & Related papers  (2023-06-14T14:44:12Z)
• Molecular Geometry-aware Transformer for accurate 3D Atomic System modeling [51.83761266429285]
  We propose a novel Transformer architecture that takes nodes (atoms) and edges (bonds and nonbonding atom pairs) as inputs and models the interactions among them. Moleformer achieves state-of-the-art on the initial state to relaxed energy prediction of OC20 and is very competitive in QM9 on predicting quantum chemical properties.
  arXiv  Detail & Related papers  (2023-02-02T03:49:57Z)
• Universality of critical dynamics with finite entanglement [68.8204255655161]
  We study how low-energy dynamics of quantum systems near criticality are modified by finite entanglement. Our result establishes the precise role played by entanglement in time-dependent critical phenomena.
  arXiv  Detail & Related papers  (2023-01-23T19:23:54Z)
• A Quantum-Classical Model of Brain Dynamics [62.997667081978825]
  Mixed Weyl symbol is used to describe brain processes at the microscopic level. Electromagnetic fields and phonon modes involved in the processes are treated either classically or semi-classically. Zero-point quantum effects can be incorporated into numerical simulations by controlling the temperature of each field mode.
  arXiv  Detail & Related papers  (2023-01-17T15:16:21Z)
• Collective dynamics Using Truncated Equations (CUT-E): simulating the collective strong coupling regime with few-molecule models [0.0]
  We exploit permutational symmetries to drastically reduce the computational cost of textitab-initio quantum dynamics simulations for large $N$. We show that addition of $k$ extra effective molecules is enough to account for phenomena whose rates scale as $mathcalO(N-k)$.
  arXiv  Detail & Related papers  (2022-09-11T22:49:29Z)
• GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles [60.12186997181117]
  Prediction of a molecule's 3D conformer ensemble from the molecular graph holds a key role in areas of cheminformatics and drug discovery. Existing generative models have several drawbacks including lack of modeling important molecular geometry elements. We propose GeoMol, an end-to-end, non-autoregressive and SE(3)-invariant machine learning approach to generate 3D conformers.
  arXiv  Detail & Related papers  (2021-06-08T14:17:59Z)
• Programmable Interactions and Emergent Geometry in an Atomic Array [0.0]
  Nonlocal interactions govern the flow of information and the formation of correlations in quantum systems. We report on the realization of programmable nonlocal interactions in an array of atomic ensembles within an optical cavity. Our work opens broader prospects for simulating frustrated magnets and topological phases, investigating quantum optimization algorithms, and engineering new entangled resource states for sensing and computation.
  arXiv  Detail & Related papers  (2021-06-08T03:00:49Z)
• SE(3)-equivariant prediction of molecular wavefunctions and electronic densities [4.2572103161049055]
  We introduce general SE(3)-equivariant operations and building blocks for constructing deep learning architectures for geometric point cloud data. Our model reduces prediction errors by up to two orders of magnitude compared to the previous state-of-the-art. We demonstrate the potential of our approach in a transfer learning application, where a model trained on low accuracy reference wavefunctions implicitly learns to correct for electronic many-body interactions.
  arXiv  Detail & Related papers  (2021-06-04T08:57:46Z)
• Graph Neural Network for Hamiltonian-Based Material Property Prediction [56.94118357003096]
  We present and compare several different graph convolution networks that are able to predict the band gap for inorganic materials. The models are developed to incorporate two different features: the information of each orbital itself and the interaction between each other. The results show that our model can get a promising prediction accuracy with cross-validation.
  arXiv  Detail & Related papers  (2020-05-27T13:32:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.