Harmonizing SO(3)-Equivariance with Neural Expressiveness: a Hybrid Deep Learning Framework Oriented to the Prediction of Electronic Structure Hamiltonian

• URL: http://arxiv.org/abs/2401.00744v9
• Date: Sun, 5 May 2024 03:51:17 GMT
• Title: Harmonizing SO(3)-Equivariance with Neural Expressiveness: a Hybrid Deep Learning Framework Oriented to the Prediction of Electronic Structure Hamiltonian
• Authors: Shi Yin, Xinyang Pan, Xudong Zhu, Tianyu Gao, Haochong Zhang, Feng Wu, Lixin He,
• Abstract summary: HarmoSE is a two-stage cascaded regression framework for deep learning. First stage predicts Hamiltonians with abundant SO(3)-equivariant features extracted. Second stage refines the first stage's output as a fine-grained prediction of Hamiltonians.
• Score: 36.13416266854978
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning for predicting the electronic structure Hamiltonian of quantum systems necessitates satisfying the covariance laws, among which achieving SO(3)-equivariance without sacrificing the non-linear expressive capability of networks remains unsolved. To navigate the harmonization between equivariance and expressiveness, we propose a deep learning method, namely HarmoSE, synergizing two distinct categories of neural mechanisms as a two-stage cascaded regression framework. The first stage corresponds to group theory-based neural mechanisms with inherent SO(3)-equivariant properties prior to the parameter learning process, while the second stage is characterized by a non-linear 3D graph Transformer network we propose featuring high capability on non-linear expressiveness. The novel combination lies in the point that, the first stage predicts baseline Hamiltonians with abundant SO(3)-equivariant features extracted, assisting the second stage in empirical learning of equivariance; and in turn, the second stage refines the first stage's output as a fine-grained prediction of Hamiltonians using powerful non-linear neural mappings, compensating for the intrinsic weakness on non-linear expressiveness capability of mechanisms in the first stage. Our method enables precise, generalizable predictions while maintaining robust SO(3)-equivariance under rotational transformations, and achieves state-of-the-art performance in Hamiltonian prediction on six benchmark databases.

Related papers

• Efficient and Scalable Density Functional Theory Hamiltonian Prediction through Adaptive Sparsity [11.415146682472127]
  We introduce an efficient and scalable equivariant network that incorporates adaptive sparsity into Hamiltonian prediction. We develop a Three-phase Sparsity Scheduler, ensuring stable convergence and achieving high performance at sparsity rates of up to 70 percent. Beyond Hamiltonian prediction, the proposed sparsification techniques also hold significant potential for improving the efficiency and scalability of other SE(3) equivariant networks.
  arXiv  Detail & Related papers  (2025-02-03T09:04:47Z)
• Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling [3.8719670789415925]
  Recurrent configuration networks (RSCNs) have shown great potential in modelling nonlinear dynamic systems with uncertainties. This paper presents an RSCN with hybrid regularization to enhance both the learning capacity and generalization performance of the network.
  arXiv  Detail & Related papers  (2024-11-26T03:06:39Z)
• Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems [2.170477444239546]
  We develop an approach that balances these two objectives: the Gaussian Process Switching Linear Dynamical System (gpSLDS) Our method builds on previous work modeling the latent state evolution via a differential equation whose nonlinear dynamics are described by a Gaussian process (GP-SDEs) Our approach resolves key limitations of the rSLDS such as artifactual oscillations in dynamics near discrete state boundaries, while also providing posterior uncertainty estimates of the dynamics.
  arXiv  Detail & Related papers  (2024-07-19T15:32:15Z)
• TraceGrad: a Framework Learning Expressive SO(3)-equivariant Non-linear Representations for Electronic-Structure Hamiltonian Prediction [1.8982950873008362]
  We propose a framework to combine strong non-linear expressiveness with strict SO(3)-equivariant in prediction of the electronic-structure Hamiltonian. Our method achieves state-of-the-art performance in prediction accuracy across eight challenging benchmark databases on Hamiltonian prediction.
  arXiv  Detail & Related papers  (2024-05-09T12:34:45Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Neural Dynamic Mode Decomposition for End-to-End Modeling of Nonlinear Dynamics [49.41640137945938]
  We propose a neural dynamic mode decomposition for estimating a lift function based on neural networks. With our proposed method, the forecast error is backpropagated through the neural networks and the spectral decomposition. Our experiments demonstrate the effectiveness of our proposed method in terms of eigenvalue estimation and forecast performance.
  arXiv  Detail & Related papers  (2020-12-11T08:34:26Z)
• Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning [58.14930566993063]
  We present connections between three models used in different research fields: weighted finite automata(WFA) from formal languages and linguistics, recurrent neural networks used in machine learning, and tensor networks. We introduce the first provable learning algorithm for linear 2-RNN defined over sequences of continuous vectors input.
  arXiv  Detail & Related papers  (2020-10-19T15:28:00Z)
• Adding machine learning within Hamiltonians: Renormalization group transformations, symmetry breaking and restoration [0.0]
  We include the predictive function of a neural network, designed for phase classification, as a conjugate variable coupled to an external field within the Hamiltonian of a system. Results show that the field can induce an order-disorder phase transition by breaking or restoring the symmetry. We conclude by discussing how the method provides an essential step toward bridging machine learning and physics.
  arXiv  Detail & Related papers  (2020-09-30T18:44:18Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
  empirical optimization is central to modern machine learning, but its role in its success is still unclear. We show that it commonly arises in parameters of discrete multiplicative noise due to variance. A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
  arXiv  Detail & Related papers  (2020-06-11T09:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.