Rapid training of Hamiltonian graph networks without gradient descent

• URL: http://arxiv.org/abs/2506.06558v1
• Date: Fri, 06 Jun 2025 22:10:05 GMT
• Title: Rapid training of Hamiltonian graph networks without gradient descent
• Authors: Atamert Rahma, Chinmay Datar, Ana Cukarska, Felix Dietrich,
• Abstract summary: Hamiltonian Graph Networks (HGN) can be trained up to 600x faster than 15 different solvers.<n>We show robust performance in diverse simulations, including N-body mass-spring systems in up to 3 dimensions with different geometries.<n>We reveal that even when trained on minimal 8-node systems, the model can generalize in a zero-shot manner to systems as large as 4096 nodes without retraining.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning dynamical systems that respect physical symmetries and constraints remains a fundamental challenge in data-driven modeling. Integrating physical laws with graph neural networks facilitates principled modeling of complex N-body dynamics and yields accurate and permutation-invariant models. However, training graph neural networks with iterative, gradient-based optimization algorithms (e.g., Adam, RMSProp, LBFGS) often leads to slow training, especially for large, complex systems. In comparison to 15 different optimizers, we demonstrate that Hamiltonian Graph Networks (HGN) can be trained up to 600x faster--but with comparable accuracy--by replacing iterative optimization with random feature-based parameter construction. We show robust performance in diverse simulations, including N-body mass-spring systems in up to 3 dimensions with different geometries, while retaining essential physical invariances with respect to permutation, rotation, and translation. We reveal that even when trained on minimal 8-node systems, the model can generalize in a zero-shot manner to systems as large as 4096 nodes without retraining. Our work challenges the dominance of iterative gradient-descent-based optimization algorithms for training neural network models for physical systems.

Related papers

• GALDS: A Graph-Autoencoder-based Latent Dynamics Surrogate model to predict neurite material transport [1.104960878651584]
  We propose a Graph-Autoencoder-based Latent Dynamics Surrogate model to streamline the simulation of material transport in neural trees.<n>Our approach achieves mean relative error of 3% with maximum relative error 8% and demonstrates a 10-fold speed improvement compared to previous surrogate model approaches.
  arXiv  Detail & Related papers  (2025-07-15T00:22:00Z)
• Training Hamiltonian neural networks without backpropagation [0.0]
  We present a backpropagation-free algorithm to accelerate the training of neural networks for approximating Hamiltonian systems. We show that our approach is more than 100 times faster with CPUs than the traditionally trained Hamiltonian Neural Networks.
  arXiv  Detail & Related papers  (2024-11-26T15:22:30Z)
• Reduced-Order Neural Operators: Learning Lagrangian Dynamics on Highly Sparse Graphs [19.1312659245072]
  We present GIOROM, a data-driven discretization invariant framework for accelerating Lagrangian simulations through reduced-order modeling (ROM)<n>We leverage a data-driven graph-based neural approximation of the PDE solution operator.<n>GIOROM achieves a 6.6$times$-32$times$ reduction in input dimensionality while maintaining high-fidelity reconstructions across diverse Lagrangian regimes.
  arXiv  Detail & Related papers  (2024-07-04T13:37:26Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Geometry-Informed Neural Operator for Large-Scale 3D PDEs [76.06115572844882]
  We propose the geometry-informed neural operator (GINO) to learn the solution operator of large-scale partial differential equations. We successfully trained GINO to predict the pressure on car surfaces using only five hundred data points.
  arXiv  Detail & Related papers  (2023-09-01T16:59:21Z)
• The Underlying Correlated Dynamics in Neural Training [6.385006149689549]
  Training of neural networks is a computationally intensive task. We propose a model based on the correlation of the parameters' dynamics, which dramatically reduces the dimensionality. This representation enhances the understanding of the underlying training dynamics and can pave the way for designing better acceleration techniques.
  arXiv  Detail & Related papers  (2022-12-18T08:34:11Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks [2.580765958706854]
  We describe and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks. PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. For inverse modeling using sequential training, we find that inverse-Dirichlet weighting protects a PINN against catastrophic forgetting.
  arXiv  Detail & Related papers  (2021-07-02T10:01:37Z)
• E(n) Equivariant Graph Neural Networks [86.75170631724548]
  This paper introduces a new model to learn graph neural networks equivariant to rotations, translations, reflections and permutations called E(n)-Equivariant Graph Neural Networks (EGNNs) In contrast with existing methods, our work does not require computationally expensive higher-order representations in intermediate layers while it still achieves competitive or better performance.
  arXiv  Detail & Related papers  (2021-02-19T10:25:33Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.