Deconstructing the Inductive Biases of Hamiltonian Neural Networks

• URL: http://arxiv.org/abs/2202.04836v2
• Date: Sat, 12 Feb 2022 01:04:45 GMT
• Title: Deconstructing the Inductive Biases of Hamiltonian Neural Networks
• Authors: Nate Gruver, Marc Finzi, Samuel Stanton, Andrew Gordon Wilson
• Abstract summary: Physics-inspired neural networks (NNs) dramatically outperform other learned dynamics models by leveraging strong inductive biases. We show that, contrary to conventional wisdom, the improved generalization of HNNs is the result of modeling acceleration directly. We show that by relaxing the inductive biases of these models, we can match or exceed performance on energy-conserving systems while dramatically improving performance on practical, non-conservative systems.
• Score: 41.37309202965647
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physics-inspired neural networks (NNs), such as Hamiltonian or Lagrangian NNs, dramatically outperform other learned dynamics models by leveraging strong inductive biases. These models, however, are challenging to apply to many real world systems, such as those that don't conserve energy or contain contacts, a common setting for robotics and reinforcement learning. In this paper, we examine the inductive biases that make physics-inspired models successful in practice. We show that, contrary to conventional wisdom, the improved generalization of HNNs is the result of modeling acceleration directly and avoiding artificial complexity from the coordinate system, rather than symplectic structure or energy conservation. We show that by relaxing the inductive biases of these models, we can match or exceed performance on energy-conserving systems while dramatically improving performance on practical, non-conservative systems. We extend this approach to constructing transition models for common Mujoco environments, showing that our model can appropriately balance inductive biases with the flexibility required for model-based control.

Related papers

• TANGO: Time-Reversal Latent GraphODE for Multi-Agent Dynamical Systems [43.39754726042369]
  We propose a simple-yet-effective self-supervised regularization term as a soft constraint that aligns the forward and backward trajectories predicted by a continuous graph neural network-based ordinary differential equation (GraphODE) It effectively imposes time-reversal symmetry to enable more accurate model predictions across a wider range of dynamical systems under classical mechanics. Experimental results on a variety of physical systems demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2023-10-10T08:52:16Z)
• Exploring Model Transferability through the Lens of Potential Energy [78.60851825944212]
  Transfer learning has become crucial in computer vision tasks due to the vast availability of pre-trained deep learning models. Existing methods for measuring the transferability of pre-trained models rely on statistical correlations between encoded static features and task labels. We present an insightful physics-inspired approach named PED to address these challenges.
  arXiv  Detail & Related papers  (2023-08-29T07:15:57Z)
• SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases [66.61789780666727]
  We show how the second-order continuity can be incorporated into GNNs while maintaining the equivariant property. We also offer theoretical insights into SEGNO, highlighting that it can learn a unique trajectory between adjacent states. Our model yields a significant improvement over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-08-25T07:15:58Z)
• MINN: Learning the dynamics of differential-algebraic equations and application to battery modeling [3.900623554490941]
  We propose a novel architecture for generating model-integrated neural networks (MINN) MINN allows integration on the level of learning physics-based dynamics of the system. We apply the proposed neural network architecture to model the electrochemical dynamics of lithium-ion batteries.
  arXiv  Detail & Related papers  (2023-04-27T09:11:40Z)
• Unravelling the Performance of Physics-informed Graph Neural Networks for Dynamical Systems [5.787429262238507]
  We evaluate the performance of graph neural networks (GNNs) and their variants with explicit constraints and different architectures. Our study demonstrates that GNNs with additional inductive biases, such as explicit constraints and decoupling of kinetic and potential energies, exhibit significantly enhanced performance. All the physics-informed GNNs exhibit zero-shot generalizability to system sizes an order of magnitude larger than the training system, thus providing a promising route to simulate large-scale realistic systems.
  arXiv  Detail & Related papers  (2022-11-10T12:29:30Z)
• Maximum entropy exploration in contextual bandits with neural networks and energy based models [63.872634680339644]
  We present two classes of models, one with neural networks as reward estimators, and the other with energy based models. We show that both techniques outperform well-known standard algorithms, where energy based models have the best overall performance. This provides practitioners with new techniques that perform well in static and dynamic settings, and are particularly well suited to non-linear scenarios with continuous action spaces.
  arXiv  Detail & Related papers  (2022-10-12T15:09:45Z)
• Enhancing the Inductive Biases of Graph Neural ODE for Modeling Dynamical Systems [19.634451472032733]
  We present a graph based neural ODE, GNODE, to learn the time evolution of dynamical systems. We show that, similar to LNN and HNN, encoding the constraints explicitly can significantly improve the training efficiency and performance of GNODE. We demonstrate that inducing these biases can enhance the performance of model by orders of magnitude in terms of both energy violation and rollout error.
  arXiv  Detail & Related papers  (2022-09-22T02:20:29Z)
• Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One [83.5162421521224]
  We propose a unique method termed E-ARM for training autoregressive generative models. E-ARM takes advantage of a well-designed energy-based learning objective. We show that E-ARM can be trained efficiently and is capable of alleviating the exposure bias problem.
  arXiv  Detail & Related papers  (2022-06-26T10:58:41Z)
• Forced Variational Integrator Networks for Prediction and Control of Mechanical Systems [7.538482310185133]
  We show that forced variational integrator networks (FVIN) architecture allows us to accurately account for energy dissipation and external forcing. This can result in highly-data efficient model-based control and can predict on real non-conservative systems.
  arXiv  Detail & Related papers  (2021-06-05T21:39:09Z)
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling [86.9726984929758]
  We focus on the integration of incomplete physics models into deep generative models. We propose a VAE architecture in which a part of the latent space is grounded by physics. We demonstrate generative performance improvements over a set of synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-02-25T20:28:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.