Incorporating Symmetry into Deep Dynamics Models for Improved Generalization

• URL: http://arxiv.org/abs/2002.03061v4
• Date: Mon, 15 Mar 2021 23:00:39 GMT
• Title: Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
• Authors: Rui Wang, Robin Walters, Rose Yu
• Abstract summary: We propose to improve accuracy and generalization by incorporating symmetries into convolutional neural networks. Our models are theoretically and experimentally robust to distributional shift by symmetry group transformations. Compared with image or text applications, our work is a significant step towards applying equivariant neural networks to high-dimensional systems.
• Score: 24.363954435050264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent work has shown deep learning can accelerate the prediction of physical dynamics relative to numerical solvers. However, limited physical accuracy and an inability to generalize under distributional shift limit its applicability to the real world. We propose to improve accuracy and generalization by incorporating symmetries into convolutional neural networks. Specifically, we employ a variety of methods each tailored to enforce a different symmetry. Our models are both theoretically and experimentally robust to distributional shift by symmetry group transformations and enjoy favorable sample complexity. We demonstrate the advantage of our approach on a variety of physical dynamics including Rayleigh B\'enard convection and real-world ocean currents and temperatures. Compared with image or text applications, our work is a significant step towards applying equivariant neural networks to high-dimensional systems with complex dynamics. We open-source our simulation, data, and code at \url{https://github.com/Rose-STL-Lab/Equivariant-Net}.

Related papers

• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Physics-informed UNets for Discovering Hidden Elasticity in Heterogeneous Materials [0.0]
  We develop a novel UNet-based neural network model for inversion in elasticity (El-UNet) We show superior performance, both in terms of accuracy and computational cost, by El-UNet compared to fully-connected physics-informed neural networks.
  arXiv  Detail & Related papers  (2023-06-01T23:35:03Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Spatial Attention Kinetic Networks with E(n)-Equivariance [0.951828574518325]
  Neural networks that are equivariant to rotations, translations, reflections, and permutations on n-dimensional geometric space have shown promise in physical modeling. We propose a simple alternative functional form that uses neurally parametrized linear combinations of edge vectors to achieve equivariance. We design spatial attention kinetic networks with E(n)-equivariance, or SAKE, which are competitive in many-body system modeling tasks while being significantly faster.
  arXiv  Detail & Related papers  (2023-01-21T05:14:29Z)
• Learning Physical Dynamics with Subequivariant Graph Neural Networks [99.41677381754678]
  Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics. Physical laws abide by symmetry, which is a vital inductive bias accounting for model generalization. Our model achieves on average over 3% enhancement in contact prediction accuracy across 8 scenarios on Physion and 2X lower rollout MSE on RigidFall.
  arXiv  Detail & Related papers  (2022-10-13T10:00:30Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics [61.49826776409194]
  We analyze a corpus of models made publicly-available for a contest to predict the generalization accuracy of neural network (NN) models. We identify what amounts to a Simpson's paradox: where "scale" metrics perform well overall but perform poorly on sub partitions of the data. We present two novel shape metrics, one data-independent, and the other data-dependent, which can predict trends in the test accuracy of a series of NNs.
  arXiv  Detail & Related papers  (2021-06-01T19:19:49Z)
• Machine learning for rapid discovery of laminar flow channel wall modifications that enhance heat transfer [56.34005280792013]
  We present a combination of accurate numerical simulations of arbitrary, flat, and non-flat channels and machine learning models predicting drag coefficient and Stanton number. We show that convolutional neural networks (CNN) can accurately predict the target properties at a fraction of the time of numerical simulations.
  arXiv  Detail & Related papers  (2021-01-19T16:14:02Z)
• Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics [26.485269202381932]
  Understanding the dynamics of neural network parameters during training is one of the key challenges in building a theoretical foundation for deep learning. We show that any such symmetry imposes stringent geometric constraints on gradients and Hessians, leading to an associated conservation law. We apply tools from finite difference methods to derive modified gradient flow, a differential equation that better approximates the numerical trajectory taken by SGD at finite learning rates.
  arXiv  Detail & Related papers  (2020-12-08T20:33:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.