Smoothness Errors in Dynamics Models and How to Avoid Them

• URL: http://arxiv.org/abs/2602.05352v1
• Date: Thu, 05 Feb 2026 06:23:25 GMT
• Title: Smoothness Errors in Dynamics Models and How to Avoid Them
• Authors: Edward Berman, Luisa Li, Jung Yeon Park, Robin Walters,
• Abstract summary: We study the smoothing effects of different GNNs for dynamics modeling and prove that unitary convolutions hurt performance for such tasks.<n>We propose relaxed unitary convolutions that balance smoothness with the natural smoothing required for physical systems.
• Score: 12.250328115115733
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern neural networks have shown promise for solving partial differential equations over surfaces, often by discretizing the surface as a mesh and learning with a mesh-aware graph neural network. However, graph neural networks suffer from oversmoothing, where a node's features become increasingly similar to those of its neighbors. Unitary graph convolutions, which are mathematically constrained to preserve smoothness, have been proposed to address this issue. Despite this, in many physical systems, such as diffusion processes, smoothness naturally increases and unitarity may be overconstraining. In this paper, we systematically study the smoothing effects of different GNNs for dynamics modeling and prove that unitary convolutions hurt performance for such tasks. We propose relaxed unitary convolutions that balance smoothness preservation with the natural smoothing required for physical systems. We also generalize unitary and relaxed unitary convolutions from graphs to meshes. In experiments on PDEs such as the heat and wave equations over complex meshes and on weather forecasting, we find that our method outperforms several strong baselines, including mesh-aware transformers and equivariant neural networks.

Related papers

• An Operator-Consistent Graph Neural Network for Learning Diffusion Dynamics on Irregular Meshes [0.0]
  Multiphysics interactions such as diffusion, damage, and healing often take place on irregular meshes.<n>We develop an operator-consistent graph neural network (OCGNN-PINN) that approximates PDE evolution under physics-informed constraints.
  arXiv  Detail & Related papers  (2025-12-05T06:58:25Z)
• Fractional Spike Differential Equations Neural Network with Efficient Adjoint Parameters Training [63.3991315762955]
  Spiking Neural Networks (SNNs) draw inspiration from biological neurons to create realistic models for brain-like computation.<n>Most existing SNNs assume a single time constant for neuronal membrane voltage dynamics, modeled by first-order ordinary differential equations (ODEs) with Markovian characteristics.<n>We propose the Fractional SPIKE Differential Equation neural network (fspikeDE), which captures long-term dependencies in membrane voltage and spike trains through fractional-order dynamics.
  arXiv  Detail & Related papers  (2025-07-22T18:20:56Z)
• A Dynamical Systems Perspective on the Analysis of Neural Networks [0.0]
  We utilize dynamical systems to analyze several aspects of machine learning algorithms.<n>We demonstrate how to re-formulate a variety of challenges from deep neural networks, (stochastic) gradient descent, and related topics into dynamical statements.
  arXiv  Detail & Related papers  (2025-07-07T16:18:49Z)
• Rapid training of Hamiltonian graph networks using random features [1.1199585259018459]
  Hamiltonian Graph Networks (HGN) can be trained up to 600x faster--but with comparable accuracy--by replacing iterative optimization with random feature-based parameter construction.<n>We show robust performance in diverse simulations, including N-body mass-spring and molecular systems in up to 3 dimensions and 10,000 particles with different geometries.<n>We reveal that even when trained on minimal 8-node systems, the model can generalize in a zero-shot manner to systems as large as 4096 nodes without retraining.
  arXiv  Detail & Related papers  (2025-06-06T22:10:05Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Uncertainty Quantification of Graph Convolution Neural Network Models of Evolving Processes [0.8749675983608172]
  We show that Stein variational inference is a viable alternative to Monte Carlo methods for complex neural network models. For our exemplars, Stein variational interference gave similar uncertainty profiles through time compared to Hamiltonian Monte Carlo.
  arXiv  Detail & Related papers  (2024-02-17T03:19:23Z)
• Spatial Attention Kinetic Networks with E(n)-Equivariance [0.951828574518325]
  Neural networks that are equivariant to rotations, translations, reflections, and permutations on n-dimensional geometric space have shown promise in physical modeling. We propose a simple alternative functional form that uses neurally parametrized linear combinations of edge vectors to achieve equivariance. We design spatial attention kinetic networks with E(n)-equivariance, or SAKE, which are competitive in many-body system modeling tasks while being significantly faster.
  arXiv  Detail & Related papers  (2023-01-21T05:14:29Z)
• Learning Physical Dynamics with Subequivariant Graph Neural Networks [99.41677381754678]
  Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics. Physical laws abide by symmetry, which is a vital inductive bias accounting for model generalization. Our model achieves on average over 3% enhancement in contact prediction accuracy across 8 scenarios on Physion and 2X lower rollout MSE on RigidFall.
  arXiv  Detail & Related papers  (2022-10-13T10:00:30Z)
• E(n) Equivariant Graph Neural Networks [86.75170631724548]
  This paper introduces a new model to learn graph neural networks equivariant to rotations, translations, reflections and permutations called E(n)-Equivariant Graph Neural Networks (EGNNs) In contrast with existing methods, our work does not require computationally expensive higher-order representations in intermediate layers while it still achieves competitive or better performance.
  arXiv  Detail & Related papers  (2021-02-19T10:25:33Z)
• Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction [79.81193813215872]
  We develop a hybrid (graph) neural network that combines a traditional graph convolutional network with an embedded differentiable fluid dynamics simulator inside the network itself. We show that we can both generalize well to new situations and benefit from the substantial speedup of neural network CFD predictions.
  arXiv  Detail & Related papers  (2020-07-08T21:23:19Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.