Understanding the Difficulty of Training Physics-Informed Neural Networks on Dynamical Systems

• URL: http://arxiv.org/abs/2203.13648v1
• Date: Fri, 25 Mar 2022 13:50:14 GMT
• Title: Understanding the Difficulty of Training Physics-Informed Neural Networks on Dynamical Systems
• Authors: Franz M. Rohrhofer, Stefan Posch, Clemens G\"o{\ss}nitzer, Bernhard C. Geiger
• Abstract summary: Physics-informed neural networks (PINNs) seamlessly integrate data and physical constraints into the solving of problems governed by differential equations. We study the physics loss function in the vicinity of fixed points of dynamical systems. We find that reducing the computational domain lowers the optimization complexity and chance of getting trapped with nonphysical solutions.
• Score: 5.878411350387833
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physics-informed neural networks (PINNs) seamlessly integrate data and physical constraints into the solving of problems governed by differential equations. In settings with little labeled training data, their optimization relies on the complexity of the embedded physics loss function. Two fundamental questions arise in any discussion of frequently reported convergence issues in PINNs: Why does the optimization often converge to solutions that lack physical behavior? And why do reduced domain methods improve convergence behavior in PINNs? We answer these questions by studying the physics loss function in the vicinity of fixed points of dynamical systems. Experiments on a simple dynamical system demonstrate that physics loss residuals are trivially minimized in the vicinity of fixed points. As a result we observe that solutions corresponding to nonphysical system dynamics can be dominant in the physics loss landscape and optimization. We find that reducing the computational domain lowers the optimization complexity and chance of getting trapped with nonphysical solutions.

Related papers

• A Physics Informed Neural Network (PINN) Methodology for Coupled Moving Boundary PDEs [0.0]
  Physics-Informed Neural Network (PINN) is a novel multi-task learning framework useful for solving physical problems modeled using differential equations (DEs) This paper reports a PINN-based approach to solve coupled systems involving multiple governing parameters (energy and species, along with multiple interface balance equations)
  arXiv  Detail & Related papers  (2024-09-17T06:00:18Z)
• An Analysis of Physics-Informed Neural Networks [0.0]
  We present a new approach to approximating the solution to physical systems - physics-informed neural networks. The concept of artificial neural networks is introduced, the objective function is defined, and optimisation strategies are discussed. The partial differential equation is then included as a constraint in the loss function for the problem, giving the network access to knowledge of the dynamics of the physical system it is modelling.
  arXiv  Detail & Related papers  (2023-03-06T04:45:53Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equations [64.78260098263489]
  In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs) We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases. We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
  arXiv  Detail & Related papers  (2022-10-14T15:01:32Z)
• Adaptive Self-supervision Algorithms for Physics-informed Neural Networks [59.822151945132525]
  Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function. We study the impact of the location of the collocation points on the trainability of these models. We propose a novel adaptive collocation scheme which progressively allocates more collocation points to areas where the model is making higher errors.
  arXiv  Detail & Related papers  (2022-07-08T18:17:06Z)
• Critical Investigation of Failure Modes in Physics-informed Neural Networks [0.9137554315375919]
  We show that a physics-informed neural network with a composite formulation produces highly non- learned loss surfaces that are difficult to optimize. We also assess the training both approaches on two elliptic problems with increasingly complex target solutions.
  arXiv  Detail & Related papers  (2022-06-20T18:43:35Z)
• Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs) [1.7403133838762446]
  Physics-Informed Neural Networks (PINNs) are powerful approximators of solutions to nonlinear partial differential equations (PDEs) We propose an Augmented Lagrangian relaxation method for PINNs (AL-PINNs) We demonstrate through various numerical experiments that AL-PINNs yield a much smaller relative error compared with that of state-of-the-art adaptive loss-balancing algorithms.
  arXiv  Detail & Related papers  (2022-04-29T08:33:11Z)
• Physics informed neural networks for continuum micromechanics [68.8204255655161]
  Recently, physics informed neural networks have successfully been applied to a broad variety of problems in applied mathematics and engineering. Due to the global approximation, physics informed neural networks have difficulties in displaying localized effects and strong non-linear solutions by optimization. It is shown, that the domain decomposition approach is able to accurately resolve nonlinear stress, displacement and energy fields in heterogeneous microstructures obtained from real-world $mu$CT-scans.
  arXiv  Detail & Related papers  (2021-10-14T14:05:19Z)
• Characterizing possible failure modes in physics-informed neural networks [55.83255669840384]
  Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
  arXiv  Detail & Related papers  (2021-09-02T16:06:45Z)
• Conditional physics informed neural networks [85.48030573849712]
  We introduce conditional PINNs (physics informed neural networks) for estimating the solution of classes of eigenvalue problems. We show that a single deep neural network can learn the solution of partial differential equations for an entire class of problems.
  arXiv  Detail & Related papers  (2021-04-06T18:29:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.