PDE-aware Optimizer for Physics-informed Neural Networks

• URL: http://arxiv.org/abs/2507.08118v1
• Date: Thu, 10 Jul 2025 19:07:55 GMT
• Title: PDE-aware Optimizer for Physics-informed Neural Networks
• Authors: Hardik Shukla, Manurag Khullar, Vismay Churiwala,
• Abstract summary: We propose a PDE-aware that adapts parameter updates based on the variance of per-sample PDE residual.<n>This method addresses gradient misalignment without incurring the heavy computational costs of second-order gradients such as SOAP.<n>Our results demonstrate the effectiveness of PDE residual-aware adaptivity in enhancing stability in PINNs training.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs) by embedding physical constraints into the loss function. However, standard optimizers such as Adam often struggle to balance competing loss terms, particularly in stiff or ill-conditioned systems. In this work, we propose a PDE-aware optimizer that adapts parameter updates based on the variance of per-sample PDE residual gradients. This method addresses gradient misalignment without incurring the heavy computational costs of second-order optimizers such as SOAP. We benchmark the PDE-aware optimizer against Adam and SOAP on 1D Burgers', Allen-Cahn and Korteweg-de Vries(KdV) equations. Across both PDEs, the PDE-aware optimizer achieves smoother convergence and lower absolute errors, particularly in regions with sharp gradients. Our results demonstrate the effectiveness of PDE residual-aware adaptivity in enhancing stability in PINNs training. While promising, further scaling on larger architectures and hardware accelerators remains an important direction for future research.

Related papers

• Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
  Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations.<n>We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
  arXiv  Detail & Related papers  (2024-11-28T13:16:20Z)
• P$^2$C$^2$Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics [38.53011684603394]
  We introduce a new PDE-Preserved Coarse Correction Network (P$2$C$2$Net) to solve PDE problems on coarse mesh grids in small data regimes. The model consists of two synergistic modules: (1) a trainable PDE block that learns to update the coarse solution (i.e., the system state), based on a high-order numerical scheme with boundary condition encoding, and (2) a neural network block that consistently corrects the solution on the fly.
  arXiv  Detail & Related papers  (2024-10-29T14:45:07Z)
• Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methods [14.791541465418263]
  We propose learning a solver, i.e., solving partial differential equations (PDEs) using a physics-informed iterative algorithm trained on data.<n>Our method learns to condition a gradient descent algorithm that automatically adapts to each PDE instance.<n>We demonstrate the effectiveness of our approach through empirical experiments on multiple datasets.
  arXiv  Detail & Related papers  (2024-10-09T12:28:32Z)
• Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers [55.0876373185983]
  We present the Universal PDE solver (Unisolver) capable of solving a wide scope of PDEs. Our key finding is that a PDE solution is fundamentally under the control of a series of PDE components. Unisolver achieves consistent state-of-the-art results on three challenging large-scale benchmarks.
  arXiv  Detail & Related papers  (2024-05-27T15:34:35Z)
• BiLO: Bilevel Local Operator Learning for PDE inverse problems [10.082649871368409]
  We propose a new neural network based method for solving inverse problems for partial differential equations (PDEs)<n>At the upper level, we minimize the data loss with respect to the PDE parameters.<n>At the lower level, we train a neural network to locally approximate the PDE solution operator in the neighborhood of a given set of PDE parameters.
  arXiv  Detail & Related papers  (2024-04-27T06:06:41Z)
• Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
  We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
  arXiv  Detail & Related papers  (2023-11-30T22:34:57Z)
• Bi-level Physics-Informed Neural Networks for PDE Constrained Optimization using Broyden's Hypergradients [29.487375792661005]
  We present a novel bi-level optimization framework to solve PDE constrained optimization problems. For the inner loop optimization, we adopt PINNs to solve the PDE constraints only. For the outer loop, we design a novel method by using Broyden'simat method based on the Implicit Function Theorem.
  arXiv  Detail & Related papers  (2022-09-15T06:21:24Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Lie Point Symmetry Data Augmentation for Neural PDE Solvers [69.72427135610106]
  We present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity. In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations. We show how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
  arXiv  Detail & Related papers  (2022-02-15T18:43:17Z)
• Physics-Informed Neural Operator for Learning Partial Differential Equations [55.406540167010014]
  PINO is the first hybrid approach incorporating data and PDE constraints at different resolutions to learn the operator. The resulting PINO model can accurately approximate the ground-truth solution operator for many popular PDE families.
  arXiv  Detail & Related papers  (2021-11-06T03:41:34Z)
• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems [2.0062792633909026]
  Deep learning has been shown to be an effective tool in solving partial differential equations (PDEs) through physics-informed neural networks (PINNs) PINNs embed the PDE residual into the loss function of the neural network, and have been successfully employed to solve diverse forward and inverse PDE problems. Here, we propose a new method, gradient-enhanced physics-informed neural networks (gPINNs) for improving the accuracy and training efficiency of PINNs.
  arXiv  Detail & Related papers  (2021-11-01T18:01:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.