Fast training of accurate physics-informed neural networks without gradient descent

• URL: http://arxiv.org/abs/2405.20836v2
• Date: Tue, 30 Sep 2025 15:20:28 GMT
• Title: Fast training of accurate physics-informed neural networks without gradient descent
• Authors: Chinmay Datar, Taniya Kapoor, Abhishek Chandra, Qing Sun, Erik Lien Bolager, Iryna Burak, Anna Veselovska, Massimo Fornasier, Felix Dietrich,
• Abstract summary: We present Frozen-PINN, a novel PINN based on the principle of space-time separation.<n>On eight PDE benchmarks, Frozen-PINNs achieve superior training efficiency and accuracy over state-of-the-art PINNs.
• Score: 4.411766183442036
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Solving time-dependent Partial Differential Equations (PDEs) is one of the most critical problems in computational science. While Physics-Informed Neural Networks (PINNs) offer a promising framework for approximating PDE solutions, their accuracy and training speed are limited by two core barriers: gradient-descent-based iterative optimization over complex loss landscapes and non-causal treatment of time as an extra spatial dimension. We present Frozen-PINN, a novel PINN based on the principle of space-time separation that leverages random features instead of training with gradient descent, and incorporates temporal causality by construction. On eight PDE benchmarks, including challenges such as extreme advection speeds, shocks, and high dimensionality, Frozen-PINNs achieve superior training efficiency and accuracy over state-of-the-art PINNs, often by several orders of magnitude. Our work addresses longstanding training and accuracy bottlenecks of PINNs, delivering quickly trainable, highly accurate, and inherently causal PDE solvers, a combination that prior methods could not realize. Our approach challenges the reliance of PINNs on stochastic gradient-descent-based methods and specialized hardware, leading to a paradigm shift in PINN training and providing a challenging benchmark for the community.

Related papers

• High precision PINNs in unbounded domains: application to singularity formulation in PDEs [83.50980325611066]
  We study the choices of neural network ansatz, sampling strategy, and optimization algorithm.<n>For 1D Burgers equation, our framework can lead to a solution with very high precision.<n>For the 2D Boussinesq equation, we obtain a solution whose loss is $4$ digits smaller than that obtained in citewang2023asymptotic with fewer training steps.
  arXiv  Detail & Related papers  (2025-06-24T02:01:44Z)
• Global Convergence of Adjoint-Optimized Neural PDEs [0.0]
  We study the convergence of the adjoint gradient descent optimization method for training neural-network PDE models in the limit where both the number of hidden units and the training time tend to infinity.<n>Specifically, for a general class of nonlinear parabolic PDEs with a neural network embedded in the source term, we prove the trained neural-network PDE solution to the target data (i.e., a global minimizer)<n>The global convergence proof poses a unique mathematical challenge that is not encountered in finite-dimensional convergence analyses.
  arXiv  Detail & Related papers  (2025-06-16T16:00:00Z)
• Solving Partial Differential Equations with Random Feature Models [1.3597551064547502]
  We introduce a random feature based framework toward efficiently solving PDEs. In contrast to the state-of-the-art solvers that face challenges with a large number of collocation points, our proposed method reduces the computational complexity.
  arXiv  Detail & Related papers  (2024-12-31T05:48:31Z)
• Solving Poisson Equations using Neural Walk-on-Spheres [80.1675792181381]
  We propose Neural Walk-on-Spheres (NWoS), a novel neural PDE solver for the efficient solution of high-dimensional Poisson equations. We demonstrate the superiority of NWoS in accuracy, speed, and computational costs.
  arXiv  Detail & Related papers  (2024-06-05T17:59:22Z)
• Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numerics [5.380276949049726]
  We develop and analyze an efficient high-dimensional Partial Differential Equations solver based on Deep Learning (DL) We show, both theoretically and numerically, that it can compete with a novel stable and accurate compressive spectral collocation method.
  arXiv  Detail & Related papers  (2024-06-03T17:16:11Z)
• Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data [0.0]
  We represent the PDE as a neural network and use an intermediate state representation similar to a Physics-Informed Neural Network (PINN) We present a penalty method and a widely used trust-region barrier method to solve this constrained optimization problem. Our results on the Burgers' and the Korteweg-De Vreis equations demonstrate that the latter constrained method outperforms the penalty method.
  arXiv  Detail & Related papers  (2024-05-30T01:55:44Z)
• Efficient Discrete Physics-informed Neural Networks for Addressing Evolutionary Partial Differential Equations [7.235476098729406]
  Physics-informed neural networks (PINNs) have shown promising potential for solving partial differential equations (PDEs) using deep learning. PINNs may violate the temporal causality property since all the temporal features in the PINNs loss are trained simultaneously. This paper proposes to use implicit time differencing schemes to enforce temporal causality, and use transfer learning to sequentially update the PINNs in space as surrogates for PDE solutions in different time frames.
  arXiv  Detail & Related papers  (2023-12-22T11:09:01Z)
• A Sequential Meta-Transfer (SMT) Learning to Combat Complexities of Physics-Informed Neural Networks: Application to Composites Autoclave Processing [1.6317061277457001]
  PINNs have gained popularity in solving nonlinear partial differential equations. PINNs are designed to approximate a specific realization of a given PDE system. They lack the necessary generalizability to efficiently adapt to new system configurations.
  arXiv  Detail & Related papers  (2023-08-12T02:46:54Z)
• A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks [52.5899851000193]
  We develop an ODE based IVP solver which prevents the network from getting ill-conditioned and runs in time linear in the number of parameters. We show that current methods based on this approach suffer from two key issues. First, following the ODE produces an uncontrolled growth in the conditioning of the problem, ultimately leading to unacceptably large numerical errors.
  arXiv  Detail & Related papers  (2023-04-28T17:28:18Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions [22.514769448363754]
  Training challenges in PINNs and XPINNs for time-dependent PDEs are discussed. We propose a new stacked-decomposition method that bridges the gap between PINNs and XPINNs. We also formulate a new time-sweeping collocation point algorithm inspired by the previous PINNs causality.
  arXiv  Detail & Related papers  (2023-02-28T01:19:21Z)
• PhyGNNet: Solving spatiotemporal PDEs with Physics-informed Graph Neural Network [12.385926494640932]
  We propose PhyGNNet for solving partial differential equations on the basics of a graph neural network. In particular, we divide the computing area into regular grids, define partial differential operators on the grids, then construct pde loss for the network to optimize to build PhyGNNet model.
  arXiv  Detail & Related papers  (2022-08-07T13:33:34Z)
• PIXEL: Physics-Informed Cell Representations for Fast and Accurate PDE Solvers [4.1173475271436155]
  We propose a new kind of data-driven PDEs solver, physics-informed cell representations (PIXEL) PIXEL elegantly combines classical numerical methods and learning-based approaches. We show that PIXEL achieves fast convergence speed and high accuracy.
  arXiv  Detail & Related papers  (2022-07-26T10:46:56Z)
• Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture [77.59766598165551]
  Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. Here, we propose Auto-PINN, which employs Neural Architecture Search (NAS) techniques to PINN design. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs.
  arXiv  Detail & Related papers  (2022-05-27T03:24:31Z)
• Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method [70.19159220248805]
  Physics-informed neural networks (PINNs) provide a deep learning framework for numerically solving partial differential equations (PDEs) We propose the generative adversarial neural network (GA-PINN), which integrates the generative adversarial (GA) mechanism with the structure of PINNs. Inspired from the weighting strategy of the Adaboost method, we then introduce a point-weighting (PW) method to improve the training efficiency of PINNs.
  arXiv  Detail & Related papers  (2022-05-18T06:50:44Z)
• Improved Training of Physics-Informed Neural Networks with Model Ensembles [81.38804205212425]
  We propose to expand the solution interval gradually to make the PINN converge to the correct solution. All ensemble members converge to the same solution in the vicinity of observed data. We show experimentally that the proposed method can improve the accuracy of the found solution.
  arXiv  Detail & Related papers  (2022-04-11T14:05:34Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Physics-Informed Neural Operator for Learning Partial Differential Equations [55.406540167010014]
  PINO is the first hybrid approach incorporating data and PDE constraints at different resolutions to learn the operator. The resulting PINO model can accurately approximate the ground-truth solution operator for many popular PDE families.
  arXiv  Detail & Related papers  (2021-11-06T03:41:34Z)
• PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs [8.220908558735884]
  Partial differential equations (PDEs) play a fundamental role in modeling and simulating problems across a wide range of disciplines. Recent advances in deep learning have shown the great potential of physics-informed neural networks (NNs) to solve PDEs as a basis for data-driven inverse analysis. We propose the novel physics-informed convolutional-recurrent learning architectures (PhyCRNet and PhCRyNet-s) for solving PDEs without any labeled data.
  arXiv  Detail & Related papers  (2021-06-26T22:22:19Z)
• Solving PDEs on Unknown Manifolds with Machine Learning [8.220217498103315]
  This paper presents a mesh-free computational framework and machine learning theory for solving elliptic PDEs on unknown manifold. We show that the proposed NN solver can robustly generalize the PDE on new data points with errors that are almost identical to generalizations on new data points.
  arXiv  Detail & Related papers  (2021-06-12T03:55:15Z)
• Efficient training of physics-informed neural networks via importance sampling [2.9005223064604078]
  Physics-In Neural Networks (PINNs) are a class of deep neural networks that are trained to compute systems governed by partial differential equations (PDEs) We show that an importance sampling approach will improve the convergence behavior of PINNs training.
  arXiv  Detail & Related papers  (2021-04-26T02:45:10Z)
• Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
  Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
  arXiv  Detail & Related papers  (2020-09-08T13:26:51Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.