Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2604.05230v1
• Date: Mon, 06 Apr 2026 22:52:34 GMT
• Title: Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks
• Abstract summary: We present advanced optimization strategies to accelerate the convergence of physics-informed neural networks (PINNs)<n>We demonstrate their performance on problems including Helmholtz equation, Stokes flow, inviscid Burgers equation, Euler equations for high-speed flows.<n>We also propose new PINN-based methods for solving the inviscid Burgers equation, and compare the resulting solutions against high-order numerical methods.
• Score: 6.7049043613619155
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Efficient and robust optimization is essential for neural networks, enabling scientific machine learning models to converge rapidly to very high accuracy -- faithfully capturing complex physical behavior governed by differential equations. In this work, we present advanced optimization strategies to accelerate the convergence of physics-informed neural networks (PINNs) for challenging partial (PDEs) and ordinary differential equations (ODEs). Specifically, we provide efficient implementations of the Natural Gradient (NG) optimizer, Self-Scaling BFGS and Broyden optimizers, and demonstrate their performance on problems including the Helmholtz equation, Stokes flow, inviscid Burgers equation, Euler equations for high-speed flows, and stiff ODEs arising in pharmacokinetics and pharmacodynamics. Beyond optimizer development, we also propose new PINN-based methods for solving the inviscid Burgers and Euler equations, and compare the resulting solutions against high-order numerical methods to provide a rigorous and fair assessment. Finally, we address the challenge of scaling these quasi-Newton optimizers for batched training, enabling efficient and scalable solutions for large data-driven problems.

Related papers

• Learning universal approximations for partial differential equations with Physics-Informed Broad Learning System [47.73467147921344]
  Partial differential equations (PDEs) play a central role in modeling complex physical, biological, and engineering systems.<n>Recent Physics-Informed Neural Networks (PINNs) offer a mesh-free alternative but frequently suffer from slow convergence and optimization instability.<n>This article proposes the Physics-Informed Broad Learning System (PIBLS), a novel backpropagation-free framework that reformulates PDE solving as a direct least-squares optimization.
  arXiv  Detail & Related papers  (2026-06-18T03:34:06Z)
• Stiff Transfer Learning for Physics-Informed Neural Networks [1.5361702135159845]
  We propose a novel approach, stiff transfer learning for physics-informed neural networks (STL-PINNs) to tackle stiff ordinary differential equations (ODEs) and partial differential equations (PDEs)<n>Our methodology involves training a Multi-Head-PINN in a low-stiff regime, and obtaining the final solution in a high stiff regime by transfer learning.<n>This addresses the failure modes related to stiffness in PINNs while maintaining computational efficiency by computing "one-shot" solutions.
  arXiv  Detail & Related papers  (2025-01-28T20:27:38Z)
• Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks [3.758814046658822]
  Physics-Informed Neural Networks (PINNs) have revolutionized the computation PDE solutions by integrating partialmagnitude equations (PDEs) into the neural network's training process as soft constraints.<n>More, physics-informed networks (PIKANs) have also been effective and comparable in accuracy.
  arXiv  Detail & Related papers  (2025-01-22T21:19:42Z)
• Metamizer: a versatile neural optimizer for fast and accurate physics simulations [4.717325308876749]
  We introduce Metamizer, a novel neural network that iteratively solves a wide range of physical systems with high accuracy.<n>We demonstrate that Metamizer achieves unprecedented accuracy for deep learning based approaches.<n>Our results suggest that Metamizer could have a profound impact on future numerical solvers.
  arXiv  Detail & Related papers  (2024-10-10T11:54:31Z)
• TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision [5.283885355422517]
  Partial differential equations (PDEs) are instrumental for modeling dynamical systems in science and engineering. In this paper, we introduce the $textitTime-Evolving Natural Gradient (TENG)$, generalizing time-dependent variational principles and optimization-based time integration. Our comprehensive development includes algorithms like TENG-Euler and its high-order variants, such as TENG-Heun, tailored for enhanced precision and efficiency.
  arXiv  Detail & Related papers  (2024-04-16T17:55:31Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• Neural Basis Functions for Accelerating Solutions to High Mach Euler Equations [63.8376359764052]
  We propose an approach to solving partial differential equations (PDEs) using a set of neural networks. We regress a set of neural networks onto a reduced order Proper Orthogonal Decomposition (POD) basis. These networks are then used in combination with a branch network that ingests the parameters of the prescribed PDE to compute a reduced order approximation to the PDE.
  arXiv  Detail & Related papers  (2022-08-02T18:27:13Z)
• Speeding up Computational Morphogenesis with Online Neural Synthetic Gradients [51.42959998304931]
  A wide range of modern science and engineering applications are formulated as optimization problems with a system of partial differential equations (PDEs) as constraints. These PDE-constrained optimization problems are typically solved in a standard discretize-then-optimize approach. We propose a general framework to speed up PDE-constrained optimization using online neural synthetic gradients (ONSG) with a novel two-scale optimization scheme.
  arXiv  Detail & Related papers  (2021-04-25T22:43:51Z)
• Meta-Solver for Neural Ordinary Differential Equations [77.8918415523446]
  We investigate how the variability in solvers' space can improve neural ODEs performance. We show that the right choice of solver parameterization can significantly affect neural ODEs models in terms of robustness to adversarial attacks.
  arXiv  Detail & Related papers  (2021-03-15T17:26:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.