Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2604.15392v1
• Date: Thu, 16 Apr 2026 07:44:52 GMT
• Title: Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks
• Abstract summary: PINNs suffer from slow convergence, training instability, and reduced accuracy on challenging partial differential equations.<n>We propose a lightweight curvature-aware optimization framework that augments existing first-order gradients with an adaptive correction based on secant information.
• Score: 10.223108587188808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physics-Informed Neural Networks (PINNs) often suffer from slow convergence, training instability, and reduced accuracy on challenging partial differential equations due to the anisotropic and rapidly varying geometry of their loss landscapes. We propose a lightweight curvature-aware optimization framework that augments existing first-order optimizers with an adaptive predictive correction based on secant information. Consecutive gradient differences are used as a cheap proxy for local geometric change, together with a step-normalized secant curvature indicator to control the correction strength. The framework is plug-and-play, computationally efficient, and broadly compatible with existing optimizers, without explicitly forming second-order matrices. Experiments on diverse PDE benchmarks show consistent improvements in convergence speed, training stability, and solution accuracy over standard optimizers and strong baselines, including on the high-dimensional heat equation, Gray--Scott system, Belousov--Zhabotinsky system, and 2D Kuramoto--Sivashinsky system.

Related papers

• Physics-Informed Stochastic Configuration Machine: A Backpropagation-Free Neural Network with Fast Training for Nonlinear Differential Equations [7.927063711454594]
  Physics-In Networks (SCNs) have emerged as a transformative paradigm for solving complex differential equations.<n>Our work provides a highly efficient and scalable foundation for next-generation, real-time Scientific Machine applications.
  arXiv  Detail & Related papers  (2026-08-27T02:40:11Z)
• Evolutionary Two-Stage Hyperparameter Optimization Strategies for Physics-Informed Neural Networks [0.6924349411126935]
  We propose to investigate a two-stage approach based on evolutionary algorithms that combines parts of training and solution accuracy.<n>We evaluate on three popular problems, and our solution outperforms standard training and achieves within computational resources.
  arXiv  Detail & Related papers  (2026-06-18T16:22:52Z)
• Learning universal approximations for partial differential equations with Physics-Informed Broad Learning System [47.73467147921344]
  Partial differential equations (PDEs) play a central role in modeling complex physical, biological, and engineering systems.<n>Recent Physics-Informed Neural Networks (PINNs) offer a mesh-free alternative but frequently suffer from slow convergence and optimization instability.<n>This article proposes the Physics-Informed Broad Learning System (PIBLS), a novel backpropagation-free framework that reformulates PDE solving as a direct least-squares optimization.
  arXiv  Detail & Related papers  (2026-06-18T03:34:06Z)
• Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning [56.24532075189964]
  We propose Zeta, a dual whitening that applies coordinate whitening and spectral whitening in a strictly ordered pipeline.<n>We prove that Zeta matches or surpasses strong baselines across language modeling (0.6B to 8B parameters), mixture-of-experts architectures, and vision tasks.
  arXiv  Detail & Related papers  (2026-06-12T07:10:17Z)
• FISMO: Fisher-Structured Momentum-Orthogonalized Optimizer [30.184978506988767]
  We introduce FISMO, which incorporates anisotropic neuralotropic geometry information through Fisher information geometry.<n> FISMO achieves superior efficiency and final performance compared to established baselines.
  arXiv  Detail & Related papers  (2026-01-29T14:05:04Z)
• Data-Driven Adaptive Gradient Recovery for Unstructured Finite Volume Computations [0.0]
  We present a novel data-driven approach for enhancing gradient reconstruction in unstructured finite volume methods for hyperbolic conservation laws.<n>Our approach extends previous structured-grid methodologies to unstructured meshes through a modified DeepONet architecture.<n>The proposed algorithm is faster and more accurate than the traditional second-order finite volume solver.
  arXiv  Detail & Related papers  (2025-07-22T13:23:57Z)
• PhysicsCorrect: A Training-Free Approach for Stable Neural PDE Simulations [4.7903561901859355]
  We present PhysicsCorrect, a training-free correction framework that enforces PDE consistency at each prediction step.<n>Our key innovation is an efficient caching strategy that precomputes the Jacobian and its pseudoinverse during an offline warm-up phase.<n>Across three representative PDE systems, PhysicsCorrect reduces prediction errors by up to 100x while adding negligible inference time.
  arXiv  Detail & Related papers  (2025-07-03T01:22:57Z)
• Enabling Automatic Differentiation with Mollified Graph Neural Operators [73.52999622724101]
  We propose the mollified graph neural operator ($m$GNO), the first method to leverage automatic differentiation and compute exact gradients on arbitrary geometries.<n>For a PDE example on regular grids, $m$GNO paired with autograd reduced the L2 relative data error by 20x compared to finite differences.<n>It can also solve PDEs on unstructured point clouds seamlessly, using physics losses only, at resolutions vastly lower than those needed for finite differences to be accurate enough.
  arXiv  Detail & Related papers  (2025-04-11T06:16:30Z)
• Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks [3.758814046658822]
  Physics-Informed Neural Networks (PINNs) have revolutionized the computation PDE solutions by integrating partialmagnitude equations (PDEs) into the neural network's training process as soft constraints.<n>More, physics-informed networks (PIKANs) have also been effective and comparable in accuracy.
  arXiv  Detail & Related papers  (2025-01-22T21:19:42Z)
• Gradient-Variation Online Learning under Generalized Smoothness [56.38427425920781]
  gradient-variation online learning aims to achieve regret guarantees that scale with variations in gradients of online functions. Recent efforts in neural network optimization suggest a generalized smoothness condition, allowing smoothness to correlate with gradient norms. We provide the applications for fast-rate convergence in games and extended adversarial optimization.
  arXiv  Detail & Related papers  (2024-08-17T02:22:08Z)
• A mechanism-driven reinforcement learning framework for shape optimization of airfoils [0.32885740436059047]
  A novel mechanism-driven reinforcement learning framework is proposed for airfoil shape optimization. An efficient solver for steady equations is employed in the reinforcement learning method. A neural network architecture is designed based on an attention mechanism to make the learning process more sensitive to the minor change of the airfoil geometry.
  arXiv  Detail & Related papers  (2024-03-07T08:48:42Z)
• SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients [99.13839450032408]
  It is desired to design a universal framework for adaptive algorithms to solve general problems. In particular, our novel framework provides adaptive methods under non convergence support for setting.
  arXiv  Detail & Related papers  (2021-06-15T15:16:28Z)
• Pushing the Envelope of Rotation Averaging for Visual SLAM [69.7375052440794]
  We propose a novel optimization backbone for visual SLAM systems. We leverage averaging to improve the accuracy, efficiency and robustness of conventional monocular SLAM systems. Our approach can exhibit up to 10x faster with comparable accuracy against the state-art on public benchmarks.
  arXiv  Detail & Related papers  (2020-11-02T18:02:26Z)
• Cogradient Descent for Bilinear Optimization [124.45816011848096]
  We introduce a Cogradient Descent algorithm (CoGD) to address the bilinear problem. We solve one variable by considering its coupling relationship with the other, leading to a synchronous gradient descent. Our algorithm is applied to solve problems with one variable under the sparsity constraint.
  arXiv  Detail & Related papers  (2020-06-16T13:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.