Curvature-aware dynamic precision approach for physics-informed neural networks

• URL: http://arxiv.org/abs/2606.04736v1
• Date: Wed, 03 Jun 2026 11:19:53 GMT
• Title: Curvature-aware dynamic precision approach for physics-informed neural networks
• Abstract summary: Recent studies show that PINN optimisation is sensitive to numerical precision.<n>Existing implementations commonly use either single precision (FP32) or double precision (FP64)<n>We propose a curvature-aware precision controller that adapts numerical precision during training.
• Score: 3.687363450234871
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physics-informed neural networks (PINNs) have become a promising framework for simulating partial differential equations (PDEs) by embedding physical laws directly into neural network training. However, recent studies show that PINN optimisation is sensitive to numerical precision. Existing implementations commonly use either single precision (FP32), which is computationally efficient but prone to failure modes, or double precision (FP64), which is robust but substantially expensive. This creates a trade-off between computational efficiency and numerical accuracy. To reduce the computational cost of double-precision training while retaining prediction accuracy, we propose a curvature-aware precision controller that adapts numerical precision during training rather than treating it as a fixed implementation choice. The proposed method reuses curvature information derived from the limited-memory BFGS (L-BFGS) optimiser to construct a precision controller, retaining FP32 when lower precision is sufficient and promoting computation to FP64 when the training dynamics indicate numerical sensitivity or precision-limited stagnation. We evaluate the proposed approach on four canonical PINN failure-mode benchmarks and an irradiance-driven ordinary differential equation example. We further test the proposed approach across different neural network architectures. The method consistently matches or even slightly exceeds full FP64 solution accuracy while reducing training time relative to full double-precision training on all benchmark equations. The obtained results indicate that precision sensitivity in PINN optimisation is phase-dependent, and that selectively applying higher precision only during numerically critical stages can lower computational cost without sacrificing predictive accuracy.

Related papers

• Randomized neural operator for parametric PDEs with fast training and conformal uncertainty quantification [47.720075875232226]
  We introduce PCA--RaNN, a neural operator that combines a PCA-based dimensionality reduction with fixed random features.<n>It recasts operator learning as fixed regression, reducing latent regression to three orders of magnitude.
  arXiv  Detail & Related papers  (2026-06-28T15:02:26Z)
• Precision Autotuning for Linear Solvers via Reinforcement Learning [0.0]
  We propose a reinforcement learning framework for adaptive precision tuning of linear solvers, and can be extended to general algorithms.<n>We show that our framework reduces computational cost while maintaining accuracy comparable to double-precision baselines.<n>This is the first work on precision autotuning with RL and verified on unseen datasets.
  arXiv  Detail & Related papers  (2026-01-02T15:59:42Z)
• Mixed Precision Training of Neural ODEs [1.3382837742547355]
  This paper presents a mixed precision training framework for neural ODEs.<n>It combines explicit ODE solvers with a custom backpropagation scheme.<n>It achieves approximately 50% memory reduction and up to 2x speedup while maintaining accuracy comparable to single-precision training.
  arXiv  Detail & Related papers  (2025-10-27T16:32:56Z)
• Beyond the Ideal: Analyzing the Inexact Muon Update [54.70108543057578]
  We show first analysis of the inexactized update at Muon's core.<n>We reveal a fundamental coupling between this inexactness and the optimal step size and momentum.
  arXiv  Detail & Related papers  (2025-10-22T18:01:07Z)
• CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training [73.46600457802693]
  We introduce a new method that counteracts the loss induced by quantization.<n>CAGE significantly improves upon the state-of-theart methods in terms of accuracy, for similar computational cost.<n>For QAT pre-training of Llama models, CAGE matches the accuracy achieved at 4-bits (W4A4) with the prior best method.
  arXiv  Detail & Related papers  (2025-10-21T16:33:57Z)
• Neural Precision Polarization: Simplifying Neural Network Inference with Dual-Level Precision [0.4124847249415279]
  A floating-point model can be trained in the cloud and then downloaded to an edge device. Network weights and activations are directly quantized to meet the edge devices' desired level, such as NF4 or INT8. We show that neural precision polarization enables approximately 464 TOPS per Watt MAC efficiency and reliability.
  arXiv  Detail & Related papers  (2024-11-06T16:02:55Z)
• Guaranteed Approximation Bounds for Mixed-Precision Neural Operators [83.64404557466528]
  We build on intuition that neural operator learning inherently induces an approximation error. We show that our approach reduces GPU memory usage by up to 50% and improves throughput by 58% with little or no reduction in accuracy.
  arXiv  Detail & Related papers  (2023-07-27T17:42:06Z)
• Training Neural Networks in Single vs Double Precision [8.036150169408241]
  Conjugate Gradient and RMSprop algorithms are optimized for mean square error. Experiments show that single-precision can keep up with double-precision if line search finds an improvement. For strongly nonlinear tasks, both algorithm classes find only solutions fairly poor in terms of mean square error.
  arXiv  Detail & Related papers  (2022-09-15T11:20:53Z)
• DEBOSH: Deep Bayesian Shape Optimization [48.80431740983095]
  We propose a novel uncertainty-based method tailored to shape optimization. It enables effective BO and increases the quality of the resulting shapes beyond that of state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-09-28T11:01:42Z)
• APQ: Joint Search for Network Architecture, Pruning and Quantization Policy [49.3037538647714]
  We present APQ for efficient deep learning inference on resource-constrained hardware. Unlike previous methods that separately search the neural architecture, pruning policy, and quantization policy, we optimize them in a joint manner. With the same accuracy, APQ reduces the latency/energy by 2x/1.3x over MobileNetV2+HAQ.
  arXiv  Detail & Related papers  (2020-06-15T16:09:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.