Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond

• URL: http://arxiv.org/abs/2608.11020v1
• Date: Tue, 11 Aug 2026 15:01:29 GMT
• Title: Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond
• Abstract summary: We investigate finite-difference derivative computation in PINNs as an alternative to automatic differentiation (AD)<n>We show that FD matches AD in accuracy on every problem while running faster across the full tested batch-size range and using substantially less GPU memory.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We systematically investigate finite-difference (FD) derivative computation in Physics-Informed Neural Networks (PINNs) as an alternative to automatic differentiation (AD). On three benchmark PDEs we show that, with a properly calibrated step size, FD matches AD in accuracy on every problem while running faster across the full tested batch-size range and using substantially less GPU memory, and that a stochastic variant we propose outperforms AD on a stationary problem. We further show that for neural architectures with inter-sample dependencies (e.g. BatchNorm, self-attention) the standard PyTorch autograd idiom is silently incorrect; the correct per-sample alternative is computationally infeasible at PINN-relevant batch sizes, while FD provides a forward-only approximation that is empirically an order of magnitude closer to the true per-sample derivative.

Related papers

• Fourier Feature Pyramids for Physics-Informed Neural Networks [47.75068288153803]
  We present an improved neural field architecture for solving partial differential equations (PDEs)<n>We call our model beignet (Bandlimited Embedding with Interpolated Grid Network)<n>We demonstrate that beignet finds significantly more accurate solutions on PDE benchmarks using fewer parameters than state-of-the-art PINN methods.
  arXiv  Detail & Related papers  (2026-05-22T23:12:56Z)
• Building Trust in PINNs: Error Estimation through Finite Difference Methods [52.92887996882041]
  Physics-informed neural networks (PINNs) constitute a flexible deep learning approach for solving partial differential equations (PDEs)<n>Despite their flexibility, PINNs offer limited insight into how their predictions deviate from the true solution, hindering trust in their prediction quality.<n>We propose a lightweight post-hoc method that addresses this gap by producing pointwise error estimates for PINN predictions.
  arXiv  Detail & Related papers  (2026-03-16T16:51:42Z)
• DInf-Grid: A Neural Differential Equation Solver with Differentiable Feature Grids [73.28614344779076]
  We present a differentiable grid-based representation for efficiently solving differential equations (DEs)<n>Our results demonstrate a 5-20x speed-up over coordinate-based methods, solving differential equations in seconds or minutes while maintaining comparable accuracy and compactness.
  arXiv  Detail & Related papers  (2026-01-15T18:59:57Z)
• Enabling Automatic Differentiation with Mollified Graph Neural Operators [73.52999622724101]
  We propose the mollified graph neural operator ($m$GNO), the first method to leverage automatic differentiation and compute exact gradients on arbitrary geometries.<n>For a PDE example on regular grids, $m$GNO paired with autograd reduced the L2 relative data error by 20x compared to finite differences.<n>It can also solve PDEs on unstructured point clouds seamlessly, using physics losses only, at resolutions vastly lower than those needed for finite differences to be accurate enough.
  arXiv  Detail & Related papers  (2025-04-11T06:16:30Z)
• Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates [34.81849268839475]
  We study the problem of efficiently computing the derivative of the fixed-point of a parametric nondifferentiable contraction map. We analyze two popular approaches: iterative differentiation (ITD) and approximate implicit differentiation (AID) We establish rates for the convergence of NSID, encompassing the best available rates in the smooth setting.
  arXiv  Detail & Related papers  (2024-03-18T11:37:53Z)
• Scaling Structured Inference with Randomization [64.18063627155128]
  We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states. Our method is widely applicable to classical DP-based inference. It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
  arXiv  Detail & Related papers  (2021-12-07T11:26:41Z)
• Efficient training of physics-informed neural networks via importance sampling [2.9005223064604078]
  Physics-In Neural Networks (PINNs) are a class of deep neural networks that are trained to compute systems governed by partial differential equations (PDEs) We show that an importance sampling approach will improve the convergence behavior of PINNs training.
  arXiv  Detail & Related papers  (2021-04-26T02:45:10Z)
• Message Passing Descent for Efficient Machine Learning [4.416484585765027]
  We propose a new iterative optimization method for the bf Data-Fitting (DF) problem in Machine Learning. The approach relies on bf Graphical Model representation of the DF problem. We suggest the bf Message Passage Descent algorithm which relies on the piece-wise-polynomial representation of the model DF function.
  arXiv  Detail & Related papers  (2021-02-16T12:22:54Z)
• Reducing the Amortization Gap in Variational Autoencoders: A Bayesian Random Function Approach [38.45568741734893]
  Inference in our GP model is done by a single feed forward pass through the network, significantly faster than semi-amortized methods. We show that our approach attains higher test data likelihood than the state-of-the-arts on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-02-05T13:01:12Z)
• Efficient Learning of Generative Models via Finite-Difference Score Matching [111.55998083406134]
  We present a generic strategy to efficiently approximate any-order directional derivative with finite difference. Our approximation only involves function evaluations, which can be executed in parallel, and no gradient computations.
  arXiv  Detail & Related papers  (2020-07-07T10:05:01Z)
• Weak SINDy For Partial Differential Equations [0.0]
  We extend our Weak SINDy (WSINDy) framework to the setting of partial differential equations (PDEs) The elimination of pointwise derivative approximations via the weak form enables effective machine-precision recovery of model coefficients from noise-free data. We demonstrate WSINDy's robustness, speed and accuracy on several challenging PDEs.
  arXiv  Detail & Related papers  (2020-07-06T16:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.