Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers

• URL: http://arxiv.org/abs/2607.22004v1
• Date: Fri, 24 Jul 2026 06:07:14 GMT
• Title: Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers
• Abstract summary: Energy natural gradient descent (ENGD) aligns parameter updates with the curvature of an underlying function-space energy.<n>We introduce EMNGDfull, a manifold optimization framework for physics-informed and variational neural PDE solvers.
• Score: 0.24554686192257416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Energy natural gradient descent (ENGD) aligns parameter updates with the curvature of an underlying function-space energy, but existing formulations assume an unconstrained Euclidean parameter domain. We introduce \EMNGDfull{}, a manifold optimization framework for physics-informed and variational neural PDE solvers whose parameters lie on a Riemannian manifold. EMNGD restricts the energy-induced quadratic model to feasible tangent directions and uses retractions to preserve parameter constraints throughout optimization. Under coercivity, we prove that the push-forward of the undamped EMNGD direction is the best feasible approximation to the function-space Newton vector in the energy metric. We establish coordinate invariance, exact reduction to ENGD in Euclidean space, global first-order convergence with Armijo backtracking, and robustness to inexact tangent solves. For quadratic residual energies and generalized Gauss--Newton pullbacks, the Woodbury identity transfers the tangent system to sample space without changing the direction. Nyström approximation provides scalable sample-space solves with controlled direction error and recovers the exact direction after iterative convergence. On the evaluated neural PDE benchmarks, EMNGD achieves higher accuracy and faster convergence than the compared state-of-the-art baselines. Woodbury preserves the EMNGD direction, while scalable-solver diagnostics quantify the accuracy--cost trade-off of preconditioning and residual subsampling.

Related papers

• Equation Recast for Canonical Operator Learning Across Parametric PDEs [59.44573033543284]
  We introduce equation recast, which reformulates parametric operator learning as the learning of a single canonical operator.<n>In high-fidelity tokamak simulations for nuclear fusion, the framework unifies electron-temperature data across four device geometries.
  arXiv  Detail & Related papers  (2026-09-02T13:15:50Z)
• Higher-Order Geometric Updates for Levenberg-Marquardt Method via Riemann Normal Coordinates [1.7812378608633406]
  least-squares optimization is central to regression, physics-informed neural networks, and other machine-learning tasks.<n>RNC-LM improves convergence and robustness in curved valleys and rank-deficient problems.<n>It achieves a 34-fold speedup over standard LM on a large-scale machine-learning potential-energy-surface fitting task.
  arXiv  Detail & Related papers  (2026-07-08T16:39:03Z)
• Parameterized Representations via Implicit Stochastic Modulation for High-Dimensional and High-Order Neural PDE Solvers [0.24554686192257416]
  Implicit Modulation (PRISM) is a plug-and-play framework for parameterized high-dimensional and high-order neural PDE solvers.<n>We show that it entangles physical parameters with the high-order automatic differentiation, causing extra memory growth and parameter variance amplification.<n>Experiments with PRISM-STDE and PRISM-SDGD on nonlinear parameterized PDEs show stable zero-shot generalization, reduced memory usage, and scalability up to 100,000 dimensions on a single GPU.
  arXiv  Detail & Related papers  (2026-06-20T17:02:30Z)
• Adjoint Method versus Physics-Informed Neural Networks in PDE-Constrained Inverse Problems [48.470700002941136]
  Inverse problems governed by partial differential equations (PDEs) are central to computational mechanics and are commonly solved by adjoint-based optimization.<n> physics-informed neural networks (PINNs) have emerged as a flexible alternative.<n>We present a fair comparison of adjoint optimization and PINNs for PDE-constrained inverse problems.
  arXiv  Detail & Related papers  (2026-06-10T17:07:09Z)
• Riemannian gradient descent for Hartree-Fock theory [0.0]
  We present an optimization framework for Hartree-Fock theory formulated directly in the Sobolev space $H1$.<n>Explicit expressions for Euclidean and Riemannian gradients, tangent-space projections, and retractions are derived.<n>The proposed formulation provides a geometrically consistent and discretization-independent perspective on electronic structure optimization.
  arXiv  Detail & Related papers  (2026-03-16T19:58:42Z)
• Slack More, Predict Better: Proximal Relaxation for Probabilistic Latent Variable Model-based Soft Sensors [12.346122868771118]
  Probabilistic Latent Variable Models (NPLVMs) are a cornerstone of soft sensor modeling.<n>NPLVMs are trained using amortized variational inference, where neural networks parameterize the variational posterior.<n>We introduce KProxNPLVM, a novel NPLVM that pivots to alleviate the objective itself and improving the NPLVM's performance.
  arXiv  Detail & Related papers  (2026-03-12T02:53:39Z)
• Parallel Diffusion Solver via Residual Dirichlet Policy Optimization [88.7827307535107]
  Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature.<n>Existing solver-based acceleration methods often face significant image quality degradation under a low-dimensional budget.<n>We propose the Ensemble Parallel Direction solver (dubbed as EPD-EPr), a novel ODE solver that mitigates these errors by incorporating multiple gradient parallel evaluations in each step.
  arXiv  Detail & Related papers  (2025-12-28T05:48:55Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Neural Optimal Transport Meets Multivariate Conformal Prediction [58.43397908730771]
  We propose a framework for conditional vectorile regression (CVQR)<n>CVQR combines neural optimal transport with quantized optimization, and apply it to predictions.
  arXiv  Detail & Related papers  (2025-09-29T19:50:19Z)
• Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computation [52.73824786627612]
  This paper establishes new convergence results for textitgeodesic strongly monotone games.<n>Our key result shows that RGD attains last-iterate linear convergence in a textitgeometry-agnostic fashion.<n>Overall, this paper presents the first geometry-agnostic last-iterate convergence analysis for games beyond the Euclidean settings.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels [78.6096486885658]
  We introduce lower bounds to the linearized Laplace approximation of the marginal likelihood. These bounds are amenable togradient-based optimization and allow to trade off estimation accuracy against computational complexity.
  arXiv  Detail & Related papers  (2023-06-06T19:02:57Z)
• Sparse Representations of Positive Functions via First and Second-Order Pseudo-Mirror Descent [15.340540198612823]
  We consider expected risk problems when the range of the estimator is required to be nonnegative. We develop first and second-order variants of approximation mirror descent employing emphpseudo-gradients. Experiments demonstrate favorable performance on ingeneous Process intensity estimation in practice.
  arXiv  Detail & Related papers  (2020-11-13T21:54:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.