Dimension reduction for derivative-informed operator learning: An analysis of approximation errors

• URL: http://arxiv.org/abs/2504.08730v1
• Date: Fri, 11 Apr 2025 17:56:52 GMT
• Title: Dimension reduction for derivative-informed operator learning: An analysis of approximation errors
• Authors: Dingcheng Luo, Thomas O'Leary-Roseberry, Peng Chen, Omar Ghattas,
• Abstract summary: We study the derivative-informed learning of nonlinear operators between infinite-dimensional separable Hilbert spaces by neural networks.<n>We analyze the approximation errors of neural operators in Sobolev norms over infinite-dimensional Gaussian input measures.
• Score: 3.7051887945349518
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the derivative-informed learning of nonlinear operators between infinite-dimensional separable Hilbert spaces by neural networks. Such operators can arise from the solution of partial differential equations (PDEs), and are used in many simulation-based outer-loop tasks in science and engineering, such as PDE-constrained optimization, Bayesian inverse problems, and optimal experimental design. In these settings, the neural network approximations can be used as surrogate models to accelerate the solution of the outer-loop tasks. However, since outer-loop tasks in infinite dimensions often require knowledge of the underlying geometry, the approximation accuracy of the operator's derivatives can also significantly impact the performance of the surrogate model. Motivated by this, we analyze the approximation errors of neural operators in Sobolev norms over infinite-dimensional Gaussian input measures. We focus on the reduced basis neural operator (RBNO), which uses linear encoders and decoders defined on dominant input/output subspaces spanned by reduced sets of orthonormal bases. To this end, we study two methods for generating the bases; principal component analysis (PCA) and derivative-informed subspaces (DIS), which use the dominant eigenvectors of the covariance of the data or the derivatives as the reduced bases, respectively. We then derive bounds for errors arising from both the dimension reduction and the latent neural network approximation, including the sampling errors associated with the empirical estimation of the PCA/DIS. Our analysis is validated on numerical experiments with elliptic PDEs, where our results show that bases informed by the map (i.e., DIS or output PCA) yield accurate reconstructions and generalization errors for both the operator and its derivatives, while input PCA may underperform unless ranks and training sample sizes are sufficiently large.

Related papers

• Diffeomorphic Latent Neural Operators for Data-Efficient Learning of Solutions to Partial Differential Equations [5.308435208832696]
  A computed approximation of the solution operator to a system of partial differential equations (PDEs) is needed in various areas of science and engineering.<n>We propose that in order to learn a PDE solution operator that can generalize across multiple domains without needing to sample enough data expressive enough, we can train instead a latent neural operator on just a few ground truth solution fields.
  arXiv  Detail & Related papers  (2024-11-27T03:16:00Z)
• DeltaPhi: Learning Physical Trajectory Residual for PDE Solving [54.13671100638092]
  We propose and formulate the Physical Trajectory Residual Learning (DeltaPhi) We learn the surrogate model for the residual operator mapping based on existing neural operator networks. We conclude that, compared to direct learning, physical residual learning is preferred for PDE solving.
  arXiv  Detail & Related papers  (2024-06-14T07:45:07Z)
• Derivative-enhanced Deep Operator Network [3.169190797722534]
  derivative-enhanced deep operator network (DE-DeepONet) System incorporates linear dimension reduction of high dimensional parameter input into DeepONet to reduce training cost. derivative loss can be extended to enhance other neural operators, such as the Fourier neural operator (FNO)
  arXiv  Detail & Related papers  (2024-02-29T15:18:37Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation [59.45669299295436]
  We propose a Monte Carlo PDE solver for training unsupervised neural solvers.<n>We use the PDEs' probabilistic representation, which regards macroscopic phenomena as ensembles of random particles.<n>Our experiments on convection-diffusion, Allen-Cahn, and Navier-Stokes equations demonstrate significant improvements in accuracy and efficiency.
  arXiv  Detail & Related papers  (2023-02-10T08:05:19Z)
• Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems [3.2548794659022393]
  We explore using neural operators, or neural network representations of nonlinear maps between function spaces, to accelerate infinite-dimensional Bayesian inverse problems. We show that a trained neural operator with error correction can achieve a quadratic reduction of its approximation error. We demonstrate that posterior representations of two BIPs produced using trained neural operators are greatly and consistently enhanced by error correction.
  arXiv  Detail & Related papers  (2022-10-06T15:57:22Z)
• Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning [3.7051887945349518]
  We propose derivative-informed neural operators (DINOs) DINOs approximate operators as infinite-dimensional mappings from input function spaces to output function spaces or quantities of interest. We show that the proposed DINO achieves significantly higher accuracy than neural operators trained without derivative information.
  arXiv  Detail & Related papers  (2022-06-21T21:40:01Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Physics-Informed Neural Operator for Learning Partial Differential Equations [55.406540167010014]
  PINO is the first hybrid approach incorporating data and PDE constraints at different resolutions to learn the operator. The resulting PINO model can accurately approximate the ground-truth solution operator for many popular PDE families.
  arXiv  Detail & Related papers  (2021-11-06T03:41:34Z)
• Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics [73.77459272878025]
  We propose to enhance the supervised signal in learning dynamics by pre-training a neural differential operator (NDO) NDO is pre-trained on a class of symbolic functions, and it learns the mapping between the trajectory samples of these functions to their derivatives. We provide theoretical guarantee on that the output of NDO can well approximate the ground truth derivatives by proper tuning the complexity of the library.
  arXiv  Detail & Related papers  (2021-06-08T08:04:47Z)
• Efficient Semi-Implicit Variational Inference [65.07058307271329]
  We propose an efficient and scalable semi-implicit extrapolational (SIVI) Our method maps SIVI's evidence to a rigorous inference of lower gradient values.
  arXiv  Detail & Related papers  (2021-01-15T11:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.