Cauchy Random Features for Operator Learning in Sobolev Space

• URL: http://arxiv.org/abs/2503.00300v1
• Date: Sat, 01 Mar 2025 02:14:25 GMT
• Title: Cauchy Random Features for Operator Learning in Sobolev Space
• Authors: Chunyang Liao, Deanna Needell, Hayden Schaeffer,
• Abstract summary: We propose a random feature operator learning method with theoretical guarantees and error bounds.<n>Compared to kernel-based method and neural network methods, the proposed method can obtain similar or better test errors.
• Score: 8.160632714063905
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Operator learning is the approximation of operators between infinite dimensional Banach spaces using machine learning approaches. While most progress in this area has been driven by variants of deep neural networks such as the Deep Operator Network and Fourier Neural Operator, the theoretical guarantees are often in the form of a universal approximation property. However, the existence theorems do not guarantee that an accurate operator network is obtainable in practice. Motivated by the recent kernel-based operator learning framework, we propose a random feature operator learning method with theoretical guarantees and error bounds. The random feature method can be viewed as a randomized approximation of a kernel method, which significantly reduces the computation requirements for training. We provide a generalization error analysis for our proposed random feature operator learning method along with comprehensive numerical results. Compared to kernel-based method and neural network methods, the proposed method can obtain similar or better test errors across benchmarks examples with significantly reduced training times. An additional advantages it that our implementation is simple and does require costly computational resources, such as GPU.

Related papers

• Operator Learning Using Random Features: A Tool for Scientific Computing [3.745868534225104]
  Supervised operator learning centers on the use of training data to estimate maps between infinite-dimensional spaces. This paper introduces the function-valued random features method. It leads to a supervised operator learning architecture that is practical for nonlinear problems.
  arXiv  Detail & Related papers  (2024-08-12T23:10:39Z)
• Linearization Turns Neural Operators into Function-Valued Gaussian Processes [23.85470417458593]
  We introduce LUNO, a novel framework for approximate Bayesian uncertainty quantification in trained neural operators.<n>Our approach leverages model linearization to push (Gaussian) weight-space uncertainty forward to the neural operator's predictions.<n>We show that this can be interpreted as a probabilistic version of the concept of currying from functional programming, yielding a function-valued (Gaussian) random process belief.
  arXiv  Detail & Related papers  (2024-06-07T16:43:54Z)
• Neural Operators with Localized Integral and Differential Kernels [77.76991758980003]
  We present a principled approach to operator learning that can capture local features under two frameworks. We prove that we obtain differential operators under an appropriate scaling of the kernel values of CNNs. To obtain local integral operators, we utilize suitable basis representations for the kernels based on discrete-continuous convolutions.
  arXiv  Detail & Related papers  (2024-02-26T18:59:31Z)
• Function-Space Regularization in Neural Networks: A Probabilistic Perspective [51.133793272222874]
  We show that we can derive a well-motivated regularization technique that allows explicitly encoding information about desired predictive functions into neural network training. We evaluate the utility of this regularization technique empirically and demonstrate that the proposed method leads to near-perfect semantic shift detection and highly-calibrated predictive uncertainty estimates.
  arXiv  Detail & Related papers  (2023-12-28T17:50:56Z)
• Guaranteed Approximation Bounds for Mixed-Precision Neural Operators [83.64404557466528]
  We build on intuition that neural operator learning inherently induces an approximation error. We show that our approach reduces GPU memory usage by up to 50% and improves throughput by 58% with little or no reduction in accuracy.
  arXiv  Detail & Related papers  (2023-07-27T17:42:06Z)
• Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains [13.56018270837999]
  We propose a simple method to extend neural operators to arbitrary domains. An efficient implementation* of such direct spectral evaluations is coupled with existing neural operator models. We demonstrate that the proposed method allows us to extend neural operators to arbitrary point distributions with significant gains in training speed over baselines.
  arXiv  Detail & Related papers  (2023-05-31T09:01:20Z)
• An Introduction to Kernel and Operator Learning Methods for Homogenization by Self-consistent Clustering Analysis [0.48747801442240574]
  The article presents a thorough analysis on the mathematical underpinnings of the operator learning paradigm. The proposed kernel operator learning method uses graph kernel networks to come up with a mechanistic reduced order method for multiscale homogenization.
  arXiv  Detail & Related papers  (2022-12-01T02:36:16Z)
• Scalable computation of prediction intervals for neural networks via matrix sketching [79.44177623781043]
  Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure. This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
  arXiv  Detail & Related papers  (2022-05-06T13:18:31Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent [95.94432031144716]
  We propose a unified non- optimization framework for the analysis of a learning network. We show that existing guarantees can be trained unified through gradient descent.
  arXiv  Detail & Related papers  (2021-06-25T17:45:00Z)
• A Simple and General Debiased Machine Learning Theorem with Finite Sample Guarantees [4.55274575362193]
  We provide a nonasymptotic debiased machine learning theorem that encompasses any global or local functional of any machine learning algorithm. Our results culminate in a simple set of conditions that an analyst can use to translate modern learning theory rates into traditional statistical inference.
  arXiv  Detail & Related papers  (2021-05-31T17:57:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.