Data adaptive RKHS Tikhonov regularization for learning kernels in operators

• URL: http://arxiv.org/abs/2203.03791v1
• Date: Tue, 8 Mar 2022 01:08:35 GMT
• Title: Data adaptive RKHS Tikhonov regularization for learning kernels in operators
• Authors: Fei Lu, Quanjun Lang and Qingci An
• Abstract summary: We present DARTR: a Data Adaptive RKHS Tikhonov Regularization method for the linear inverse problem of nonparametric learning of function parameters in operators. A key ingredient is a system intrinsic data-adaptive (SIDA) RKHS, whose norm restricts the learning to take place in the function space of identifiability.
• Score: 1.5039745292757671
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present DARTR: a Data Adaptive RKHS Tikhonov Regularization method for the linear inverse problem of nonparametric learning of function parameters in operators. A key ingredient is a system intrinsic data-adaptive (SIDA) RKHS, whose norm restricts the learning to take place in the function space of identifiability. DARTR utilizes this norm and selects the regularization parameter by the L-curve method. We illustrate its performance in examples including integral operators, nonlinear operators and nonlocal operators with discrete synthetic data. Numerical results show that DARTR leads to an accurate estimator robust to both numerical error due to discrete data and noise in data, and the estimator converges at a consistent rate as the data mesh refines under different levels of noises, outperforming two baseline regularizers using $l^2$ and $L^2$ norms.

Related papers

• Learning Dynamical Systems from Noisy Data with Inverse-Explicit Integrators [0.0]
  We introduce the mean inverse integrator (MII) to increase the accuracy when training neural networks to approximate vector fields from noisy data. We show that the class of mono-implicit Runge-Kutta methods (MIRK) has particular advantages when used in connection with MII.
  arXiv  Detail & Related papers  (2023-06-06T09:50:38Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• A Data-Adaptive Prior for Bayesian Learning of Kernels in Operators [4.0868955102696365]
  We introduce a data-adaptive prior to achieve a stable posterior whose mean always has a small noise limit. Numerical tests show that a fixed prior can lead to a divergent posterior mean in the presence of any of the four types of errors.
  arXiv  Detail & Related papers  (2022-12-29T03:15:51Z)
• Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces [52.35063796758121]
  We formalize a framework to learn the Koopman operator from finite data trajectories of the dynamical system. We link the risk with the estimation of the spectral decomposition of the Koopman operator. Our results suggest RRR might be beneficial over other widely used estimators.
  arXiv  Detail & Related papers  (2022-05-27T14:57:48Z)
• Nonparametric learning of kernels in nonlocal operators [6.314604944530131]
  We provide a rigorous identifiability analysis and convergence study for the learning of kernels in nonlocal operators. We propose a nonparametric regression algorithm with a novel data adaptive RKHS Tikhonov regularization method based on the function space of identifiability.
  arXiv  Detail & Related papers  (2022-05-23T02:47:55Z)
• A Priori Denoising Strategies for Sparse Identification of Nonlinear Dynamical Systems: A Comparative Study [68.8204255655161]
  We investigate and compare the performance of several local and global smoothing techniques to a priori denoise the state measurements. We show that, in general, global methods, which use the entire measurement data set, outperform local methods, which employ a neighboring data subset around a local point.
  arXiv  Detail & Related papers  (2022-01-29T23:31:25Z)
• On the Double Descent of Random Features Models Trained with SGD [78.0918823643911]
  We study properties of random features (RF) regression in high dimensions optimized by gradient descent (SGD) We derive precise non-asymptotic error bounds of RF regression under both constant and adaptive step-size SGD setting. We observe the double descent phenomenon both theoretically and empirically.
  arXiv  Detail & Related papers  (2021-10-13T17:47:39Z)
• Convergence Rates for Learning Linear Operators from Noisy Data [6.4423565043274795]
  We study the inverse problem of learning a linear operator on a space from its noisy pointwise evaluations on random input data. We establish posterior contraction rates with respect to a family of Bochner norms as the number of data tend to infinity lower on the estimation error. These convergence rates highlight and quantify the difficulty of learning linear operators in comparison with the learning of bounded or compact ones.
  arXiv  Detail & Related papers  (2021-08-27T22:09:53Z)
• Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification [69.26747803963907]
  Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters. First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension. We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
  arXiv  Detail & Related papers  (2021-04-11T16:37:32Z)
• Understanding Implicit Regularization in Over-Parameterized Single Index Model [55.41685740015095]
  We design regularization-free algorithms for the high-dimensional single index model. We provide theoretical guarantees for the induced implicit regularization phenomenon.
  arXiv  Detail & Related papers  (2020-07-16T13:27:47Z)
• A Doubly Regularized Linear Discriminant Analysis Classifier with Automatic Parameter Selection [24.027886914804775]
  Linear discriminant analysis (LDA) based classifiers tend to falter in many practical settings where the training data size is smaller than, or comparable to, the number of features. We propose a doubly regularized LDA classifier that we denote as R2LDA. Results obtained from both synthetic and real data demonstrate the consistency and effectiveness of the proposed R2LDA approach.
  arXiv  Detail & Related papers  (2020-04-28T07:09:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.