Kernel PCA with the Nystr\"om method

• URL: http://arxiv.org/abs/2109.05578v1
• Date: Sun, 12 Sep 2021 18:08:31 GMT
• Title: Kernel PCA with the Nystr\"om method
• Authors: Fredrik Hallgren
• Abstract summary: We derive kernel PCA with the Nystr"om method and study its accuracy. We present kernel principal component regression with the Nystr"om method.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Kernel methods are powerful but computationally demanding techniques for non-linear learning. A popular remedy, the Nystr\"om method has been shown to be able to scale up kernel methods to very large datasets with little loss in accuracy. However, kernel PCA with the Nystr\"om method has not been widely studied. In this paper we derive kernel PCA with the Nystr\"om method and study its accuracy, providing a finite-sample confidence bound on the difference between the Nystr\"om and standard empirical reconstruction errors. The behaviours of the method and bound are illustrated through extensive computer experiments on real-world data. As an application of the method we present kernel principal component regression with the Nystr\"om method.

Related papers

• On the Approximation of Kernel functions [0.0]
  The paper addresses approximations of the kernel itself. For the Hilbert Gauss kernel on the unit cube, the paper establishes an upper bound of the associated eigenfunctions. This improvement confirms low rank approximation methods such as the Nystr"om method.
  arXiv  Detail & Related papers  (2024-03-11T13:50:07Z)
• Geometric Learning with Positively Decomposable Kernels [6.5497574505866885]
  We propose the use of reproducing kernel Krein space (RKKS) based methods, which require only kernels that admit a positive decomposition. We show that one does not need to access this decomposition in order to learn in RKKS.
  arXiv  Detail & Related papers  (2023-10-20T21:18:04Z)
• Predicted Embedding Power Regression for Large-Scale Out-of-Distribution Detection [77.1596426383046]
  We develop a novel approach that calculates the probability of the predicted class label based on label distributions learned during the training process. Our method performs better than current state-of-the-art methods with only a negligible increase in compute cost.
  arXiv  Detail & Related papers  (2023-03-07T18:28:39Z)
• Improved Convergence Rates for Sparse Approximation Methods in Kernel-Based Learning [48.08663378234329]
  Kernel-based models such as kernel ridge regression and Gaussian processes are ubiquitous in machine learning applications. Existing sparse approximation methods can yield a significant reduction in the computational cost. We provide novel confidence intervals for the Nystr"om method and the sparse variational Gaussian processes approximation method.
  arXiv  Detail & Related papers  (2022-02-08T17:22:09Z)
• Meta-Learning Hypothesis Spaces for Sequential Decision-making [79.73213540203389]
  We propose to meta-learn a kernel from offline data (Meta-KeL) Under mild conditions, we guarantee that our estimated RKHS yields valid confidence sets. We also empirically evaluate the effectiveness of our approach on a Bayesian optimization task.
  arXiv  Detail & Related papers  (2022-02-01T17:46:51Z)
• Nystr\"om Kernel Mean Embeddings [92.10208929236826]
  We propose an efficient approximation procedure based on the Nystr"om method. It yields sufficient conditions on the subsample size to obtain the standard $n-1/2$ rate. We discuss applications of this result for the approximation of the maximum mean discrepancy and quadrature rules.
  arXiv  Detail & Related papers  (2022-01-31T08:26:06Z)
• The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methods [0.0]
  We show the importance of a data-dependent feature extraction step that is key to the obtain good performance in convolutional kernel methods. We scale this method to the challenging ImageNet dataset, showing such a simple approach can exceed all existing non-learned representation methods.
  arXiv  Detail & Related papers  (2021-01-19T09:30:58Z)
• A Bregman Method for Structure Learning on Sparse Directed Acyclic Graphs [84.7328507118758]
  We develop a Bregman proximal gradient method for structure learning. We measure the impact of curvature against a highly nonlinear iteration. We test our method on various synthetic and real sets.
  arXiv  Detail & Related papers  (2020-11-05T11:37:44Z)
• A Perturbation-Based Kernel Approximation Framework [0.0]
  We derive a perturbation-based kernel approximation framework based on classical perturbation theory. We show that our framework gives rise to new kernel approximation schemes, that can be tuned to take advantage of the structure of the approximated kernel matrix.
  arXiv  Detail & Related papers  (2020-09-07T09:14:18Z)
• Improved guarantees and a multiple-descent curve for Column Subset Selection and the Nystr\"om method [76.73096213472897]
  We develop techniques which exploit spectral properties of the data matrix to obtain improved approximation guarantees. Our approach leads to significantly better bounds for datasets with known rates of singular value decay. We show that both our improved bounds and the multiple-descent curve can be observed on real datasets simply by varying the RBF parameter.
  arXiv  Detail & Related papers  (2020-02-21T00:43:06Z)
• Face Verification via learning the kernel matrix [9.414572104591027]
  kernel function is introduced to solve the nonlinear pattern recognition problem. A promising approach is to learn the kernel from data automatically. In this paper, the nonlinear face verification via learning the kernel matrix is proposed.
  arXiv  Detail & Related papers  (2020-01-21T03:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.