Notes on Kernel Methods in Machine Learning

• URL: http://arxiv.org/abs/2511.14485v1
• Date: Tue, 18 Nov 2025 13:29:07 GMT
• Title: Notes on Kernel Methods in Machine Learning
• Authors: Diego Armando Pérez-Rosero, Danna Valentina Salazar-Dubois, Juan Camilo Lugo-Rojas, Andrés Marino Álvarez-Meza, Germán Castellanos-Dominguez,
• Abstract summary: We develop the theory of positive definite kernels, reproducing kernel Hilbert spaces (RKHS), and Hilbert-Schmidt operators.<n>We also introduce kernel density estimation, kernel embeddings of distributions, and the Maximum Mean Discrepancy (MMD)<n>The exposition is designed to serve as a foundation for more advanced topics, including Gaussian processes, kernel Bayesian inference, and functional analytic approaches to modern machine learning.
• Score: 0.8435614464136675
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: These notes provide a self-contained introduction to kernel methods and their geometric foundations in machine learning. Starting from the construction of Hilbert spaces, we develop the theory of positive definite kernels, reproducing kernel Hilbert spaces (RKHS), and Hilbert-Schmidt operators, emphasizing their role in statistical estimation and representation of probability measures. Classical concepts such as covariance, regression, and information measures are revisited through the lens of Hilbert space geometry. We also introduce kernel density estimation, kernel embeddings of distributions, and the Maximum Mean Discrepancy (MMD). The exposition is designed to serve as a foundation for more advanced topics, including Gaussian processes, kernel Bayesian inference, and functional analytic approaches to modern machine learning.

Related papers

• Universal kernels via harmonic analysis on Riemannian symmetric spaces [3.7141182051230914]
  kernels are of fundamental importance in the theoretical underpinning of kernel methods in machine learning.<n>We establish tools for investigating universality properties of kernels in symmetric spaces.<n>We provide theoretical justification for their use in applications involving manifold-valued data.
  arXiv  Detail & Related papers  (2025-06-24T02:03:25Z)
• Reproducing kernel Hilbert spaces in the mean field limit [6.844996517347866]
  kernels are function spaces generated by kernels, so called reproducing kernel Hilbert spaces. We show the rigorous mean field limit of kernels and provide a detailed analysis of the limiting reproducing Hilbert space.
  arXiv  Detail & Related papers  (2023-02-28T09:46:44Z)
• Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces [43.877478563933316]
  In to symmetries is one of the most fundamental forms of prior information one can consider. In this work, we develop constructive and practical techniques for building stationary Gaussian processes on a very large class of non-Euclidean spaces.
  arXiv  Detail & Related papers  (2023-01-30T17:27:12Z)
• Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I: the compact case [43.877478563933316]
  In to symmetries is one of the most fundamental forms of prior information one can consider. In this work, we develop constructive and practical techniques for building stationary Gaussian processes on a very large class of non-Euclidean spaces.
  arXiv  Detail & Related papers  (2022-08-31T16:40:40Z)
• Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces [96.53463532832939]
  We develop techniques for broadening the applicability of Gaussian processes. We introduce a wide class of efficient approximations built from this viewpoint. We develop a collection of Gaussian process models over non-Euclidean spaces.
  arXiv  Detail & Related papers  (2022-02-22T01:42:57Z)
• Meta-Learning Hypothesis Spaces for Sequential Decision-making [79.73213540203389]
  We propose to meta-learn a kernel from offline data (Meta-KeL) Under mild conditions, we guarantee that our estimated RKHS yields valid confidence sets. We also empirically evaluate the effectiveness of our approach on a Bayesian optimization task.
  arXiv  Detail & Related papers  (2022-02-01T17:46:51Z)
• Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Equivariant Projected Kernels [108.60991563944351]
  We present a recipe for constructing gauge equivariant kernels, which induce vector-valued Gaussian processes coherent with geometry. We extend standard Gaussian process training methods, such as variational inference, to this setting.
  arXiv  Detail & Related papers  (2021-10-27T13:31:10Z)
• A Note on Optimizing Distributions using Kernel Mean Embeddings [94.96262888797257]
  Kernel mean embeddings represent probability measures by their infinite-dimensional mean embeddings in a reproducing kernel Hilbert space. We show that when the kernel is characteristic, distributions with a kernel sum-of-squares density are dense. We provide algorithms to optimize such distributions in the finite-sample setting.
  arXiv  Detail & Related papers  (2021-06-18T08:33:45Z)
• Reproducing Kernel Hilbert Space, Mercer's Theorem, Eigenfunctions, Nystr\"om Method, and Use of Kernels in Machine Learning: Tutorial and Survey [5.967999555890417]
  We start with reviewing the history of kernels in functional analysis and machine learning. We introduce types of use of kernels in machine learning including kernel methods, kernel learning by semi-definite programming, Hilbert-Schmidt independence criterion, maximum mean discrepancy, kernel mean embedding, and kernel dimensionality reduction. This paper can be useful for various fields of science including machine learning, dimensionality reduction, functional analysis in mathematics, and mathematical physics in quantum mechanics.
  arXiv  Detail & Related papers  (2021-06-15T21:29:12Z)
• Advanced Stationary and Non-Stationary Kernel Designs for Domain-Aware Gaussian Processes [0.0]
  We propose advanced kernel designs that only allow for functions with certain desirable characteristics to be elements of the reproducing kernel Hilbert space (RKHS) We will show the impact of advanced kernel designs on Gaussian processes using several synthetic and two scientific data sets.
  arXiv  Detail & Related papers  (2021-02-05T22:07:56Z)
• A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces [53.47210316424326]
  KeRNS is an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes. We prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time.
  arXiv  Detail & Related papers  (2020-07-09T21:37:13Z)
• Mat\'ern Gaussian processes on Riemannian manifolds [81.15349473870816]
  We show how to generalize the widely-used Mat'ern class of Gaussian processes. We also extend the generalization from the Mat'ern to the widely-used squared exponential process.
  arXiv  Detail & Related papers  (2020-06-17T21:05:42Z)
• Lecture notes: Efficient approximation of kernel functions [4.177892889752434]
  Notes endeavour to collect in one place the mathematical background required to understand the properties of kernels in general. We briefly motivate the use of kernels in Machine Learning with the example of the support vector machine. After a brief discussion of Hilbert spaces, including the Reproducing Kernel Hilbert Space construction, we present Mercer's theorem.
  arXiv  Detail & Related papers  (2020-05-04T15:30:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.