Learning in RKHM: a $C^*$-Algebraic Twist for Kernel Machines

• URL: http://arxiv.org/abs/2210.11855v3
• Date: Wed, 26 Jun 2024 00:29:54 GMT
• Title: Learning in RKHM: a $C^*$-Algebraic Twist for Kernel Machines
• Authors: Yuka Hashimoto, Masahiro Ikeda, Hachem Kadri,
• Abstract summary: Supervised learning in reproducing kernel Hilbert space (RKHS) and vector-valued RKHS (vvRKHS) has been investigated for more than 30 years. We provide a new twist by generalizing supervised learning in RKHS and vvRKHS to reproducing kernel Hilbert $C*$-module (RKHM) We show how to construct effective positive-definite kernels by considering the perspective of $C*$-algebra.
• Score: 13.23700804428796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Supervised learning in reproducing kernel Hilbert space (RKHS) and vector-valued RKHS (vvRKHS) has been investigated for more than 30 years. In this paper, we provide a new twist to this rich literature by generalizing supervised learning in RKHS and vvRKHS to reproducing kernel Hilbert $C^*$-module (RKHM), and show how to construct effective positive-definite kernels by considering the perspective of $C^*$-algebra. Unlike the cases of RKHS and vvRKHS, we can use $C^*$-algebras to enlarge representation spaces. This enables us to construct RKHMs whose representation power goes beyond RKHSs, vvRKHSs, and existing methods such as convolutional neural networks. Our framework is suitable, for example, for effectively analyzing image data by allowing the interaction of Fourier components.

Related papers

• Convolutional Filtering with RKHS Algebras [110.06688302593349]
  We develop a theory of convolutional signal processing and neural networks for Reproducing Kernel Hilbert Spaces (RKHS) We show that any RKHS allows the formal definition of multiple algebraic convolutional models. We present a set of numerical experiments on real data in which wireless coverage is predicted from measurements captured by unmaned aerial vehicles.
  arXiv  Detail & Related papers  (2024-11-02T18:53:44Z)
• Geometric Learning with Positively Decomposable Kernels [6.5497574505866885]
  We propose the use of reproducing kernel Krein space (RKKS) based methods, which require only kernels that admit a positive decomposition. We show that one does not need to access this decomposition in order to learn in RKKS.
  arXiv  Detail & Related papers  (2023-10-20T21:18:04Z)
• Deep Learning with Kernels through RKHM and the Perron-Frobenius Operator [14.877070496733966]
  Reproducing kernel Hilbert $C*$-module (RKHM) is a generalization of reproducing kernel Hilbert space (RKHS) by means of $C*$-algebra. We derive a new Rademacher generalization bound in this setting and provide a theoretical interpretation of benign overfitting by means of Perron-Frobenius operators.
  arXiv  Detail & Related papers  (2023-05-23T01:38:41Z)
• Neural Networks can Learn Representations with Gradient Descent [68.95262816363288]
  In specific regimes, neural networks trained by gradient descent behave like kernel methods. In practice, it is known that neural networks strongly outperform their associated kernels.
  arXiv  Detail & Related papers  (2022-06-30T09:24:02Z)
• DASHA: Distributed Nonconvex Optimization with Communication Compression, Optimal Oracle Complexity, and No Client Synchronization [77.34726150561087]
  We develop and analyze DASHA: a new family of methods for noneps distributed optimization problems. Unlike MARINA, the new methods DASHA, DASHA-MVR send compressed vectors only and never synchronize the nodes, which makes them more practical for learning.
  arXiv  Detail & Related papers  (2022-02-02T20:10:40Z)
• Neural Fields as Learnable Kernels for 3D Reconstruction [101.54431372685018]
  We present a novel method for reconstructing implicit 3D shapes based on a learned kernel ridge regression. Our technique achieves state-of-the-art results when reconstructing 3D objects and large scenes from sparse oriented points.
  arXiv  Detail & Related papers  (2021-11-26T18:59:04Z)
• Scaling Neural Tangent Kernels via Sketching and Random Features [53.57615759435126]
  Recent works report that NTK regression can outperform finitely-wide neural networks trained on small-scale datasets. We design a near input-sparsity time approximation algorithm for NTK, by sketching the expansions of arc-cosine kernels. We show that a linear regressor trained on our CNTK features matches the accuracy of exact CNTK on CIFAR-10 dataset while achieving 150x speedup.
  arXiv  Detail & Related papers  (2021-06-15T04:44:52Z)
• Reproducing kernel Hilbert C*-module and kernel mean embeddings [12.268585269921404]
  We propose a novel data analysis framework with reproducing kernel Hilbert $C*$-module (RKHM) and kernel mean embedding (KME) in RKHM. We show a branch of theories for RKHM to apply to data analysis, including the representer theorem, and the injectivity and generalize of the proposed KME.
  arXiv  Detail & Related papers  (2021-01-27T14:02:18Z)
• On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces [208.67848059021915]
  We study the exploration-exploitation tradeoff at the core of reinforcement learning. In particular, we prove that the complexity of the function class $mathcalF$ characterizes the complexity of the function. Our regret bounds are independent of the number of episodes.
  arXiv  Detail & Related papers  (2020-11-09T18:32:22Z)
• Analysis via Orthonormal Systems in Reproducing Kernel Hilbert $C^*$-Modules and Applications [12.117553807794382]
  We propose a novel data analysis framework with reproducing kernel Hilbert $C*$-module (RKHM) We show the theoretical validity for the construction of orthonormal systems in Hilbert $C*$-modules, and derive concrete procedures for orthonormalization in RKHMs. We apply those to generalize with RKHM kernel principal component analysis and the analysis of dynamical systems with Perron-Frobenius operators.
  arXiv  Detail & Related papers  (2020-03-02T10:01:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.