Multiple Locally Linear Kernel Machines

• URL: http://arxiv.org/abs/2401.09629v1
• Date: Wed, 17 Jan 2024 22:43:00 GMT
• Title: Multiple Locally Linear Kernel Machines
• Authors: David Picard
• Abstract summary: We propose a new non-linear classifier based on a combination of locally linear classifiers. We provide a scalable generic MKL training algorithm handling streaming kernels.
• Score: 14.282867638200699
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we propose a new non-linear classifier based on a combination of locally linear classifiers. A well known optimization formulation is given as we cast the problem in a $\ell_1$ Multiple Kernel Learning (MKL) problem using many locally linear kernels. Since the number of such kernels is huge, we provide a scalable generic MKL training algorithm handling streaming kernels. With respect to the inference time, the resulting classifier fits the gap between high accuracy but slow non-linear classifiers (such as classical MKL) and fast but low accuracy linear classifiers.

Related papers

• Fast and Scalable Multi-Kernel Encoder Classifier [4.178980693837599]
  The proposed method facilitates fast and scalable kernel matrix embedding, and seamlessly integrates multiple kernels to enhance the learning process. Our theoretical analysis offers a population-level characterization of this approach using random variables.
  arXiv  Detail & Related papers  (2024-06-04T10:34:40Z)
• Fast Evaluation of Additive Kernels: Feature Arrangement, Fourier Methods, and Kernel Derivatives [0.5735035463793009]
  We present a technique based on the non-equispaced fast Fourier transform (NFFT) with rigorous error analysis. We show that this approach is also well suited to allow the approximation of the matrix that arises when the kernel is differentiated. We illustrate the performance of the additive kernel scheme with fast matrix vector products on a number of data sets.
  arXiv  Detail & Related papers  (2024-04-26T11:50:16Z)
• Generating Multi-Center Classifier via Conditional Gaussian Distribution [7.77615886942767]
  In real-world data, one class can contain several local clusters, e.g., birds of different poses. We create a conditional Gaussian distribution for each class and then sample multiple sub-centers. This approach allows the model to capture intra-class local structures more efficiently.
  arXiv  Detail & Related papers  (2024-01-29T08:06:33Z)
• Local Sample-weighted Multiple Kernel Clustering with Consensus Discriminative Graph [73.68184322526338]
  Multiple kernel clustering (MKC) is committed to achieving optimal information fusion from a set of base kernels. This paper proposes a novel local sample-weighted multiple kernel clustering model. Experimental results demonstrate that our LSWMKC possesses better local manifold representation and outperforms existing kernel or graph-based clustering algo-rithms.
  arXiv  Detail & Related papers  (2022-07-05T05:00:38Z)
• Generalized Reference Kernel for One-class Classification [100.53532594448048]
  We formulate a new generalized reference kernel to improve the original base kernel using a set of reference vectors. Our analysis and experimental results show that the new formulation provides approaches to regularize, adjust the rank, and incorporate additional information into the kernel itself.
  arXiv  Detail & Related papers  (2022-05-01T18:36:55Z)
• High-Dimensional Sparse Bayesian Learning without Covariance Matrices [66.60078365202867]
  We introduce a new inference scheme that avoids explicit construction of the covariance matrix. Our approach couples a little-known diagonal estimation result from numerical linear algebra with the conjugate gradient algorithm. On several simulations, our method scales better than existing approaches in computation time and memory.
  arXiv  Detail & Related papers  (2022-02-25T16:35:26Z)
• Learning in High-Dimensional Feature Spaces Using ANOVA-Based Fast Matrix-Vector Multiplication [0.0]
  kernel matrix is typically dense and large-scale. Depending on the dimension of the feature space even the computation of all of its entries in reasonable time becomes a challenging task. We propose the use of an ANOVA kernel, where we construct several kernels based on lower-dimensional feature spaces for which we provide fast algorithms realizing the matrix-vector products. Based on a feature grouping approach, we then show how the fast matrix-vector products can be embedded into a learning method choosing kernel ridge regression and the preconditioned conjugate gradient solver.
  arXiv  Detail & Related papers  (2021-11-19T10:29:39Z)
• Kernel Identification Through Transformers [54.3795894579111]
  Kernel selection plays a central role in determining the performance of Gaussian Process (GP) models. This work addresses the challenge of constructing custom kernel functions for high-dimensional GP regression models. We introduce a novel approach named KITT: Kernel Identification Through Transformers.
  arXiv  Detail & Related papers  (2021-06-15T14:32:38Z)
• Kernelized Classification in Deep Networks [49.47339560731506]
  We propose a kernelized classification layer for deep networks. We advocate a nonlinear classification layer by using the kernel trick on the softmax cross-entropy loss function during training. We show the usefulness of the proposed nonlinear classification layer on several datasets and tasks.
  arXiv  Detail & Related papers  (2020-12-08T21:43:19Z)
• SimpleMKKM: Simple Multiple Kernel K-means [49.500663154085586]
  We propose a simple yet effective multiple kernel clustering algorithm, termed simple multiple kernel k-means (SimpleMKKM) Our criterion is given by an intractable minimization-maximization problem in the kernel coefficient and clustering partition matrix. We theoretically analyze the performance of SimpleMKKM in terms of its clustering generalization error.
  arXiv  Detail & Related papers  (2020-05-11T10:06:40Z)
• Nonlinear classifiers for ranking problems based on kernelized SVM [0.0]
  Many classification problems focus on maximizing the performance only on the samples with the highest relevance instead of all samples. In this paper, we derive a general framework including several classes of these linear classification problems. We dualize the problems, add kernels and propose a componentwise dual ascent method.
  arXiv  Detail & Related papers  (2020-02-26T12:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.