Learning Kernels by Alignment for Multiclass Bayes Classification

• URL: http://arxiv.org/abs/2609.06474v1
• Date: Sun, 06 Sep 2026 08:46:36 GMT
• Title: Learning Kernels by Alignment for Multiclass Bayes Classification
• Abstract summary: We show that Collaborative Learning and Inference can be viewed as a kernel alignment process.<n>We also prove that Collaborative Inference is equivalent to kernel Bayes classification with Parzen-window density estimation.
• Score: 1.988145627448243
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Kernel methods separate data representation from decision-making, but typically require the kernel to be chosen in advance. We show that this kernel can instead be learned by alignment, and develop the resulting framework through the recently introduced Collaborative Learning and Inference (CLaI). We show that Collaborative Learning can be viewed as a kernel alignment process, in which an embedding is trained so that its induced similarity matches a label-derived target kernel. We also prove that Collaborative Inference is equivalent to kernel Bayes classification with Parzen-window density estimation. Motivated by these perspectives, we generalise CLaI by replacing cosine similarity with a learned Mahalanobis distance and extend it to multiclass classification. On CIFAR-10, PathMNIST, and SleepEDF, the Mahalanobis formulation improves accuracy, converges faster, and yields lower calibration error than the cosine-based variant. Auxiliary experiments further support these connections, showing that CLaI produces latent signals of the same form as a Gaussian process, while achieving competitive calibration on sepsis prediction. Together, these results establish a principled learned-kernel framework that unifies representation learning, kernel alignment, and Bayesian classification, and extends naturally to the multiclass setting.

Related papers

• Scalable Pairwise Kernel Learning with Stochastic Vec Trick [0.8411385346896413]
  We introduce SPaiK, a new scalable kernel learning method tailored for pairwise settings.<n>Our approach preserves the expressive power of kernel methods while substantially reducing computational memory requirements.<n>We evaluate the performance of SPaiK on seven real-world drug-target affinity datasets.
  arXiv  Detail & Related papers  (2026-06-15T17:15:56Z)
• MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependence [97.93517982908007]
  In cross-domain few-shot classification, NCC aims to learn representations to construct a metric space where few-shot classification can be performed. In this paper, we find that there exist high similarities between NCC-learned representations of two samples from different classes. We propose a bi-level optimization framework, emphmaximizing optimized kernel dependence (MOKD) to learn a set of class-specific representations that match the cluster structures indicated by labeled data.
  arXiv  Detail & Related papers  (2024-05-29T05:59:52Z)
• Joint Embedding Self-Supervised Learning in the Kernel Regime [21.80241600638596]
  Self-supervised learning (SSL) produces useful representations of data without access to any labels for classifying the data. We extend this framework to incorporate algorithms based on kernel methods where embeddings are constructed by linear maps acting on the feature space of a kernel. We analyze our kernel model on small datasets to identify common features of self-supervised learning algorithms and gain theoretical insights into their performance on downstream tasks.
  arXiv  Detail & Related papers  (2022-09-29T15:53:19Z)
• Variational Autoencoder Kernel Interpretation and Selection for Classification [59.30734371401315]
  This work proposed kernel selection approaches for probabilistic classifiers based on features produced by the convolutional encoder of a variational autoencoder. In the proposed implementation, each latent variable was sampled from the distribution associated with a single kernel of the last encoder's convolution layer, as an individual distribution was created for each kernel. choosing relevant features on the sampled latent variables makes it possible to perform kernel selection, filtering the uninformative features and kernels.
  arXiv  Detail & Related papers  (2022-09-10T17:22:53Z)
• Local Sample-weighted Multiple Kernel Clustering with Consensus Discriminative Graph [73.68184322526338]
  Multiple kernel clustering (MKC) is committed to achieving optimal information fusion from a set of base kernels. This paper proposes a novel local sample-weighted multiple kernel clustering model. Experimental results demonstrate that our LSWMKC possesses better local manifold representation and outperforms existing kernel or graph-based clustering algo-rithms.
  arXiv  Detail & Related papers  (2022-07-05T05:00:38Z)
• Generalized Reference Kernel for One-class Classification [100.53532594448048]
  We formulate a new generalized reference kernel to improve the original base kernel using a set of reference vectors. Our analysis and experimental results show that the new formulation provides approaches to regularize, adjust the rank, and incorporate additional information into the kernel itself.
  arXiv  Detail & Related papers  (2022-05-01T18:36:55Z)
• Meta-Learning Hypothesis Spaces for Sequential Decision-making [79.73213540203389]
  We propose to meta-learn a kernel from offline data (Meta-KeL) Under mild conditions, we guarantee that our estimated RKHS yields valid confidence sets. We also empirically evaluate the effectiveness of our approach on a Bayesian optimization task.
  arXiv  Detail & Related papers  (2022-02-01T17:46:51Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)
• Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior [26.52863547394537]
  We present a novel probabilistic algorithm to learn a kernel composition by handling the sparsity in the kernel selection with Horseshoe prior. Our model can capture characteristics of time series with significant reductions in computational time and have competitive regression performance on real-world data sets.
  arXiv  Detail & Related papers  (2020-12-21T13:41:15Z)
• Generalized vec trick for fast learning of pairwise kernel models [3.867363075280544]
  We present a comprehensive review of pairwise kernels, that have been proposed for incorporating prior knowledge about the relationship between the objects. We show how all the reviewed kernels can be expressed as sums of Kronecker products, allowing the use of generalized vec trick for speeding up their computation.
  arXiv  Detail & Related papers  (2020-09-02T13:27:51Z)
• Algorithms for Learning Kernels Based on Centered Alignment [39.7160959329339]
  This paper presents new and effective algorithms for learning kernels. Our algorithms are based on the notion of centered alignment. We show that these algorithms consistently outperform the so-called uniform combination solution.
  arXiv  Detail & Related papers  (2012-03-02T19:20:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.