Manifold Approximation leads to Robust Kernel Alignment

• URL: http://arxiv.org/abs/2510.22953v1
• Date: Mon, 27 Oct 2025 03:16:15 GMT
• Title: Manifold Approximation leads to Robust Kernel Alignment
• Authors: Mohammad Tariqul Islam, Du Liu, Deblina Sarkar,
• Abstract summary: Centered kernel alignment (CKA) is a popular metric for comparing representations.<n>We propose approximated Kernel Alignment (MKA) which incorporates manifold geometry into the alignment task.<n>Our findings suggest that manifold-aware kernel alignment provides a more robust foundation for measuring representations.
• Score: 0.48174297895861273
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Centered kernel alignment (CKA) is a popular metric for comparing representations, determining equivalence of networks, and neuroscience research. However, CKA does not account for the underlying manifold and relies on numerous heuristics that cause it to behave differently at different scales of data. In this work, we propose Manifold approximated Kernel Alignment (MKA), which incorporates manifold geometry into the alignment task. We derive a theoretical framework for MKA. We perform empirical evaluations on synthetic datasets and real-world examples to characterize and compare MKA to its contemporaries. Our findings suggest that manifold-aware kernel alignment provides a more robust foundation for measuring representations, with potential applications in representation learning.

Related papers

• Learning Reconstructive Embeddings in Reproducing Kernel Hilbert Spaces via the Representer Theorem [2.0573301822495553]
  This work proposes new algorithms for reconstruction-based manifold learning within Reproducing Kernel Hilbert Spaces (RKHS)<n>A separable operator-valued kernel extends the formulation to vector-valued data while retaining the simplicity of a single scalar similarity function.<n>A subsequent kernel-alignment task projects the data into a lower-dimensional latent space whose Gram matrix aims to match the high-dimensional reconstruction kernel.
  arXiv  Detail & Related papers  (2026-01-09T14:35:19Z)
• Evaluating Representational Similarity Measures from the Lens of Functional Correspondence [3.1883014716361635]
  Neuroscience and artificial intelligence (AI) both face the challenge of interpreting high-dimensional neural data.<n>Despite the widespread use of representational comparisons, a critical question remains: which metrics are most suitable for these comparisons?
  arXiv  Detail & Related papers  (2024-11-21T23:53:58Z)
• Self-Supervised Graph Embedding Clustering [70.36328717683297]
  K-means one-step dimensionality reduction clustering method has made some progress in addressing the curse of dimensionality in clustering tasks. We propose a unified framework that integrates manifold learning with K-means, resulting in the self-supervised graph embedding framework.
  arXiv  Detail & Related papers  (2024-09-24T08:59:51Z)
• MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependence [97.93517982908007]
  In cross-domain few-shot classification, NCC aims to learn representations to construct a metric space where few-shot classification can be performed. In this paper, we find that there exist high similarities between NCC-learned representations of two samples from different classes. We propose a bi-level optimization framework, emphmaximizing optimized kernel dependence (MOKD) to learn a set of class-specific representations that match the cluster structures indicated by labeled data.
  arXiv  Detail & Related papers  (2024-05-29T05:59:52Z)
• Rethinking Centered Kernel Alignment in Knowledge Distillation [17.67732046242562]
  Centered Kernel Alignment (CKA) is widely used to measure representation similarity and has been applied in several knowledge distillation methods. This paper first provides a theoretical perspective to illustrate the effectiveness of CKA, which decouples CKA to the upper bound of Maximum Mean Discrepancy(MMD) and a constant term. We propose a novel Relation-Centered Kernel Alignment(RCKA) framework, which practically establishes a connection between CKA and MMD.
  arXiv  Detail & Related papers  (2024-01-22T10:37:59Z)
• Posterior Contraction Rates for Mat\'ern Gaussian Processes on Riemannian Manifolds [51.68005047958965]
  We show that intrinsic Gaussian processes can achieve better performance in practice. Our work shows that finer-grained analyses are needed to distinguish between different levels of data-efficiency.
  arXiv  Detail & Related papers  (2023-09-19T20:30:58Z)
• Rethinking k-means from manifold learning perspective [122.38667613245151]
  We present a new clustering algorithm which directly detects clusters of data without mean estimation. Specifically, we construct distance matrix between data points by Butterworth filter. To well exploit the complementary information embedded in different views, we leverage the tensor Schatten p-norm regularization.
  arXiv  Detail & Related papers  (2023-05-12T03:01:41Z)
• Reliability of CKA as a Similarity Measure in Deep Learning [17.555458413538233]
  We present analysis that characterizes CKA sensitivity to a large class of simple transformations. We investigate several weaknesses of the CKA similarity metric, demonstrating situations in which it gives unexpected or counter-intuitive results. Our results illustrate that, in many cases, the CKA value can be easily manipulated without substantial changes to the functional behaviour of the models.
  arXiv  Detail & Related papers  (2022-10-28T14:32:52Z)
• Measuring dissimilarity with diffeomorphism invariance [94.02751799024684]
  We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces. We prove that DID enjoys properties which make it relevant for theoretical study and practical use.
  arXiv  Detail & Related papers  (2022-02-11T13:51:30Z)
• Revisiting Memory Efficient Kernel Approximation: An Indefinite Learning Perspective [0.8594140167290097]
  Matrix approximations are a key element in large-scale machine learning approaches. We extend MEKA to be applicable not only for shift-invariant kernels but also for non-stationary kernels. We present a Lanczos-based estimation of a spectrum shift to develop a stable positive semi-definite MEKA approximation.
  arXiv  Detail & Related papers  (2021-12-18T10:01:34Z)
• Effective Data-aware Covariance Estimator from Compressed Data [63.16042585506435]
  We propose a data-aware weighted sampling based covariance matrix estimator, namely DACE, which can provide an unbiased covariance matrix estimation. We conduct extensive experiments on both synthetic and real-world datasets to demonstrate the superior performance of our DACE.
  arXiv  Detail & Related papers  (2020-10-10T10:10:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.