Differentiable Optimization of Similarity Scores Between Models and Brains

• URL: http://arxiv.org/abs/2407.07059v2
• Date: Mon, 21 Oct 2024 16:34:48 GMT
• Title: Differentiable Optimization of Similarity Scores Between Models and Brains
• Authors: Nathan Cloos, Moufan Li, Markus Siegel, Scott L. Brincat, Earl K. Miller, Guangyu Robert Yang, Christopher J. Cueva,
• Abstract summary: Similarity measures such as linear regression, Centered Kernel Alignment (CKA), Normalized Bures Similarity (NBS), and angular Procrustes distance are often used to quantify this similarity. Here, we introduce a novel tool to investigate what drives high similarity scores and what constitutes a "good" score. Surprisingly, we find that high similarity scores do not guarantee encoding task-relevant information in a manner consistent with neural data.
• Score: 1.5391321019692434
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How do we know if two systems - biological or artificial - process information in a similar way? Similarity measures such as linear regression, Centered Kernel Alignment (CKA), Normalized Bures Similarity (NBS), and angular Procrustes distance, are often used to quantify this similarity. However, it is currently unclear what drives high similarity scores and even what constitutes a "good" score. Here, we introduce a novel tool to investigate these questions by differentiating through similarity measures to directly maximize the score. Surprisingly, we find that high similarity scores do not guarantee encoding task-relevant information in a manner consistent with neural data; and this is particularly acute for CKA and even some variations of cross-validated and regularized linear regression. We find no consistent threshold for a good similarity score - it depends on both the measure and the dataset. In addition, synthetic datasets optimized to maximize similarity scores initially learn the highest variance principal component of the target dataset, but some methods like angular Procrustes capture lower variance dimensions much earlier than methods like CKA. To shed light on this, we mathematically derive the sensitivity of CKA, angular Procrustes, and NBS to the variance of principal component dimensions, and explain the emphasis CKA places on high variance components. Finally, by jointly optimizing multiple similarity measures, we characterize their allowable ranges and reveal that some similarity measures are more constraining than others. While current measures offer a seemingly straightforward way to quantify the similarity between neural systems, our work underscores the need for careful interpretation. We hope the tools we developed will be used by practitioners to better understand current and future similarity measures.

Related papers

• Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networks [4.437949196235149]
  Centred Kernel Alignment (CKA) has recently emerged as a popular metric to compare activations from biological and artificial neural networks (ANNs) In this paper we highlight issues that the community should take into account if using CKA as an alignment metric with neural data.
  arXiv  Detail & Related papers  (2024-05-02T05:27:12Z)
• Multilayer Multiset Neuronal Networks -- MMNNs [55.2480439325792]
  The present work describes multilayer multiset neuronal networks incorporating two or more layers of coincidence similarity neurons. The work also explores the utilization of counter-prototype points, which are assigned to the image regions to be avoided.
  arXiv  Detail & Related papers  (2023-08-28T12:55:13Z)
• Rethinking k-means from manifold learning perspective [122.38667613245151]
  We present a new clustering algorithm which directly detects clusters of data without mean estimation. Specifically, we construct distance matrix between data points by Butterworth filter. To well exploit the complementary information embedded in different views, we leverage the tensor Schatten p-norm regularization.
  arXiv  Detail & Related papers  (2023-05-12T03:01:41Z)
• Predictive change point detection for heterogeneous data [1.1720726814454114]
  "Predict and Compare" is a change point detection framework assisted by a predictive machine learning model. It outperforms online CPD routines in terms of false positive rate and out-of-control average run length. The power of the method is demonstrated in a tribological case study.
  arXiv  Detail & Related papers  (2023-05-11T07:59:18Z)
• Reliability of CKA as a Similarity Measure in Deep Learning [17.555458413538233]
  We present analysis that characterizes CKA sensitivity to a large class of simple transformations. We investigate several weaknesses of the CKA similarity metric, demonstrating situations in which it gives unexpected or counter-intuitive results. Our results illustrate that, in many cases, the CKA value can be easily manipulated without substantial changes to the functional behaviour of the models.
  arXiv  Detail & Related papers  (2022-10-28T14:32:52Z)
• Efficient Approximate Kernel Based Spike Sequence Classification [56.2938724367661]
  Machine learning models, such as SVM, require a definition of distance/similarity between pairs of sequences. Exact methods yield better classification performance, but they pose high computational costs. We propose a series of ways to improve the performance of the approximate kernel in order to enhance its predictive performance.
  arXiv  Detail & Related papers  (2022-09-11T22:44:19Z)
• Towards Similarity-Aware Time-Series Classification [51.2400839966489]
  We study time-series classification (TSC), a fundamental task of time-series data mining. We propose Similarity-Aware Time-Series Classification (SimTSC), a framework that models similarity information with graph neural networks (GNNs)
  arXiv  Detail & Related papers  (2022-01-05T02:14:57Z)
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty [58.144520501201995]
  Bi-Lipschitz regularization of neural network layers preserve relative distances between data instances in the feature spaces of each layer. With the use of an attentive set encoder, we propose to meta learn either diagonal or diagonal plus low-rank factors to efficiently construct task specific covariance matrices. We also propose an inference procedure which utilizes scaled energy to achieve a final predictive distribution.
  arXiv  Detail & Related papers  (2021-10-12T22:04:19Z)
• Near-Optimal Comparison Based Clustering [7.930242839366938]
  We show that our method can recover a planted clustering using a near-optimal number of comparisons. We empirically validate our theoretical findings and demonstrate the good behaviour of our method on real data.
  arXiv  Detail & Related papers  (2020-10-08T12:03:13Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)
• Learning similarity measures from data [1.4766350834632755]
  Defining similarity measures is a requirement for some machine learning methods. Data sets are typically gathered as part of constructing a CBR or machine learning system. Our objective is to investigate how to apply machine learning to effectively learn a similarity measure.
  arXiv  Detail & Related papers  (2020-01-15T13:29:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.