What Representational Similarity Measures Imply about Decodable Information

• URL: http://arxiv.org/abs/2411.08197v1
• Date: Tue, 12 Nov 2024 21:37:10 GMT
• Title: What Representational Similarity Measures Imply about Decodable Information
• Authors: Sarah E. Harvey, David Lipshutz, Alex H. Williams,
• Abstract summary: We show that some neural network similarity measures can be equivalently motivated from a decoding perspective. Measures like CKA and CCA quantify the average alignment between optimal linear readouts across a distribution of decoding tasks. Overall, our work demonstrates a tight link between the geometry of neural representations and the ability to linearly decode information.
• Score: 6.5879381737929945
• License:
• Abstract: Neural responses encode information that is useful for a variety of downstream tasks. A common approach to understand these systems is to build regression models or ``decoders'' that reconstruct features of the stimulus from neural responses. Popular neural network similarity measures like centered kernel alignment (CKA), canonical correlation analysis (CCA), and Procrustes shape distance, do not explicitly leverage this perspective and instead highlight geometric invariances to orthogonal or affine transformations when comparing representations. Here, we show that many of these measures can, in fact, be equivalently motivated from a decoding perspective. Specifically, measures like CKA and CCA quantify the average alignment between optimal linear readouts across a distribution of decoding tasks. We also show that the Procrustes shape distance upper bounds the distance between optimal linear readouts and that the converse holds for representations with low participation ratio. Overall, our work demonstrates a tight link between the geometry of neural representations and the ability to linearly decode information. This perspective suggests new ways of measuring similarity between neural systems and also provides novel, unifying interpretations of existing measures.

Related papers

• Canonical Correlation Guided Deep Neural Network [14.188285111418516]
  We present a canonical correlation guided learning framework, which allows to be realized by deep neural networks (CCDNN) In the proposed method, the optimization formulation is not restricted to maximize correlation, instead we make canonical correlation as a constraint. To reduce the redundancy induced by correlation, a redundancy filter is designed.
  arXiv  Detail & Related papers  (2024-09-28T16:08:44Z)
• Differentiable Optimization of Similarity Scores Between Models and Brains [1.5391321019692434]
  Similarity measures such as linear regression, Centered Kernel Alignment (CKA), Normalized Bures Similarity (NBS), and angular Procrustes distance are often used to quantify this similarity. Here, we introduce a novel tool to investigate what drives high similarity scores and what constitutes a "good" score. Surprisingly, we find that high similarity scores do not guarantee encoding task-relevant information in a manner consistent with neural data.
  arXiv  Detail & Related papers  (2024-07-09T17:31:47Z)
• Soft Matching Distance: A metric on neural representations that captures single-neuron tuning [6.5714523708869566]
  Common measures of neural representational (dis)similarity are designed to be insensitive to rotations and reflections of the neural activation space. We propose a new metric to measure distances between networks with different sizes.
  arXiv  Detail & Related papers  (2023-11-16T00:13:00Z)
• Heterogenous Memory Augmented Neural Networks [84.29338268789684]
  We introduce a novel heterogeneous memory augmentation approach for neural networks. By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead. We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
  arXiv  Detail & Related papers  (2023-10-17T01:05:28Z)
• Multilayer Multiset Neuronal Networks -- MMNNs [55.2480439325792]
  The present work describes multilayer multiset neuronal networks incorporating two or more layers of coincidence similarity neurons. The work also explores the utilization of counter-prototype points, which are assigned to the image regions to be avoided.
  arXiv  Detail & Related papers  (2023-08-28T12:55:13Z)
• Learning Partial Correlation based Deep Visual Representation for Image Classification [61.0532370259644]
  We formulate sparse inverse covariance estimation (SICE) as a novel structured layer of CNN. Our work obtains a partial correlation based deep visual representation and mitigates the small sample problem. Experiments show the efficacy and superior classification performance of our model.
  arXiv  Detail & Related papers  (2023-04-23T10:09:01Z)
• Large-Margin Representation Learning for Texture Classification [67.94823375350433]
  This paper presents a novel approach combining convolutional layers (CLs) and large-margin metric learning for training supervised models on small datasets for texture classification. The experimental results on texture and histopathologic image datasets have shown that the proposed approach achieves competitive accuracy with lower computational cost and faster convergence when compared to equivalent CNNs.
  arXiv  Detail & Related papers  (2022-06-17T04:07:45Z)
• Learning Optical Flow from a Few Matches [67.83633948984954]
  We show that the dense correlation volume representation is redundant and accurate flow estimation can be achieved with only a fraction of elements in it. Experiments show that our method can reduce computational cost and memory use significantly, while maintaining high accuracy.
  arXiv  Detail & Related papers  (2021-04-05T21:44:00Z)
• MetaSDF: Meta-learning Signed Distance Functions [85.81290552559817]
  Generalizing across shapes with neural implicit representations amounts to learning priors over the respective function space. We formalize learning of a shape space as a meta-learning problem and leverage gradient-based meta-learning algorithms to solve this task.
  arXiv  Detail & Related papers  (2020-06-17T05:14:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.