Similarity of Neural Network Models: A Survey of Functional and Representational Measures

• URL: http://arxiv.org/abs/2305.06329v3
• Date: Thu, 22 Aug 2024 15:52:38 GMT
• Title: Similarity of Neural Network Models: A Survey of Functional and Representational Measures
• Authors: Max Klabunde, Tobias Schumacher, Markus Strohmaier, Florian Lemmerich,
• Abstract summary: Measuring similarity of neural networks to understand and improve their behavior has become an issue of great importance and research interest. We provide a comprehensive overview of two complementary perspectives of measuring neural network similarity: (i) representational similarity, which considers how activations of intermediate layers differ, and (ii) functional similarity, which considers how models differ in their outputs.
• Score: 2.56552999376511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Measuring similarity of neural networks to understand and improve their behavior has become an issue of great importance and research interest. In this survey, we provide a comprehensive overview of two complementary perspectives of measuring neural network similarity: (i) representational similarity, which considers how activations of intermediate layers differ, and (ii) functional similarity, which considers how models differ in their outputs. In addition to providing detailed descriptions of existing measures, we summarize and discuss results on the properties of and relationships between these measures, and point to open research problems. We hope our work lays a foundation for more systematic research on the properties and applicability of similarity measures for neural network models.

Related papers

• Evaluating Representational Similarity Measures from the Lens of Functional Correspondence [1.7811840395202345]
  Neuroscience and artificial intelligence (AI) both face the challenge of interpreting high-dimensional neural data. Despite the widespread use of representational comparisons, a critical question remains: which metrics are most suitable for these comparisons?
  arXiv  Detail & Related papers  (2024-11-21T23:53:58Z)
• ReSi: A Comprehensive Benchmark for Representational Similarity Measures [2.3532263743300432]
  This paper presents the first comprehensive benchmark for evaluating representational similarity measures. The representational similarity (ReSi) benchmark consists of (i) six carefully designed tests for similarity measures, (ii) 23 similarity measures, (iii) eleven neural network architectures, and (iv) six datasets, spanning over the graph, language, and vision domains.
  arXiv  Detail & Related papers  (2024-08-01T13:08:02Z)
• Relational Composition in Neural Networks: A Survey and Call to Action [54.47858085003077]
  Many neural nets appear to represent data as linear combinations of "feature vectors" We argue that this success is incomplete without an understanding of relational composition.
  arXiv  Detail & Related papers  (2024-07-19T20:50:57Z)
• Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory [64.06519549649495]
  We provide the definition of what we call functionally equivalent features. These features produce equivalent output under certain transformations. We propose an efficient algorithm named Iterative Feature Merging.
  arXiv  Detail & Related papers  (2023-10-10T16:27:12Z)
• Multilayer Multiset Neuronal Networks -- MMNNs [55.2480439325792]
  The present work describes multilayer multiset neuronal networks incorporating two or more layers of coincidence similarity neurons. The work also explores the utilization of counter-prototype points, which are assigned to the image regions to be avoided.
  arXiv  Detail & Related papers  (2023-08-28T12:55:13Z)
• Similarity of Neural Architectures using Adversarial Attack Transferability [47.66096554602005]
  We design a quantitative and scalable similarity measure between neural architectures. We conduct a large-scale analysis on 69 state-of-the-art ImageNet classifiers. Our results provide insights into why developing diverse neural architectures with distinct components is necessary.
  arXiv  Detail & Related papers  (2022-10-20T16:56:47Z)
• Generalized Shape Metrics on Neural Representations [26.78835065137714]
  We provide a family of metric spaces that quantify representational dissimilarity. We modify existing representational similarity measures based on canonical correlation analysis to satisfy the triangle inequality. We identify relationships between neural representations that are interpretable in terms of anatomical features and model performance.
  arXiv  Detail & Related papers  (2021-10-27T19:48:55Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• Towards Interaction Detection Using Topological Analysis on Neural Networks [55.74562391439507]
  In neural networks, any interacting features must follow a strongly weighted connection to common hidden units. We propose a new measure for quantifying interaction strength, based upon the well-received theory of persistent homology. A Persistence Interaction detection(PID) algorithm is developed to efficiently detect interactions.
  arXiv  Detail & Related papers  (2020-10-25T02:15:24Z)
• Similarity of Neural Networks with Gradients [8.804507286438781]
  We propose to leverage both feature vectors and gradient ones into designing the representation of a neural network. We show that the proposed approach provides a state-of-the-art method for computing similarity of neural networks.
  arXiv  Detail & Related papers  (2020-03-25T17:04:10Z)
• Feature Importance Estimation with Self-Attention Networks [0.0]
  Black-box neural network models are widely used in industry and science, yet are hard to understand and interpret. Recently, the attention mechanism was introduced, offering insights into the inner workings of neural language models. This paper explores the use of attention-based neural networks mechanism for estimating feature importance, as means for explaining the models learned from propositional (tabular) data.
  arXiv  Detail & Related papers  (2020-02-11T15:15:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.