Representational Similarity via Interpretable Visual Concepts

• URL: http://arxiv.org/abs/2503.15699v2
• Date: Sun, 30 Mar 2025 03:02:15 GMT
• Title: Representational Similarity via Interpretable Visual Concepts
• Authors: Neehar Kondapaneni, Oisin Mac Aodha, Pietro Perona,
• Abstract summary: We introduce an interpretable representational similarity method to compare two networks.<n>We show that some aspects of model differences can be attributed to unique concepts discovered by one model that are not well represented in the other.
• Score: 27.72186215265676
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: How do two deep neural networks differ in how they arrive at a decision? Measuring the similarity of deep networks has been a long-standing open question. Most existing methods provide a single number to measure the similarity of two networks at a given layer, but give no insight into what makes them similar or dissimilar. We introduce an interpretable representational similarity method (RSVC) to compare two networks. We use RSVC to discover shared and unique visual concepts between two models. We show that some aspects of model differences can be attributed to unique concepts discovered by one model that are not well represented in the other. Finally, we conduct extensive evaluation across different vision model architectures and training protocols to demonstrate its effectiveness.

Related papers

• What Makes Two Language Models Think Alike? [6.244579327420724]
  We propose a new approach, based on metric-learning encoding models (MLEMs), as a first step to answer this question. MLEMs offer a transparent comparison, by identifying the specific linguistic features responsible for similarities and differences. The approach can straightforwardly be extended to other domains, such as speech and vision, and to other neural systems, including human brains.
  arXiv  Detail & Related papers  (2024-06-18T13:45:50Z)
• Revealing Similar Semantics Inside CNNs: An Interpretable Concept-based Comparison of Feature Spaces [0.0]
  Safety-critical applications require transparency in artificial intelligence components. convolutional neural networks (CNNs) widely used for perception tasks lack inherent interpretability. We propose two methods for estimating the layer-wise similarity between semantic information inside CNN latent spaces.
  arXiv  Detail & Related papers  (2023-04-30T13:53:39Z)
• Model Stitching: Looking For Functional Similarity Between Representations [5.657258033928475]
  We expand on a previous work which used model stitching to compare representations of the same shapes learned by differently seeded and/or trained neural networks of the same architecture. We reveal unexpected behavior of model stitching. Namely, we find that stitching, based on convolutions, for small ResNets, can reach high accuracy if those layers come later in the first (sender) network than in the second (receiver)
  arXiv  Detail & Related papers  (2023-03-20T17:12:42Z)
• Similarity of Neural Architectures using Adversarial Attack Transferability [47.66096554602005]
  We design a quantitative and scalable similarity measure between neural architectures. We conduct a large-scale analysis on 69 state-of-the-art ImageNet classifiers. Our results provide insights into why developing diverse neural architectures with distinct components is necessary.
  arXiv  Detail & Related papers  (2022-10-20T16:56:47Z)
• Attributable Visual Similarity Learning [90.69718495533144]
  This paper proposes an attributable visual similarity learning (AVSL) framework for a more accurate and explainable similarity measure between images. Motivated by the human semantic similarity cognition, we propose a generalized similarity learning paradigm to represent the similarity between two images with a graph. Experiments on the CUB-200-2011, Cars196, and Stanford Online Products datasets demonstrate significant improvements over existing deep similarity learning methods.
  arXiv  Detail & Related papers  (2022-03-28T17:35:31Z)
• Comparing Deep Neural Nets with UMAP Tour [12.910602784766562]
  UMAP Tour is built to visually inspect and compare internal behavior of real-world neural network models. We find concepts learned in state-of-the-art models and dissimilarities between them, such as GoogLeNet and ResNet.
  arXiv  Detail & Related papers  (2021-10-18T15:59:13Z)
• Interpreting Face Inference Models using Hierarchical Network Dissection [10.852613235927958]
  Hierarchical Network Dissection is a pipeline to interpret the internal representation of face-centric inference models. Our pipeline is inspired by Network Dissection, a popular interpretability model for object-centric and scene-centric models.
  arXiv  Detail & Related papers  (2021-08-23T18:52:47Z)
• Interpretable Multi-dataset Evaluation for Named Entity Recognition [110.64368106131062]
  We present a general methodology for interpretable evaluation for the named entity recognition (NER) task. The proposed evaluation method enables us to interpret the differences in models and datasets, as well as the interplay between them. By making our analysis tool available, we make it easy for future researchers to run similar analyses and drive progress in this area.
  arXiv  Detail & Related papers  (2020-11-13T10:53:27Z)
• Toward Scalable and Unified Example-based Explanation and Outlier Detection [128.23117182137418]
  We argue for a broader adoption of prototype-based student networks capable of providing an example-based explanation for their prediction. We show that our prototype-based networks beyond similarity kernels deliver meaningful explanations and promising outlier detection results without compromising classification accuracy.
  arXiv  Detail & Related papers  (2020-11-11T05:58:17Z)
• Visual Concept Reasoning Networks [93.99840807973546]
  A split-transform-merge strategy has been broadly used as an architectural constraint in convolutional neural networks for visual recognition tasks. We propose to exploit this strategy and combine it with our Visual Concept Reasoning Networks (VCRNet) to enable reasoning between high-level visual concepts. Our proposed model, VCRNet, consistently improves the performance by increasing the number of parameters by less than 1%.
  arXiv  Detail & Related papers  (2020-08-26T20:02:40Z)
• Few-shot Visual Reasoning with Meta-analogical Contrastive Learning [141.2562447971]
  We propose to solve a few-shot (or low-shot) visual reasoning problem, by resorting to analogical reasoning. We extract structural relationships between elements in both domains, and enforce them to be as similar as possible with analogical learning. We validate our method on RAVEN dataset, on which it outperforms state-of-the-art method, with larger gains when the training data is scarce.
  arXiv  Detail & Related papers  (2020-07-23T14:00:34Z)
• ReMarNet: Conjoint Relation and Margin Learning for Small-Sample Image Classification [49.87503122462432]
  We introduce a novel neural network termed Relation-and-Margin learning Network (ReMarNet) Our method assembles two networks of different backbones so as to learn the features that can perform excellently in both of the aforementioned two classification mechanisms. Experiments on four image datasets demonstrate that our approach is effective in learning discriminative features from a small set of labeled samples.
  arXiv  Detail & Related papers  (2020-06-27T13:50:20Z)
• Domain Siamese CNNs for Sparse Multispectral Disparity Estimation [15.065764374430783]
  We propose a new CNN architecture able to do disparity estimation between images from different spectrum. Our method was tested using the publicly available LITIV 2014 and LITIV 2018 datasets.
  arXiv  Detail & Related papers  (2020-04-30T20:29:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.