Explaining the Impact of Training on Vision Models via Activation Clustering

• URL: http://arxiv.org/abs/2411.19700v3
• Date: Sun, 23 Mar 2025 17:37:14 GMT
• Title: Explaining the Impact of Training on Vision Models via Activation Clustering
• Authors: Ahcène Boubekki, Samuel G. Fadel, Sebastian Mair,
• Abstract summary: This paper introduces Neuro-Activated Vision Explanations (NAVE)<n>NAVE is a method for extracting and visualizing the internal representations of vision model encoders.<n>By clustering feature activations, NAVE provides insights into learned semantics without fine-tuning.
• Score: 2.8792218859042453
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces Neuro-Activated Vision Explanations (NAVE), a method for extracting and visualizing the internal representations of vision model encoders. By clustering feature activations, NAVE provides insights into learned semantics without fine-tuning. Using object localization, we show that NAVE's concepts align with image semantics. Through extensive experiments, we analyze the impact of training strategies and architectures on encoder representation capabilities. Additionally, we apply NAVE to study training artifacts in vision transformers and reveal how weak training strategies and spurious correlations degrade model performance. Our findings establish NAVE as a valuable tool for post-hoc model inspection and improving transparency in vision models.

Related papers

• Sparse Autoencoders for Scientifically Rigorous Interpretation of Vision Models [27.806966289284528]
  We present a unified framework using sparse autoencoders (SAEs) to discover human-interpretable visual features. We show that SAEs can reliably identify and manipulate interpretable visual features without model re-training.
  arXiv  Detail & Related papers  (2025-02-10T18:32:41Z)
• ViTOC: Vision Transformer and Object-aware Captioner [0.0]
  ViTOC is a vision-language model for image captioning that addresses the challenges of accuracy and diversity in generated descriptions. By utilizing pretrained visual model parameters, ViTOC achieves efficient end-to-end training.
  arXiv  Detail & Related papers  (2024-11-09T13:13:49Z)
• Enhancing Neural Network Interpretability Through Conductance-Based Information Plane Analysis [0.0]
  The Information Plane is a conceptual framework used to analyze the flow of information in neural networks. This paper introduces a new approach that uses layer conductance, a measure of sensitivity to input features, to enhance the Information Plane analysis.
  arXiv  Detail & Related papers  (2024-08-26T23:10:42Z)
• Zero-Shot Object-Centric Representation Learning [72.43369950684057]
  We study current object-centric methods through the lens of zero-shot generalization. We introduce a benchmark comprising eight different synthetic and real-world datasets. We find that training on diverse real-world images improves transferability to unseen scenarios.
  arXiv  Detail & Related papers  (2024-08-17T10:37:07Z)
• Learning Object-Centric Representation via Reverse Hierarchy Guidance [73.05170419085796]
  Object-Centric Learning (OCL) seeks to enable Neural Networks to identify individual objects in visual scenes. RHGNet introduces a top-down pathway that works in different ways in the training and inference processes. Our model achieves SOTA performance on several commonly used datasets.
  arXiv  Detail & Related papers  (2024-05-17T07:48:27Z)
• Automatic Discovery of Visual Circuits [66.99553804855931]
  We explore scalable methods for extracting the subgraph of a vision model's computational graph that underlies recognition of a specific visual concept. We find that our approach extracts circuits that causally affect model output, and that editing these circuits can defend large pretrained models from adversarial attacks.
  arXiv  Detail & Related papers  (2024-04-22T17:00:57Z)
• Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models [81.71651422951074]
  Chain-of-Spot (CoS) method is a novel approach that enhances feature extraction by focusing on key regions of interest. This technique allows LVLMs to access more detailed visual information without altering the original image resolution. Our empirical findings demonstrate a significant improvement in LVLMs' ability to understand and reason about visual content.
  arXiv  Detail & Related papers  (2024-03-19T17:59:52Z)
• Data-efficient Large Vision Models through Sequential Autoregression [58.26179273091461]
  We develop an efficient, autoregression-based vision model on a limited dataset. We demonstrate how this model achieves proficiency in a spectrum of visual tasks spanning both high-level and low-level semantic understanding. Our empirical evaluations underscore the model's agility in adapting to various tasks, heralding a significant reduction in the parameter footprint.
  arXiv  Detail & Related papers  (2024-02-07T13:41:53Z)
• Manipulating Feature Visualizations with Gradient Slingshots [54.31109240020007]
  We introduce a novel method for manipulating Feature Visualization (FV) without significantly impacting the model's decision-making process. We evaluate the effectiveness of our method on several neural network models and demonstrate its capabilities to hide the functionality of arbitrarily chosen neurons.
  arXiv  Detail & Related papers  (2024-01-11T18:57:17Z)
• Advancing Ante-Hoc Explainable Models through Generative Adversarial Networks [24.45212348373868]
  This paper presents a novel concept learning framework for enhancing model interpretability and performance in visual classification tasks. Our approach appends an unsupervised explanation generator to the primary classifier network and makes use of adversarial training. This work presents a significant step towards building inherently interpretable deep vision models with task-aligned concept representations.
  arXiv  Detail & Related papers  (2024-01-09T16:16:16Z)
• TVE: Learning Meta-attribution for Transferable Vision Explainer [76.68234965262761]
  We introduce a Transferable Vision Explainer (TVE) that can effectively explain various vision models in downstream tasks. TVE is realized through a pre-training process on large-scale datasets towards learning the meta-attribution. This meta-attribution leverages the versatility of generic backbone encoders to comprehensively encode the attribution knowledge for the input instance, which enables TVE to seamlessly transfer to explain various downstream tasks.
  arXiv  Detail & Related papers  (2023-12-23T21:49:23Z)
• Visualizing Transferred Knowledge: An Interpretive Model of Unsupervised Domain Adaptation [70.85686267987744]
  Unsupervised domain adaptation problems can transfer knowledge from a labeled source domain to an unlabeled target domain. We propose an interpretive model of unsupervised domain adaptation, as the first attempt to visually unveil the mystery of transferred knowledge. Our method provides an intuitive explanation for the base model's predictions and unveils transfer knowledge by matching the image patches with the same semantics across both source and target domains.
  arXiv  Detail & Related papers  (2023-03-04T03:02:12Z)
• A Detailed Study of Interpretability of Deep Neural Network based Top Taggers [3.8541104292281805]
  Recent developments in explainable AI (XAI) allow researchers to explore the inner workings of deep neural networks (DNNs) We explore interpretability of models designed to identify jets coming from top quark decay in high energy proton-proton collisions at the Large Hadron Collider (LHC) Our studies uncover some major pitfalls of existing XAI methods and illustrate how they can be overcome to obtain consistent and meaningful interpretation of these models.
  arXiv  Detail & Related papers  (2022-10-09T23:02:42Z)
• Entity-Conditioned Question Generation for Robust Attention Distribution in Neural Information Retrieval [51.53892300802014]
  We show that supervised neural information retrieval models are prone to learning sparse attention patterns over passage tokens. Using a novel targeted synthetic data generation method, we teach neural IR to attend more uniformly and robustly to all entities in a given passage.
  arXiv  Detail & Related papers  (2022-04-24T22:36:48Z)
• Perception Visualization: Seeing Through the Eyes of a DNN [5.9557391359320375]
  We develop a new form of explanation that is radically different in nature from current explanation methods, such as Grad-CAM. Perception visualization provides a visual representation of what the DNN perceives in the input image by depicting what visual patterns the latent representation corresponds to. Results of our user study demonstrate that humans can better understand and predict the system's decisions when perception visualizations are available.
  arXiv  Detail & Related papers  (2022-04-21T07:18:55Z)
• T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP [0.0]
  This paper presents the design and implementation of a visual analytic framework for assisting researchers in such process. Our framework offers an intuitive overview that allows the user to explore different facets of the model. It allows a suite of built-in algorithms that compute the importance of model components and different parts of the input sequence.
  arXiv  Detail & Related papers  (2021-08-31T02:20:46Z)
• VinVL: Revisiting Visual Representations in Vision-Language Models [96.39332942534368]
  We develop an improved object detection model to provide object-centric representations of images. New visual features significantly improve the performance across all vision language (VL) tasks. We will release the new object detection model to public.
  arXiv  Detail & Related papers  (2021-01-02T23:35:27Z)
• Guided Variational Autoencoder for Disentanglement Learning [79.02010588207416]
  We propose an algorithm, guided variational autoencoder (Guided-VAE), that is able to learn a controllable generative model by performing latent representation disentanglement learning. We design an unsupervised strategy and a supervised strategy in Guided-VAE and observe enhanced modeling and controlling capability over the vanilla VAE.
  arXiv  Detail & Related papers  (2020-04-02T20:49:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.