Describe-and-Dissect: Interpreting Neurons in Vision Networks with Language Models

• URL: http://arxiv.org/abs/2403.13771v1
• Date: Wed, 20 Mar 2024 17:33:02 GMT
• Title: Describe-and-Dissect: Interpreting Neurons in Vision Networks with Language Models
• Authors: Nicholas Bai, Rahul A. Iyer, Tuomas Oikarinen, Tsui-Wei Weng,
• Abstract summary: Describe-and-Dissect (DnD) is a novel method to describe the roles of hidden neurons in vision networks. DnD produces complex natural language descriptions without the need for labeled training data or a predefined set of concepts.
• Score: 9.962488213825859
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose Describe-and-Dissect (DnD), a novel method to describe the roles of hidden neurons in vision networks. DnD utilizes recent advancements in multimodal deep learning to produce complex natural language descriptions, without the need for labeled training data or a predefined set of concepts to choose from. Additionally, DnD is training-free, meaning we don't train any new models and can easily leverage more capable general purpose models in the future. We have conducted extensive qualitative and quantitative analysis to show that DnD outperforms prior work by providing higher quality neuron descriptions. Specifically, our method on average provides the highest quality labels and is more than 2 times as likely to be selected as the best explanation for a neuron than the best baseline.

Related papers

• Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision? [62.12375949429938]
  Building transferable Graph Neural Networks (GNNs) with CLIP pipeline is challenging because of three fundamental issues. We leverage multi-modal prompt learning to effectively adapt pre-trained GNN to downstream tasks and data. Our new paradigm embeds the graphs directly in the same space as the Large Language Models (LLMs) by learning both graph prompts and text prompts simultaneously.
  arXiv  Detail & Related papers  (2024-12-11T08:03:35Z)
• NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals [58.83169560132308]
  We introduce NNsight and NDIF, technologies that work in tandem to enable scientific study of very large neural networks. NNsight is an open-source system that extends PyTorch to introduce deferred remote execution. NDIF is a scalable inference service that executes NNsight requests, allowing users to share GPU resources and pretrained models.
  arXiv  Detail & Related papers  (2024-07-18T17:59:01Z)
• Towards Generating Informative Textual Description for Neurons in Language Models [6.884227665279812]
  We propose a framework that ties textual descriptions to neurons. In particular, our experiment shows that the proposed approach achieves 75% precision@2, and 50% recall@2
  arXiv  Detail & Related papers  (2024-01-30T04:06:25Z)
• Investigating the Encoding of Words in BERT's Neurons using Feature Textualization [11.943486282441143]
  We propose a technique to produce representations of neurons in embedding word space. We find that the produced representations can provide insights about the encoded knowledge in individual neurons.
  arXiv  Detail & Related papers  (2023-11-14T15:21:49Z)
• Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks [17.12834153477201]
  We propose a novel resource-efficient supervised feature selection method using sparse neural networks. By gradually pruning the uninformative features from the input layer of a sparse neural network trained from scratch, NeuroFS derives an informative subset of features efficiently. NeuroFS achieves the highest ranking-based score among the considered state-of-the-art supervised feature selection models.
  arXiv  Detail & Related papers  (2023-03-10T17:09:55Z)
• Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation [46.825957917649795]
  Sign Language (SL), as the mother tongue of the deaf community, is a special visual language that most hearing people cannot understand. We propose a novel neural SLT model with multi-modal feature fusion based on the dynamic graph. We are the first to introduce graph neural networks, for fusing multi-modal information, into neural sign language translation models.
  arXiv  Detail & Related papers  (2022-11-01T15:26:22Z)
• Language Model Pre-Training with Sparse Latent Typing [66.75786739499604]
  We propose a new pre-training objective, Sparse Latent Typing, which enables the model to sparsely extract sentence-level keywords with diverse latent types. Experimental results show that our model is able to learn interpretable latent type categories in a self-supervised manner without using any external knowledge.
  arXiv  Detail & Related papers  (2022-10-23T00:37:08Z)
• CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks [7.8858544147141085]
  We propose CLIP-Dissect, a new technique to automatically describe the function of individual hidden neurons inside vision networks. We show that CLIP-Dissect provides more accurate descriptions than existing methods for last layer neurons. CLIP-Dissect is computationally efficient and can label all neurons from five layers of ResNet-50 in just 4 minutes.
  arXiv  Detail & Related papers  (2022-04-23T00:40:02Z)
• Modeling Token-level Uncertainty to Learn Unknown Concepts in SLU via Calibrated Dirichlet Prior RNN [98.4713940310056]
  One major task of spoken language understanding (SLU) in modern personal assistants is to extract semantic concepts from an utterance. Recent research collected question and answer annotated data to learn what is unknown and should be asked. We incorporate softmax-based slot filling neural architectures to model the sequence uncertainty without question supervision.
  arXiv  Detail & Related papers  (2020-10-16T02:12:30Z)
• Non-linear Neurons with Human-like Apical Dendrite Activations [81.18416067005538]
  We show that a standard neuron followed by our novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. We conduct experiments on six benchmark data sets from computer vision, signal processing and natural language processing.
  arXiv  Detail & Related papers  (2020-02-02T21:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.