Automated Natural Language Explanation of Deep Visual Neurons with Large
Models
- URL: http://arxiv.org/abs/2310.10708v1
- Date: Mon, 16 Oct 2023 17:04:51 GMT
- Title: Automated Natural Language Explanation of Deep Visual Neurons with Large
Models
- Authors: Chenxu Zhao, Wei Qian, Yucheng Shi, Mengdi Huai, Ninghao Liu
- Abstract summary: This paper proposes a novel post-hoc framework for generating semantic explanations of neurons with large foundation models.
Our framework is designed to be compatible with various model architectures and datasets, automated and scalable neuron interpretation.
- Score: 43.178568768100305
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep neural networks have exhibited remarkable performance across a wide
range of real-world tasks. However, comprehending the underlying reasons for
their effectiveness remains a challenging problem. Interpreting deep neural
networks through examining neurons offers distinct advantages when it comes to
exploring the inner workings of neural networks. Previous research has
indicated that specific neurons within deep vision networks possess semantic
meaning and play pivotal roles in model performance. Nonetheless, the current
methods for generating neuron semantics heavily rely on human intervention,
which hampers their scalability and applicability. To address this limitation,
this paper proposes a novel post-hoc framework for generating semantic
explanations of neurons with large foundation models, without requiring human
intervention or prior knowledge. Our framework is designed to be compatible
with various model architectures and datasets, facilitating automated and
scalable neuron interpretation. Experiments are conducted with both qualitative
and quantitative analysis to verify the effectiveness of our proposed approach.
Related papers
- Enhancing learning in artificial neural networks through cellular heterogeneity and neuromodulatory signaling [52.06722364186432]
We propose a biologically-informed framework for enhancing artificial neural networks (ANNs)
Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors.
We outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, bioinspiration and complexity.
arXiv Detail & Related papers (2024-07-05T14:11:28Z) - Manipulating Feature Visualizations with Gradient Slingshots [54.31109240020007]
We introduce a novel method for manipulating Feature Visualization (FV) without significantly impacting the model's decision-making process.
We evaluate the effectiveness of our method on several neural network models and demonstrate its capabilities to hide the functionality of arbitrarily chosen neurons.
arXiv Detail & Related papers (2024-01-11T18:57:17Z) - Identifying Interpretable Visual Features in Artificial and Biological
Neural Systems [3.604033202771937]
Single neurons in neural networks are often interpretable in that they represent individual, intuitively meaningful features.
Many neurons exhibit $textitmixed selectivity$, i.e., they represent multiple unrelated features.
We propose an automated method for quantifying visual interpretability and an approach for finding meaningful directions in network activation space.
arXiv Detail & Related papers (2023-10-17T17:41:28Z) - Seeking Next Layer Neurons' Attention for Error-Backpropagation-Like
Training in a Multi-Agent Network Framework [6.446189857311325]
We propose a local objective for neurons that align them to exhibit similarities to error-backpropagation.
We examine a neural network comprising decentralized, self-interested neurons seeking to maximize their local objective.
We demonstrate the learning capacity of these multi-agent neural networks through experiments on three datasets.
arXiv Detail & Related papers (2023-10-15T21:07:09Z) - Addressing caveats of neural persistence with deep graph persistence [54.424983583720675]
We find that the variance of network weights and spatial concentration of large weights are the main factors that impact neural persistence.
We propose an extension of the filtration underlying neural persistence to the whole neural network instead of single layers.
This yields our deep graph persistence measure, which implicitly incorporates persistent paths through the network and alleviates variance-related issues.
arXiv Detail & Related papers (2023-07-20T13:34:11Z) - Learning to Act through Evolution of Neural Diversity in Random Neural
Networks [9.387749254963595]
In most artificial neural networks (ANNs), neural computation is abstracted to an activation function that is usually shared between all neurons.
We propose the optimization of neuro-centric parameters to attain a set of diverse neurons that can perform complex computations.
arXiv Detail & Related papers (2023-05-25T11:33:04Z) - Spiking neural network for nonlinear regression [68.8204255655161]
Spiking neural networks carry the potential for a massive reduction in memory and energy consumption.
They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware.
A framework for regression using spiking neural networks is proposed.
arXiv Detail & Related papers (2022-10-06T13:04:45Z) - Overcoming the Domain Gap in Contrastive Learning of Neural Action
Representations [60.47807856873544]
A fundamental goal in neuroscience is to understand the relationship between neural activity and behavior.
We generated a new multimodal dataset consisting of the spontaneous behaviors generated by fruit flies.
This dataset and our new set of augmentations promise to accelerate the application of self-supervised learning methods in neuroscience.
arXiv Detail & Related papers (2021-11-29T15:27:51Z) - Neuron-level Interpretation of Deep NLP Models: A Survey [22.035813865470956]
A plethora of research has been carried out to analyze and understand components of the deep neural network models.
Recent work has concentrated on interpretability at a more granular level, analyzing neurons and groups of neurons in large models.
arXiv Detail & Related papers (2021-08-30T11:54:21Z) - On Interpretability of Artificial Neural Networks: A Survey [21.905647127437685]
We systematically review recent studies in understanding the mechanism of neural networks, describe applications of interpretability especially in medicine.
We discuss future directions of interpretability research, such as in relation to fuzzy logic and brain science.
arXiv Detail & Related papers (2020-01-08T13:40:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.