ConceptLens: from Pixels to Understanding

• URL: http://arxiv.org/abs/2410.05311v1
• Date: Fri, 4 Oct 2024 20:49:12 GMT
• Title: ConceptLens: from Pixels to Understanding
• Authors: Abhilekha Dalal, Pascal Hitzler,
• Abstract summary: ConceptLens is an innovative tool designed to illuminate the intricate workings of deep neural networks (DNNs) by visualizing hidden neuron activations. By integrating deep learning with symbolic methods, ConceptLens offers users a unique way to understand what triggers neuron activations.
• Score: 1.3466710708566176
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: ConceptLens is an innovative tool designed to illuminate the intricate workings of deep neural networks (DNNs) by visualizing hidden neuron activations. By integrating deep learning with symbolic methods, ConceptLens offers users a unique way to understand what triggers neuron activations and how they respond to various stimuli. The tool uses error-margin analysis to provide insights into the confidence levels of neuron activations, thereby enhancing the interpretability of DNNs. This paper presents an overview of ConceptLens, its implementation, and its application in real-time visualization of neuron activations and error margins through bar charts.

Related papers

• Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attribution [16.460751105639623]
  We introduce NeuronLens, a novel range-based interpretation and manipulation framework. It provides a finer view of neuron activation distributions to localize concept attribution within a neuron.
  arXiv  Detail & Related papers  (2025-02-04T03:33:55Z)
• Discovering Chunks in Neural Embeddings for Interpretability [53.80157905839065]
  We propose leveraging the principle of chunking to interpret artificial neural population activities. We first demonstrate this concept in recurrent neural networks (RNNs) trained on artificial sequences with imposed regularities. We identify similar recurring embedding states corresponding to concepts in the input, with perturbations to these states activating or inhibiting the associated concepts.
  arXiv  Detail & Related papers  (2025-02-03T20:30:46Z)
• Compositional Concept-Based Neuron-Level Interpretability for Deep Reinforcement Learning [2.9539724161670167]
  Deep reinforcement learning (DRL) has successfully addressed many complex control problems. Current DRL interpretability methods largely treat neural networks as black boxes. We propose a novel concept-based interpretability method that provides fine-grained explanations of DRL models at the neuron level.
  arXiv  Detail & Related papers  (2025-02-02T06:05:49Z)
• LLM-assisted Concept Discovery: Automatically Identifying and Explaining Neuron Functions [15.381209058506078]
  Prior works have associated concepts with neurons based on examples of concepts or a pre-defined set of concepts. We propose to leverage multimodal large language models for automatic and open-ended concept discovery. We validate each concept by generating examples and counterexamples and evaluating the neuron's response on this new set of images.
  arXiv  Detail & Related papers  (2024-06-12T18:19:37Z)
• Simple and Effective Transfer Learning for Neuro-Symbolic Integration [50.592338727912946]
  A potential solution to this issue is Neuro-Symbolic Integration (NeSy), where neural approaches are combined with symbolic reasoning. Most of these methods exploit a neural network to map perceptions to symbols and a logical reasoner to predict the output of the downstream task. They suffer from several issues, including slow convergence, learning difficulties with complex perception tasks, and convergence to local minima. This paper proposes a simple yet effective method to ameliorate these problems.
  arXiv  Detail & Related papers  (2024-02-21T15:51:01Z)
• Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks [74.3099028063756]
  We develop a new method with neuronal operations based on lateral connections and Hebbian learning. We show that Hebbian and anti-Hebbian learning on recurrent lateral connections can effectively extract the principal subspace of neural activities. Our method consistently solves for spiking neural networks with nearly zero forgetting.
  arXiv  Detail & Related papers  (2024-02-19T09:29:37Z)
• Manipulating Feature Visualizations with Gradient Slingshots [54.31109240020007]
  We introduce a novel method for manipulating Feature Visualization (FV) without significantly impacting the model's decision-making process. We evaluate the effectiveness of our method on several neural network models and demonstrate its capabilities to hide the functionality of arbitrarily chosen neurons.
  arXiv  Detail & Related papers  (2024-01-11T18:57:17Z)
• Global Concept-Based Interpretability for Graph Neural Networks via Neuron Analysis [0.0]
  Graph neural networks (GNNs) are highly effective on a variety of graph-related tasks. They lack interpretability and transparency. Current explainability approaches are typically local and treat GNNs as black-boxes. We propose a novel approach for producing global explanations for GNNs using neuron-level concepts.
  arXiv  Detail & Related papers  (2022-08-22T21:30:55Z)
• Neuro-Symbolic Learning of Answer Set Programs from Raw Data [54.56905063752427]
  Neuro-Symbolic AI aims to combine interpretability of symbolic techniques with the ability of deep learning to learn from raw data. We introduce Neuro-Symbolic Inductive Learner (NSIL), an approach that trains a general neural network to extract latent concepts from raw data. NSIL learns expressive knowledge, solves computationally complex problems, and achieves state-of-the-art performance in terms of accuracy and data efficiency.
  arXiv  Detail & Related papers  (2022-05-25T12:41:59Z)
• NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks [18.62960153659548]
  NeuroCartography is an interactive system that summarizes and visualizes concepts learned by neural networks. It automatically discovers and groups neurons that detect the same concepts. It describes how such neuron groups interact to form higher-level concepts and the subsequent predictions.
  arXiv  Detail & Related papers  (2021-08-29T22:43:52Z)
• Compositional Explanations of Neurons [52.71742655312625]
  We describe a procedure for explaining neurons in deep representations by identifying compositional logical concepts. We use this procedure to answer several questions on interpretability in models for vision and natural language processing.
  arXiv  Detail & Related papers  (2020-06-24T20:37:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.