What is Missing? Explaining Neurons Activated by Absent Concepts

• URL: http://arxiv.org/abs/2603.09787v1
• Date: Tue, 10 Mar 2026 15:21:52 GMT
• Title: What is Missing? Explaining Neurons Activated by Absent Concepts
• Abstract summary: XAI aims to provide human-interpretable insights into the behavior of deep neural networks (DNNs)<n>In existing work, this causal structure often includes relationships where the presence of a concept is associated with a strong activation of a neuron.<n>We show that a largely overlooked type of causal relationship is that of encoded absences, where the absence of a concept increases neural activation.
• Score: 57.02903530185128
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Explainable artificial intelligence (XAI) aims to provide human-interpretable insights into the behavior of deep neural networks (DNNs), typically by estimating a simplified causal structure of the model. In existing work, this causal structure often includes relationships where the presence of a concept is associated with a strong activation of a neuron. For example, attribution methods primarily identify input pixels that contribute most to a prediction, and feature visualization methods reveal inputs that cause high activation of a target neuron - the former implicitly assuming that the relevant information resides in the input, and the latter that neurons encode the presence of concepts. However, a largely overlooked type of causal relationship is that of encoded absences, where the absence of a concept increases neural activation. In this work, we show that such missing but relevant concepts are common and that mainstream XAI methods struggle to reveal them when applied in their standard form. To address this, we propose two simple extensions to attribution and feature visualization techniques that uncover encoded absences. Across experiments, we show how mainstream XAI methods can be used to reveal and explain encoded absences, how ImageNet models exploit them, and that debiasing can be improved when considering them.

Related papers

• Concept-Guided Interpretability via Neural Chunking [64.6429903327095]
  We show that neural networks exhibit patterns in their raw population activity that mirror regularities in the training data.<n>We propose three methods to extract recurring chunks on a neural population level.<n>Our work points to a new direction for interpretability, one that harnesses both cognitive principles and the structure of naturalistic data.
  arXiv  Detail & Related papers  (2025-05-16T13:49:43Z)
• Formal Explanations for Neuro-Symbolic AI [28.358183683756028]
  This paper proposes a formal approach to explaining the decisions of neuro-symbolic systems. It first computes a formal explanation for the symbolic component of the system, which serves to identify a subset of the individual parts of neural information that needs to be explained. This is followed by explaining only those individual neural inputs, independently of each other, which facilitates succinctness of hierarchical formal explanations.
  arXiv  Detail & Related papers  (2024-10-18T07:08:31Z)
• Artificial Kuramoto Oscillatory Neurons [65.16453738828672]
  It has long been known in both neuroscience and AI that ''binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represent more abstract concepts in deeper layers of the network.<n>We introduce Artificial rethinking together with arbitrary connectivity designs such as fully connected convolutional, or attentive mechanisms.<n>We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, uncertainty, quantification, and reasoning.
  arXiv  Detail & Related papers  (2024-10-17T17:47:54Z)
• ConceptLens: from Pixels to Understanding [1.3466710708566176]
  ConceptLens is an innovative tool designed to illuminate the intricate workings of deep neural networks (DNNs) by visualizing hidden neuron activations. By integrating deep learning with symbolic methods, ConceptLens offers users a unique way to understand what triggers neuron activations.
  arXiv  Detail & Related papers  (2024-10-04T20:49:12Z)
• The Relational Bottleneck as an Inductive Bias for Efficient Abstraction [3.19883356005403]
  We show that neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs. We review a family of models that employ this approach to induce abstractions in a data-efficient manner.
  arXiv  Detail & Related papers  (2023-09-12T22:44:14Z)
• Searching for the Essence of Adversarial Perturbations [73.96215665913797]
  We show that adversarial perturbations contain human-recognizable information, which is the key conspirator responsible for a neural network's erroneous prediction. This concept of human-recognizable information allows us to explain key features related to adversarial perturbations.
  arXiv  Detail & Related papers  (2022-05-30T18:04:57Z)
• ACRE: Abstract Causal REasoning Beyond Covariation [90.99059920286484]
  We introduce the Abstract Causal REasoning dataset for systematic evaluation of current vision systems in causal induction. Motivated by the stream of research on causal discovery in Blicket experiments, we query a visual reasoning system with the following four types of questions in either an independent scenario or an interventional scenario. We notice that pure neural models tend towards an associative strategy under their chance-level performance, whereas neuro-symbolic combinations struggle in backward-blocking reasoning.
  arXiv  Detail & Related papers  (2021-03-26T02:42:38Z)
• Object-based attention for spatio-temporal reasoning: Outperforming neuro-symbolic models with flexible distributed architectures [15.946511512356878]
  We show that a fully-learned neural network with the right inductive biases can perform substantially better than all previous neural-symbolic models. Our model makes critical use of both self-attention and learned "soft" object-centric representations.
  arXiv  Detail & Related papers  (2020-12-15T18:57:40Z)
• Compositional Explanations of Neurons [52.71742655312625]
  We describe a procedure for explaining neurons in deep representations by identifying compositional logical concepts. We use this procedure to answer several questions on interpretability in models for vision and natural language processing.
  arXiv  Detail & Related papers  (2020-06-24T20:37:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.