From Activation to Causality: Discovery of Causal Visual Representations in the Human Brain

• URL: http://arxiv.org/abs/2605.23895v1
• Date: Fri, 22 May 2026 17:56:37 GMT
• Title: From Activation to Causality: Discovery of Causal Visual Representations in the Human Brain
• Abstract summary: We introduce BrainCause, an automated framework that synthesizes controlled stimuli and validate neural representations.<n>BrainCause searches for representations that respond specifically to the target concept over correlated alternatives.<n>It recovers known functional localizations and identifies new candidate representations across dozens of concepts, validated on both predicted and measured fMRI data.
• Score: 32.51402978872901
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identifying which brain regions represent a visual concept in the human brain is a central challenge in neuroscience. Existing approaches have localized coarse functional regions (e.g., faces, places) through activation maximization, identifying regions that activate strongly for a target concept relative to other concepts. Yet strong activation alone does not establish that a region represents the concept itself, as responses may instead be driven by correlated visual or semantic cues. We introduce BrainCause, an automated framework that combines generative and brain models to synthesize controlled stimuli and validate neural representations through targeted causal testing. Given a query specifying a concept of interest, our framework constructs targeted stimulus sets comprising concept images, counterfactual edits that remove the target concept while preserving other image content, and images with candidate correlated distractors. It then uses an image-to-fMRI encoding model to predict brain responses and searches for representations that respond specifically to the target concept over correlated alternatives. BrainCause returns validated candidate representations and proposes follow-up fMRI experiments to further test or extend its discoveries. Our approach successfully recovers known functional localizations and identifies new candidate representations across dozens of concepts, validated on both predicted and measured fMRI data. Critically, we show that without causal validation, a large fraction of localizations would be false positives, confirming that activation alone is insufficient evidence of representation.

Related papers

• BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language [65.1211456110319]
  BrainJanus is the first unified brain model that integrates brain, vision, and language within a single framework.<n>Our framework exhibits zero-shot generalization preserves and interpretable biological topography, highlighting its potential as a general-purpose brain modeling paradigm.
  arXiv  Detail & Related papers  (2026-06-29T14:02:15Z)
• What is Missing? Explaining Neurons Activated by Absent Concepts [57.02903530185128]
  XAI aims to provide human-interpretable insights into the behavior of deep neural networks (DNNs)<n>In existing work, this causal structure often includes relationships where the presence of a concept is associated with a strong activation of a neuron.<n>We show that a largely overlooked type of causal relationship is that of encoded absences, where the absence of a concept increases neural activation.
  arXiv  Detail & Related papers  (2026-03-10T15:21:52Z)
• Towards Interpretable Visual Decoding with Attention to Brain Representations [3.254716591226115]
  Recent work has demonstrated that complex visual stimuli can be decoded from human brain activity using deep generative models.<n>We propose NeuroAdapter, a visual decoding framework that directly conditions a latent diffusion model on brain representations.<n>Our results highlight the potential of end-to-end brain-to-image decoding and establish a path toward interpreting diffusion models through the lens of visual neuroscience.
  arXiv  Detail & Related papers  (2025-09-28T01:55:55Z)
• MindSimulator: Exploring Brain Concept Localization via Synthetic FMRI [11.463093725231971]
  Concept-selective regions within the human cerebral cortex exhibit significant activation in response to specific visual stimuli associated with particular concepts.<n> Conventional experiment-driven approaches hinge on manually constructed visual stimulus collections and corresponding brain activity recordings.<n>By synthesizing extensive brain activity recordings, we statistically localize various concept-selective regions.
  arXiv  Detail & Related papers  (2025-03-04T07:20:42Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• Identifying Shared Decodable Concepts in the Human Brain Using Image-Language Foundation Models [2.213723689024101]
  We introduce a method that takes advantage of high-quality pretrained multimodal representations to explore fine-grained semantic networks in the human brain. To identify such brain regions, we developed a data-driven approach to uncover visual concepts that are decodable from a massive functional magnetic resonance imaging (fMRI) dataset.
  arXiv  Detail & Related papers  (2023-06-06T03:29:47Z)
• Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models [6.866437017874623]
  We introduce a data-driven approach in which we synthesize images predicted to activate a given brain region using paired natural images and fMRI recordings. Our approach builds on recent generative methods by combining large-scale diffusion models with brain-guided image synthesis. These results advance our understanding of the fine-grained functional organization of human visual cortex.
  arXiv  Detail & Related papers  (2023-06-05T17:59:05Z)
• Joint fMRI Decoding and Encoding with Latent Embedding Alignment [77.66508125297754]
  We introduce a unified framework that addresses both fMRI decoding and encoding. Our model concurrently recovers visual stimuli from fMRI signals and predicts brain activity from images within a unified framework.
  arXiv  Detail & Related papers  (2023-03-26T14:14:58Z)
• Semantic Brain Decoding: from fMRI to conceptually similar image reconstruction of visual stimuli [0.29005223064604074]
  We propose a novel approach to brain decoding that also relies on semantic and contextual similarity. We employ an fMRI dataset of natural image vision and create a deep learning decoding pipeline inspired by the existence of both bottom-up and top-down processes in human vision. We produce reconstructions of visual stimuli that match the original content very well on a semantic level, surpassing the state of the art in previous literature.
  arXiv  Detail & Related papers  (2022-12-13T16:54:08Z)
• BI AVAN: Brain inspired Adversarial Visual Attention Network [67.05560966998559]
  We propose a brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity. Our model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-10-27T22:20:36Z)
• Adapting Brain-Like Neural Networks for Modeling Cortical Visual Prostheses [68.96380145211093]
  Cortical prostheses are devices implanted in the visual cortex that attempt to restore lost vision by electrically stimulating neurons. Currently, the vision provided by these devices is limited, and accurately predicting the visual percepts resulting from stimulation is an open challenge. We propose to address this challenge by utilizing 'brain-like' convolutional neural networks (CNNs), which have emerged as promising models of the visual system.
  arXiv  Detail & Related papers  (2022-09-27T17:33:19Z)
• Functional2Structural: Cross-Modality Brain Networks Representation Learning [55.24969686433101]
  Graph mining on brain networks may facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. We propose a novel graph learning framework, known as Deep Signed Brain Networks (DSBN), with a signed graph encoder. We validate our framework on clinical phenotype and neurodegenerative disease prediction tasks using two independent, publicly available datasets.
  arXiv  Detail & Related papers  (2022-05-06T03:45:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.