A Cognitive Process-Inspired Architecture for Subject-Agnostic Brain Visual Decoding

• URL: http://arxiv.org/abs/2511.02565v1
• Date: Tue, 04 Nov 2025 13:39:34 GMT
• Title: A Cognitive Process-Inspired Architecture for Subject-Agnostic Brain Visual Decoding
• Abstract summary: We propose Visual Cortex Flow Architecture (VCFlow), a novel hierarchical decoding framework.<n>VCFlow captures diverse and complementary cognitive information essential for visual reconstruction.<n> VCFlow generates each reconstructed video in 10 seconds without any retraining, offering a fast and clinically scalable solution.
• Score: 28.952331262050226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Subject-agnostic brain decoding, which aims to reconstruct continuous visual experiences from fMRI without subject-specific training, holds great potential for clinical applications. However, this direction remains underexplored due to challenges in cross-subject generalization and the complex nature of brain signals. In this work, we propose Visual Cortex Flow Architecture (VCFlow), a novel hierarchical decoding framework that explicitly models the ventral-dorsal architecture of the human visual system to learn multi-dimensional representations. By disentangling and leveraging features from early visual cortex, ventral, and dorsal streams, VCFlow captures diverse and complementary cognitive information essential for visual reconstruction. Furthermore, we introduce a feature-level contrastive learning strategy to enhance the extraction of subject-invariant semantic representations, thereby enhancing subject-agnostic applicability to previously unseen subjects. Unlike conventional pipelines that need more than 12 hours of per-subject data and heavy computation, VCFlow sacrifices only 7\% accuracy on average yet generates each reconstructed video in 10 seconds without any retraining, offering a fast and clinically scalable solution. The source code will be released upon acceptance of the paper.

Related papers

• Visual Semantic Decoding of Electrocorticography from Video Stimuli using End-to-End Deep Learning [1.229249703238137]
  ECoG-based visual semantic decoding enables inference of semantic interpretation of visual perception from complex, noisy brain activity.<n>This study examines the feasibility of visual semantic decoding using an end-to-end deep learning framework using electrocorticography (ECoG)
  arXiv  Detail & Related papers  (2026-07-21T10:06:32Z)
• Meta-learning In-Context Enables Training-Free Cross Subject Brain Decoding [19.258438045449846]
  We introduce a meta-optimized approach for semantic visual decoding from fMRI.<n>Our approach is explicitly optimized for in-context learning of the new subject's encoding model.<n>We demonstrate strong cross-subject and cross-scanner generalization across diverse visual backbones without retraining or fine-tuning.
  arXiv  Detail & Related papers  (2026-04-09T17:59:32Z)
• Learning Brain Representation with Hierarchical Visual Embeddings [30.701493890961284]
  We propose a brain-image alignment strategy that leverages pre-trained visual encoders with distinct inductive biases to capture hierarchical and multi-scale visual representations.<n>Our method achieves a favorable balance between retrieval accuracy and reconstruction fidelity.
  arXiv  Detail & Related papers  (2026-02-07T11:14:03Z)
• NeuroSwift: A Lightweight Cross-Subject Framework for fMRI Visual Reconstruction of Complex Scenes [8.32275773383994]
  Cross-subject reconstruction of visual stimuli remains challenging and computationally demanding.<n>We propose NeuroSwift, which integrates adapters via diffusion: AutoKL for low-level features and CLIP for semantics.<n>For cross-subject generalization, we pretrain on one subject and then fine-tune only 17 percent of parameters for new subjects, while freezing other components.
  arXiv  Detail & Related papers  (2025-10-02T17:45:43Z)
• DecoFuse: Decomposing and Fusing the "What", "Where", and "How" for Brain-Inspired fMRI-to-Video Decoding [82.91021399231184]
  Existing fMRI-to-video methods often focus on semantic content while overlooking spatial and motion information.<n>We propose DecoFuse, a novel brain-inspired framework for decoding videos from fMRI signals.<n>It first decomposes the video into three components - semantic, spatial, and motion - then decodes each component separately before fusing them to reconstruct the video.
  arXiv  Detail & Related papers  (2025-04-01T05:28:37Z)
• VITAL: More Understandable Feature Visualization through Distribution Alignment and Relevant Information Flow [57.96482272333649]
  Feature visualization (FV) is a powerful tool to decode what information neurons are responding to.<n>We propose to guide FV through statistics of prototypical image features combined with measures of relevant network flow to generate images.<n>Our approach yields human-understandable visualizations that both qualitatively and quantitatively improve over state-of-the-art FVs.
  arXiv  Detail & Related papers  (2025-03-28T13:08:18Z)
• Top-Down Guidance for Learning Object-Centric Representations [30.06924788022504]
  Top-Down Guided Network (TDGNet) is a top-down pathway to improve object-centric representations.<n>We show that TDGNet outperforms current object-centric models on multiple datasets of varying complexity.
  arXiv  Detail & Related papers  (2024-05-17T07:48:27Z)
• Wills Aligner: Multi-Subject Collaborative Brain Visual Decoding [19.538200208523467]
  We introduce Wills Aligner, a novel approach to achieve multi-subject collaborative brain visual decoding.<n>Wills Aligner begins by aligning the fMRI data from different subjects at the anatomical level.<n>It then employs delicate mixture-of-brain-expert adapters and a meta-learning strategy to account for individual fMRI pattern differences.
  arXiv  Detail & Related papers  (2024-04-20T06:01:09Z)
• See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI [32.40827290083577]
  Deciphering visual content from functional Magnetic Resonance Imaging (fMRI) helps illuminate the human vision system. Previous approaches primarily employ subject-specific models, sensitive to training sample size. We propose shallow subject-specific adapters to map cross-subject fMRI data into unified representations. During training, we leverage both visual and textual supervision for multi-modal brain decoding.
  arXiv  Detail & Related papers  (2024-03-11T01:18:49Z)
• Brain Captioning: Decoding human brain activity into images and text [1.5486926490986461]
  We present an innovative method for decoding brain activity into meaningful images and captions. Our approach takes advantage of cutting-edge image captioning models and incorporates a unique image reconstruction pipeline. We evaluate our methods using quantitative metrics for both generated captions and images.
  arXiv  Detail & Related papers  (2023-05-19T09:57:19Z)
• Controllable Mind Visual Diffusion Model [58.83896307930354]
  Brain signal visualization has emerged as an active research area, serving as a critical interface between the human visual system and computer vision models. We propose a novel approach, referred to as Controllable Mind Visual Model Diffusion (CMVDM) CMVDM extracts semantic and silhouette information from fMRI data using attribute alignment and assistant networks. We then leverage a control model to fully exploit the extracted information for image synthesis, resulting in generated images that closely resemble the visual stimuli in terms of semantics and silhouette.
  arXiv  Detail & Related papers  (2023-05-17T11:36:40Z)
• Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images [79.07468367923619]
  We develop convolutional neural generative coding (Conv-NGC) We implement a flexible neurobiologically-motivated algorithm that progressively refines latent state maps. We study the effectiveness of our brain-inspired neural system on the tasks of reconstruction and image denoising.
  arXiv  Detail & Related papers  (2022-11-22T06:42:41Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• Deep Recurrent Encoder: A scalable end-to-end network to model brain signals [122.1055193683784]
  We propose an end-to-end deep learning architecture trained to predict the brain responses of multiple subjects at once. We successfully test this approach on a large cohort of magnetoencephalography (MEG) recordings acquired during a one-hour reading task.
  arXiv  Detail & Related papers  (2021-03-03T11:39:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.