Neuroscience-inspired Staged Representation Learning with Disentangled Coarse- and Fine-Grained Semantics for EEG Visual Decoding

• URL: http://arxiv.org/abs/2605.16923v3
• Date: Thu, 21 May 2026 03:22:39 GMT
• Title: Neuroscience-inspired Staged Representation Learning with Disentangled Coarse- and Fine-Grained Semantics for EEG Visual Decoding
• Abstract summary: Existing EEG visual decoding methods mainly focus on learning a single global EEG embedding for cross-modal alignment.<n>We propose a neuroscience-inspired staged representation learning framework that reformulates EEG visual decoding as a stage-specific representation decomposition problem.<n>The proposed framework organizes EEG representation learning into three complementary phases: low-level visual representation learning, high-level semantic representation learning, and integrative information fusion.
• Score: 20.814486415153404
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decoding visual information from electroencephalography (EEG) signals remains a fundamental challenge in brain-computer interfaces and medical rehabilitation. Existing EEG visual decoding methods mainly focus on learning a single global EEG embedding for cross-modal alignment, but they largely overlook the staged and hierarchical characteristics of human visual processing. To address this limitation, we propose a neuroscience-inspired staged representation learning framework that reformulates EEG visual decoding as a stage-specific representation decomposition problem. The proposed framework organizes EEG representation learning into three complementary phases: low-level visual representation learning, high-level semantic representation learning, and integrative information fusion. To strengthen semantic modeling, we further introduce a multimodal dual-level semantic learning mechanism that separates coarse label-level semantics from fine image-level visual-semantic information. In addition, semantic latent channels are introduced as computational representation channels generated from observed visual EEG signals, expanding the channel-level semantic representation space for structured semantic abstraction and cross-modal alignment. Extensive experiments on the THINGS-EEG benchmark demonstrate that the proposed method achieves superior performance under subject-dependent zero-shot evaluation and improved exact retrieval under subject-independent zero-shot evaluation. Additional analyses, including layer-wise retrieval, temporal accumulation, expanded multi-image retrieval, and ablation studies, further support the effectiveness of staged decomposition and structured semantic modeling. These results suggest that explicitly modeling staged perceptual, semantic, and integrative representations provides an effective neuroscience-inspired framework for EEG-based visual decoding.

Related papers

• ViQ: Text-Aligned Visual Quantized Representations at Any Resolution [91.46185855575789]
  We present ViQ, a Visual Quantized Representations framework, which is designed to balance semantics and details in discrete representations.<n>Our approach structures quantization learning into two stages: text-aligned pre-training and feature discretization.<n>Experiments on multimodal tasks demonstrate that ViQ achieves competitive performance compared to state-of-the-art multimodal vision encoders.
  arXiv  Detail & Related papers  (2026-06-25T17:29:27Z)
• Structure-Guided Diffusion Model for EEG-Based Visual Cognition Reconstruction [2.77392799295445]
  Decoding visual information from electroencephalography (EEG) is an important problem in neuroscience and brain-computer interface (BCI) research.<n>We propose a Structure-Guided Diffusion Model (SGDM) that incorporates explicit structural information for EEG-based visual reconstruction.
  arXiv  Detail & Related papers  (2026-04-24T15:20:59Z)
• Aligning What EEG Can See: Structural Representations for Brain-Vision Matching [26.36030588128271]
  Visual decoding from electroencephalography (EEG) has emerged as a highly promising avenue for non-invasive brain-computer interfaces.<n>Existing EEG-based decoding methods predominantly align brain signals with the final-layer semantic embeddings of deep visual models.<n>We introduce the concept of Neural Visibility and accordingly propose the EEG-Visible Layer Selection Strategy.
  arXiv  Detail & Related papers  (2026-03-07T07:12:08Z)
• Bridging Cognitive Gap: Hierarchical Description Learning for Artistic Image Aesthetics Assessment [51.40989269202702]
  aesthetic quality assessment task is crucial for developing a human-aligned quantitative evaluation system for AIGC.<n>We propose ArtQuant, an aesthetics assessment framework for artistic images which couples isolated aesthetic dimensions through description generation.<n>Our approach achieves epoch state-of-the-art performance on several datasets while requiring only 33% of conventional trainings.
  arXiv  Detail & Related papers  (2025-12-29T12:18:26Z)
• WaveMind: Towards a Conversational EEG Foundation Model Aligned to Textual and Visual Modalities [55.00677513249723]
  EEG signals simultaneously encode both cognitive processes and intrinsic neural states.<n>We map EEG signals and their corresponding modalities into a unified semantic space to achieve generalized interpretation.<n>The resulting model demonstrates robust classification accuracy while supporting flexible, open-ended conversations.
  arXiv  Detail & Related papers  (2025-09-26T06:21:51Z)
• Interpretable EEG-to-Image Generation with Semantic Prompts [6.712646807032639]
  Our model bypasses direct EEG-to-image generation by aligning EEG signals with semantic captions.<n>A transformer-based EEG encoder maps brain activity to these captions through contrastive learning.<n>This text-mediated framework yields state-of-the-art visual decoding on the EEGCVPR dataset.
  arXiv  Detail & Related papers  (2025-07-09T17:18:06Z)
• ViEEG: Hierarchical Visual Neural Representation for EEG Brain Decoding [18.51835182602402]
  ViEEG is a neuro-We further adopt hierarchical contrastive learning for EEG-CLIP representation alignment, enabling zero-shot object recognition.<n>Our framework sets a new paradigm for EEG brain decoding.<n>ViEEG decomposes each visual stimulus into three biologically aligned components-contour, foreground object, and contextual scene-serving anchors for a three-stream EEG encoder.
  arXiv  Detail & Related papers  (2025-05-18T13:19:08Z)
• CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information [61.1904164368732]
  We propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals.<n>Specifically, CognitionCapturer trains Modality Experts for each modality to extract cross-modal information from the EEG modality.<n>The framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities.
  arXiv  Detail & Related papers  (2024-12-13T16:27:54Z)
• Visual Neural Decoding via Improved Visual-EEG Semantic Consistency [3.4061238650474657]
  Methods that directly map EEG features to the CLIP embedding space may introduce mapping bias and cause semantic inconsistency. We propose a Visual-EEG Semantic Decouple Framework that explicitly extracts the semantic-related features of these two modalities to facilitate optimal alignment. Our method achieves state-of-the-art results in zero-shot neural decoding tasks.
  arXiv  Detail & Related papers  (2024-08-13T10:16:10Z)
• MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning [48.97640824497327]
  We propose a novel framework leveraging domain-specific medical knowledge as guiding signals to integrate language information into the visual domain through image-text contrastive learning. Our model includes global contrastive learning with our designed divergence encoder, local token-knowledge-patch alignment contrastive learning, and knowledge-guided category-level contrastive learning with expert knowledge. Notably, MLIP surpasses state-of-the-art methods even with limited annotated data, highlighting the potential of multimodal pre-training in advancing medical representation learning.
  arXiv  Detail & Related papers  (2024-02-03T05:48:50Z)
• Controllable Mind Visual Diffusion Model [58.83896307930354]
  Brain signal visualization has emerged as an active research area, serving as a critical interface between the human visual system and computer vision models. We propose a novel approach, referred to as Controllable Mind Visual Model Diffusion (CMVDM) CMVDM extracts semantic and silhouette information from fMRI data using attribute alignment and assistant networks. We then leverage a control model to fully exploit the extracted information for image synthesis, resulting in generated images that closely resemble the visual stimuli in terms of semantics and silhouette.
  arXiv  Detail & Related papers  (2023-05-17T11:36:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.