Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval

• URL: http://arxiv.org/abs/2604.17782v1
• Date: Mon, 20 Apr 2026 04:10:30 GMT
• Title: Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval
• Abstract summary: We propose a subject-aware multi-granularity alignment (SAMGA) framework for zero-shot EEG-to-image retrieval.<n>SAMGA first constructs a subject-aware visual supervision target by adaptively aggregating multiple intermediate representations from a pretrained vision encoder.<n>Experiments on the THINGS-EEG benchmark demonstrate that the proposed method achieves 91.3% Top-1 and 98.8% Top-5 accuracy in the intra-subject setting, and 34.4% Top-1 and 64.8% Top-5 accuracy in the inter-subject setting.
• Score: 6.9700202682590024
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot EEG-to-image retrieval aims to decode perceived visual content from electroencephalography (EEG) by aligning neural responses with pretrained visual representations, providing a promising route toward scalable visual neural decoding and practical brain-computer interfaces. However, robust EEG-to-image retrieval remains challenging, because prior methods usually rely on either a single fixed visual target or a subject-invariant target construction scheme. Such designs overlook two important properties of visually evoked EEG signals: they preserve information across multiple representational scales, and the visual granularity best matched to EEG may vary across subjects. To address these issues, subject-aware multi-granularity alignment (SAMGA) framework is proposed for zero-shot EEG-to-image retrieval. SAMGA first constructs a subject-aware visual supervision target by adaptively aggregating multiple intermediate representations from a pretrained vision encoder, allowing the model to absorb subject-dependent granularity deviations during training while preserving subject-agnostic inference. Building on this adaptive target construction, a coarse-to-fine cross-modal alignment strategy is further designed with a shared encoder wherein the coarse stage stabilizes the shared semantic geometry and reduces subject-induced distribution shift, and the fine stage further improves instance-level retrieval discrimination. Extensive experiments on the THINGS-EEG benchmark demonstrate that the proposed method achieves 91.3% Top-1 and 98.8% Top-5 accuracy in the intra-subject setting, and 34.4% Top-1 and 64.8% Top-5 accuracy in the inter-subject setting, outperforming recent state-of-the-art methods.

Related papers

• SCORE: Subject Coordinate Recovery for Label-Free Cross-Subject EEG-to-Image Retrieval [11.943434510766949]
  Current EEG-to-image retrieval methods perform far below their within-subject counterparts for new users without labeled calibration.<n>We propose Subject Coordinate Recovery (SCORE), a target label-free framework combining recovery-aware source training with coordinate alignment at deployment.
  arXiv  Detail & Related papers  (2026-08-19T17:27:29Z)
• What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment [16.54359569394021]
  We introduce a unified multiview EEG representation learning framework for aligning brain responses with visual semantic embeddings.<n>We show that our method achieves state-of-the-art performance, with 54.8% Top-1 and 85.6% Top-5 accuracy in the within-subject setting and 15.3% Top-1 and 45.4% Top-5 accuracy in the cross-subject setting.
  arXiv  Detail & Related papers  (2026-06-24T11:33:21Z)
• DARE-EEG: A Foundation Model for Mining Dual-Aligned Representation of EEG [25.4837936625773]
  We propose DARE-EEG, a self-supervised foundation model that enforces the mask-invariance property through dual-aligned representation learning.<n>Specifically, we introduce mask alignment that constrains representations from multiple masked views of the same EEG sample.<n>We also propose conv-linear-probing, a parameter-efficient strategy that adapts pre-trained representations to heterogeneous electrode configurations.
  arXiv  Detail & Related papers  (2026-05-18T12:18:43Z)
• Multi-Level Bidirectional Biomimetic Learning for EEG-Based Visual Decoding [10.0396148246889]
  EEG-based visual neural decoding aims to align neural responses with visual stimuli for tasks such as image retrieval.<n>We propose MB2L, a Multi-Level Bidirectional Biomimetic Learning framework that incorporates structured physiological biases into representation learning.<n>Experiments show MB2L achieves 80.5% Top-1 and 97.6% Top-5 accuracy on zero-shot EEG-to-image retrieval.
  arXiv  Detail & Related papers  (2026-05-06T09:31:35Z)
• Aligning What EEG Can See: Structural Representations for Brain-Vision Matching [26.36030588128271]
  Visual decoding from electroencephalography (EEG) has emerged as a highly promising avenue for non-invasive brain-computer interfaces.<n>Existing EEG-based decoding methods predominantly align brain signals with the final-layer semantic embeddings of deep visual models.<n>We introduce the concept of Neural Visibility and accordingly propose the EEG-Visible Layer Selection Strategy.
  arXiv  Detail & Related papers  (2026-03-07T07:12:08Z)
• Autoregressive Visual Decoding from EEG Signals [14.213172378363216]
  We present AVDE, a lightweight and efficient framework for visual decoding from EEG signals.<n>We adopt an autoregressive generative framework based on a "next-scale prediction" strategy.<n> Experiments on two datasets show that AVDE outperforms previous state-of-the-art methods in both image retrieval and reconstruction tasks.
  arXiv  Detail & Related papers  (2026-02-26T02:49:04Z)
• SYNAPSE: Synergizing an Adapter and Finetuning for High-Fidelity EEG Synthesis from a CLIP-Aligned Encoder [0.0]
  SYNAPSE is a two-stage framework that bridges EEG signal representation learning and high-fidelity image synthesis.<n>Our method achieves a semantically coherent latent space and state-of-the-art perceptual fidelity on the CVPR40 dataset.
  arXiv  Detail & Related papers  (2025-11-11T02:53:49Z)
• CATVis: Context-Aware Thought Visualization [2.8298952038412706]
  We propose a novel 5-stage framework for decoding visual representations from EEG signals.<n>We enable context-aware EEG-to-image generation through cross-modal alignment and re-ranking.<n> Experimental results demonstrate that our method generates high-quality images aligned with visual stimuli.
  arXiv  Detail & Related papers  (2025-07-15T17:47:01Z)
• CRIA: A Cross-View Interaction and Instance-Adapted Pre-training Framework for Generalizable EEG Representations [52.251569042852815]
  CRIA is an adaptive framework that utilizes variable-length and variable-channel coding to achieve a unified representation of EEG data across different datasets.<n>The model employs a cross-attention mechanism to fuse temporal, spectral, and spatial features effectively.<n> Experimental results on the Temple University EEG corpus and the CHB-MIT dataset show that CRIA outperforms existing methods with the same pre-training conditions.
  arXiv  Detail & Related papers  (2025-06-19T06:31:08Z)
• CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information [61.1904164368732]
  We propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals.<n>Specifically, CognitionCapturer trains Modality Experts for each modality to extract cross-modal information from the EEG modality.<n>The framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities.
  arXiv  Detail & Related papers  (2024-12-13T16:27:54Z)
• Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning [2.087148326341881]
  This paper introduces a MUltimodal Similarity-keeping contrastivE learning framework for zero-shot EEG-based image classification. We develop a series of multivariate time-series encoders tailored for EEG signals and assess the efficacy of regularized contrastive EEG-Image pretraining. Our method achieves state-of-the-art performance, with a top-1 accuracy of 19.3% and a top-5 accuracy of 48.8% in 200-way zero-shot image classification.
  arXiv  Detail & Related papers  (2024-06-05T16:42:23Z)
• Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection [106.39544368711427]
  We study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods. We present a novel forgery-aware adaptive transformer approach, namely FatFormer. Our approach tuned on 4-class ProGAN data attains an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.
  arXiv  Detail & Related papers  (2023-12-27T17:36:32Z)
• CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversion [49.1574468325115]
  A key aim in EEG analysis is to extract the underlying neural activation (content) as well as to account for the individual subject variability (style) Inspired by recent advancements in voice conversion technologies, we propose a novel contrastive split-latent permutation autoencoder (CSLP-AE) framework that directly optimize for EEG conversion.
  arXiv  Detail & Related papers  (2023-11-13T22:46:43Z)
• Robust Single Image Dehazing Based on Consistent and Contrast-Assisted Reconstruction [95.5735805072852]
  We propose a novel density-variational learning framework to improve the robustness of the image dehzing model. Specifically, the dehazing network is optimized under the consistency-regularized framework. Our method significantly surpasses the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-29T08:11:04Z)
• Gradient-Induced Co-Saliency Detection [81.54194063218216]
  Co-saliency detection (Co-SOD) aims to segment the common salient foreground in a group of relevant images. In this paper, inspired by human behavior, we propose a gradient-induced co-saliency detection method.
  arXiv  Detail & Related papers  (2020-04-28T08:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.