SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding

• URL: http://arxiv.org/abs/2606.16615v2
• Date: Thu, 18 Jun 2026 11:13:48 GMT
• Title: SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding
• Abstract summary: Non-invasive brain-computer interfaces exhibit significant performance degradation when moving from controlled laboratory stimuli to real-world natural images.<n>This degradation occurs because multimodal contrastive representation learning models fail to account for semantic consistency and inter-subject variability in neural representation and selective attention.<n>We propose SUP-MCRL, a unified framework integrating three collaborative mechanisms to overcome these limitations.
• Score: 4.793866271591739
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Non-invasive brain-computer interfaces exhibit significant performance degradation when moving from controlled laboratory stimuli to real-world natural images. This degradation occurs because conventional multimodal contrastive representation learning models focus exclusively on optimizing geometric distance alignment, thereby failing to account for semantic consistency and inter-subject variability in neural representation and selective attention. As a result, these models are prone to producing spurious zero-shot matches. To address these limitations, we propose SUP-MCRL, a unified framework integrating three collaborative mechanisms: (1) a Semantic-entity Aware Visual Encoder (SAVE) that learns spatial attention to extract semantic content without relying on pre-trained saliency models; (2) a Unified EEG Enhancer (UEE) that employs multi-scale atrous convolutions and inter-band attention for adaptive cross-subject robustness; and (3) a Prototype-based Progressive Augmenter (PPA) that maintains an EMA-updated pseudo-feature pool to prevent representation collapse. Zero-shot experiments on the THINGS-EEG achieve 66.0%/91.9% (Top-1/Top-5) intra-subject and 24.0%/52.9% LOSO accuracy, significantly surpassing state-of-the-art methods and demonstrating that structured alignment supervision is key to overcoming the limitations of cross-modal decoding. Code is available at https://github.com/NZWANG/SUP-MCRL.

Related papers

• DAC-Pose: Dual-Agent Collaborative Framework for Pose-Guided Human Generation [14.847948534749001]
  In pose-guided human generation, conventional methods inevitably produce severe visual artifacts under drastic viewpoint shifts.<n>We propose DAC-Pose, a novel agent-driven multimodal framework that reformulates single-view human generation as a collaborative dual-agent system.<n>We show that DAC-Pose excels in preserving texture alignment and identity consistency under drastic viewpoint changes.
  arXiv  Detail & Related papers  (2026-08-05T09:27:12Z)
• Where Does Generative Difficulty Reside? An Empirical Study of Target Representations [62.54876287800644]
  The target representation defines the distribution an image generator must learn, yet it is often treated as an interchangeable interface.<n>We study raw pixels, SD-VAE latents and DINOv2 as well as MAE representation-autoencoder features within a unified masked autoregressive rectified-flow model.<n>Our results show that compression, reconstruction fidelity, token dimensionality, and visible semantic clustering do not individually predict generative behavior.
  arXiv  Detail & Related papers  (2026-08-01T12:32:31Z)
• Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment [0.4790056963046066]
  MST-CLIPIQA is a multi-scale two-stream framework that achieves hierarchical vision-language alignment through explicit representational decoupling.<n>Our architecture leverages dual CLIP encoders with complementary patch granularities, capturing global semantic coherence.<n>Experiments establish new state-of-the-art results, achieving average improvements of 1.11 percent SRCC on quality and 2.35 percent SRCC on text-image correspondence prediction.
  arXiv  Detail & Related papers  (2026-06-15T14:40:30Z)
• MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding [36.958608375007124]
  Visual decoding from brain signals is a key challenge at the intersection of computer vision and neuroscience.<n>We introduce a tri-modal contrastive framework for EEG-based visual decoding that aligns EEG, visual, and textual representations.
  arXiv  Detail & Related papers  (2026-05-23T11:23:21Z)
• Hyperbolic Enhanced Representation Learning for Incomplete Multi-view Clustering [57.38215918201251]
  We propose HERL, a Hyperbolic Enhanced Representation Learning framework for Incomplete Multi-View Clustering.<n> operating within the Poincaré ball, HERL constructs a structure-aware latent space to enhance representation learning.<n>We show that HERL consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-04-18T10:50:46Z)
• GRAFNet: Multiscale Retinal Processing via Guided Cortical Attention Feedback for Enhancing Medical Image Polyp Segmentation [6.834321209531585]
  We propose GRAFNet, a biologically inspired architecture that emulates the hierarchical organisation of the human visual system.<n>GRAFNet integrates three key modules: (1) a Guided Asymmetric Attention Module (GAAM) that mimics orientation-tuned cortical neurones to polyp boundaries, (2) a MultiScale Retinal Module (MSRM) that replicates retinal ganglion cell pathways for parallel multi-feature analysis, and (3) a Guided Cortical Attention Feedback Module (GCAFM) that applies predictive coding for iterative refinement.
  arXiv  Detail & Related papers  (2026-02-15T17:29:37Z)
• SSVP: Synergistic Semantic-Visual Prompting for Industrial Zero-Shot Anomaly Detection [55.54007781679915]
  We propose Synergistic Semantic-Visual Prompting (SSVP), that efficiently fuses diverse visual encodings to elevate model's fine-grained perception.<n>SSVP achieves state-of-the-art performance with 93.0% Image-AUROC and 92.2% Pixel-AUROC on MVTec-AD, significantly outperforming existing zero-shot approaches.
  arXiv  Detail & Related papers  (2026-01-14T04:42:19Z)
• STORE: Semantic Tokenization, Orthogonal Rotation and Efficient Attention for Scaling Up Ranking Models [11.965535230928372]
  Store is a unified and scalable token-based ranking framework built upon three core innovations.<n>Our framework consistently improves prediction accuracy(online CTR by 2.71%, AUC by 1.195%) and training effeciency (1.84 throughput)
  arXiv  Detail & Related papers  (2025-11-24T06:20:02Z)
• Filling the Gaps: A Multitask Hybrid Multiscale Generative Framework for Missing Modality in Remote Sensing Semantic Segmentation [28.992992584085787]
  Multimodal learning has shown significant performance boost compared to ordinary unimodal models.<n>In real-world scenarios, multimodal signals are susceptible to missing because of sensor failures and adverse weather conditions.<n>We propose a novel Generative-Enhanced MultiModal learning Network (GEMMNet) to tackle these limitations.
  arXiv  Detail & Related papers  (2025-09-14T05:40:35Z)
• Towards Efficient General Feature Prediction in Masked Skeleton Modeling [59.46799426434277]
  We propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling.<n>Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations.
  arXiv  Detail & Related papers  (2025-09-03T18:05:02Z)
• CCL-LGS: Contrastive Codebook Learning for 3D Language Gaussian Splatting [53.15827818829865]
  Methods that rely on 2D priors are prone to a critical challenge: cross-view semantic inconsistencies.<n>We propose CCL-LGS, a novel framework that enforces view-consistent semantic supervision by integrating multi-view semantic cues.<n>Our framework explicitly resolves semantic conflicts while preserving category discriminability.
  arXiv  Detail & Related papers  (2025-05-26T19:09:33Z)
• Manifold-aware Representation Learning for Degradation-agnostic Image Restoration [135.90908995927194]
  Image Restoration (IR) aims to recover high quality images from degraded inputs affected by various corruptions such as noise, blur, haze, rain, and low light conditions.<n>We present MIRAGE, a unified framework for all in one IR that explicitly decomposes the input feature space into three semantically aligned parallel branches.<n>This modular decomposition significantly improves generalization and efficiency across diverse degradations.
  arXiv  Detail & Related papers  (2025-05-24T12:52:10Z)
• CFMD: Dynamic Cross-layer Feature Fusion for Salient Object Detection [7.262250906929891]
  Cross-layer feature pyramid networks (CFPNs) have achieved notable progress in multi-scale feature fusion and boundary detail preservation for salient object detection.<n>To address these challenges, we propose CFMD, a novel cross-layer feature pyramid network that introduces two key innovations.<n>First, we design a context-aware feature aggregation module (CFLMA), which incorporates the state-of-the-art Mamba architecture to construct a dynamic weight distribution mechanism.<n>Second, we introduce an adaptive dynamic upsampling unit (CFLMD) that preserves spatial details during resolution recovery.
  arXiv  Detail & Related papers  (2025-04-02T03:22:36Z)
• Semi-supervised Semantic Segmentation with Multi-Constraint Consistency Learning [81.02648336552421]
  We propose a Multi-Constraint Consistency Learning approach to facilitate the staged enhancement of the encoder and decoder.<n>Self-adaptive feature masking and noise injection are designed in an instance-specific manner to perturb the features for robust learning of the decoder.<n> Experimental results on Pascal VOC2012 and Cityscapes datasets demonstrate that our proposed MCCL achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-23T03:21:33Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR [50.435592120607815]
  Scene graph generation (SGG) of surgical procedures is crucial in enhancing holistically cognitive intelligence in the operating room (OR) Previous works have primarily relied on multi-stage learning, where the generated semantic scene graphs depend on intermediate processes with pose estimation and object detection. In this study, we introduce a novel single-stage bi-modal transformer framework for SGG in the OR, termed S2Former-OR.
  arXiv  Detail & Related papers  (2024-02-22T11:40:49Z)
• A Generically Contrastive Spatiotemporal Representation Enhancement for 3D Skeleton Action Recognition [10.403751563214113]
  We propose a Contrastive Spatiotemporal Representation Enhancement (CSRE) framework to obtain more discriminative representations from the sequences.<n>Specifically, we decompose the representation into spatial- and temporal-specific features to explore fine-grained motion patterns.<n>To explicitly exploit the latent data distributions, we employ the attentive features to contrastive learning, which models the cross-sequence semantic relations.
  arXiv  Detail & Related papers  (2023-12-23T02:54:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.