CORTIVA: Candidate-Score Fusion of Complementary Visual Teachers for EEG- and MEG-to-Image Retrieval

• URL: http://arxiv.org/abs/2608.01355v1
• Date: Sun, 02 Aug 2026 16:14:22 GMT
• Title: CORTIVA: Candidate-Score Fusion of Complementary Visual Teachers for EEG- and MEG-to-Image Retrieval
• Abstract summary: Decoding visual experience from non-invasive brain activity is central to neuroscience and brain-computer interfaces.<n>EEG and MEG provide millisecond resolution, making image retrieval compelling.<n>Most systems collapse heterogeneous visual supervision into a single embedding before ranking.<n>We propose CORTIVA, a candidate-score fusion framework that preserves this complementary evidence.
• Score: 4.19305370463618
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decoding visual experience from non-invasive brain activity is central to neuroscience and brain-computer interfaces. Functional magnetic resonance imaging (fMRI) offers fine spatial detail, but its slow hemodynamics and burdensome acquisition limit temporally resolved decoding. Electroencephalography (EEG) and magnetoencephalography (MEG) provide millisecond resolution, making image retrieval compelling: identify the viewed image from one neural response and a fixed candidate bank. Contrastive alignment to pretrained visual representations enables zero-shot retrieval from EEG and MEG, but most systems collapse heterogeneous visual supervision into a single embedding before ranking. This early consolidation imposes one similarity geometry on every candidate order and removes encoder-specific disagreements from the final ranking. We propose CORTIVA, a candidate-score fusion framework that preserves this complementary evidence. Three decoding routes are aligned to heterogeneous visual targets, score the same indexed candidates independently, and combine only their temperature-scaled score vectors before ranking. On the 200-way THINGS-EEG2 benchmark, CORTIVA reaches 73.5% Top-1 and 95.3% Top-5 across ten participants, exceeding the strongest reported baseline by 10.3 and 5.4 percentage points. With a modality-specific neural encoder, the same fusion principle reaches 42.4% Top-1 on THINGS-MEG. Matched route-removal retraining and four weight controls demonstrate that CORTIVA's gain arises from integrating complementary route scores and persists with uniform weighting, without requiring a specialized weighting rule. Independent DINOv2 analyses further reproduce the local error neighborhoods and posterior neural-visual correspondence. These results establish candidate-score fusion as a simple and testable alternative to embedding-level consolidation for neural image retrieval.

Related papers

• Medical Foundation Model Features as Perceptual Loss for Brain MRI Contrast Dose Simulation [1.5123472914752591]
  Most perceptual losses are still computed with natural-image backbones such as VGG16 or ResNet50.<n>This mismatch may weaken supervision for anatomy, contrast enhancement, and acquisition variability.<n>We test whether medical foundation model features provide a more suitable perceptual loss for brain MRI contrast dose simulation.
  arXiv  Detail & Related papers  (2026-08-28T18:30:34Z)
• SCORE: Subject Coordinate Recovery for Label-Free Cross-Subject EEG-to-Image Retrieval [11.943434510766949]
  Current EEG-to-image retrieval methods perform far below their within-subject counterparts for new users without labeled calibration.<n>We propose Subject Coordinate Recovery (SCORE), a target label-free framework combining recovery-aware source training with coordinate alignment at deployment.
  arXiv  Detail & Related papers  (2026-08-19T17:27:29Z)
• What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment [16.54359569394021]
  We introduce a unified multiview EEG representation learning framework for aligning brain responses with visual semantic embeddings.<n>We show that our method achieves state-of-the-art performance, with 54.8% Top-1 and 85.6% Top-5 accuracy in the within-subject setting and 15.3% Top-1 and 45.4% Top-5 accuracy in the cross-subject setting.
  arXiv  Detail & Related papers  (2026-06-24T11:33:21Z)
• Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval [6.9700202682590024]
  We propose a subject-aware multi-granularity alignment (SAMGA) framework for zero-shot EEG-to-image retrieval.<n>SAMGA first constructs a subject-aware visual supervision target by adaptively aggregating multiple intermediate representations from a pretrained vision encoder.<n>Experiments on the THINGS-EEG benchmark demonstrate that the proposed method achieves 91.3% Top-1 and 98.8% Top-5 accuracy in the intra-subject setting, and 34.4% Top-1 and 64.8% Top-5 accuracy in the inter-subject setting.
  arXiv  Detail & Related papers  (2026-04-20T04:10:30Z)
• Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstruction [65.67001243986981]
  We propose MindHier, a coarse-to-fine fMRI-to-image reconstruction framework built on scale-wise autoregressive modeling.<n>MindHier achieves superior semantic fidelity, 4.67x faster inference, and more deterministic results than the diffusion-based baselines.
  arXiv  Detail & Related papers  (2025-10-25T15:40:07Z)
• Brain Tumor Classification on MRI in Light of Molecular Markers [56.99710477905796]
  Co-deletion of the 1p/19q gene is associated with clinical outcomes in low-grade gliomas.<n>This study aims to utilize a specially MRI-based convolutional neural network for brain cancer detection.
  arXiv  Detail & Related papers  (2024-09-29T07:04:26Z)
• Brain decoding: toward real-time reconstruction of visual perception [1.3654846342364308]
  In the past five years, the use of generative and foundational AI systems has greatly improved the decoding of brain activity. Visual perception can now be decoded from functional Magnetic Resonance Imaging (fMRI) with remarkable fidelity. Here, we propose an alternative approach based on magnetoencephalography (MEG), a neuroimaging device capable of measuring brain activity with high temporal resolution.
  arXiv  Detail & Related papers  (2023-10-18T09:51:38Z)
• ECG Heartbeat Classification Using Multimodal Fusion [13.524306011331303]
  We propose two computationally efficient multimodal fusion frameworks for ECG heart beat classification. In MFF, we extracted features from penultimate layer of CNNs and fused them to get unique and interdependent information. We achieved classification accuracy of 99.7% and 99.2% on arrhythmia and MI classification, respectively.
  arXiv  Detail & Related papers  (2021-07-21T03:48:35Z)
• Improved Slice-wise Tumour Detection in Brain MRIs by Computing Dissimilarities between Latent Representations [68.8204255655161]
  Anomaly detection for Magnetic Resonance Images (MRIs) can be solved with unsupervised methods. We have proposed a slice-wise semi-supervised method for tumour detection based on the computation of a dissimilarity function in the latent space of a Variational AutoEncoder. We show that by training the models on higher resolution images and by improving the quality of the reconstructions, we obtain results which are comparable with different baselines.
  arXiv  Detail & Related papers  (2020-07-24T14:02:09Z)
• Context-Aware Refinement Network Incorporating Structural Connectivity Prior for Brain Midline Delineation [50.868845400939314]
  We propose a context-aware refinement network (CAR-Net) to refine and integrate the feature pyramid representation generated by the UNet. For keeping the structural connectivity of the brain midline, we introduce a novel connectivity regular loss. The proposed method requires fewer parameters and outperforms three state-of-the-art methods in terms of four evaluation metrics.
  arXiv  Detail & Related papers  (2020-07-10T14:01:20Z)
• A Global Benchmark of Algorithms for Segmenting Late Gadolinium-Enhanced Cardiac Magnetic Resonance Imaging [90.29017019187282]
  " 2018 Left Atrium Challenge" using 154 3D LGE-MRIs, currently the world's largest cardiac LGE-MRI dataset. Analyse of the submitted algorithms using technical and biological metrics was performed. Results show the top method achieved a dice score of 93.2% and a mean surface to a surface distance of 0.7 mm.
  arXiv  Detail & Related papers  (2020-04-26T08:49:17Z)
• AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching [76.4844593082362]
  We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS) Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2020-04-26T05:43:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.