Lost in Visual Translation: A VLM-Assisted Perceptual-Semantic Coherence Framework for EEG-to-Image Reconstruction

• URL: http://arxiv.org/abs/2607.12364v1
• Date: Tue, 14 Jul 2026 05:40:23 GMT
• Title: Lost in Visual Translation: A VLM-Assisted Perceptual-Semantic Coherence Framework for EEG-to-Image Reconstruction
• Abstract summary: EEG-to-image evaluation should distinguish visual fidelity from recoverable meaning.<n>SSIM, LPIPS, and CLIP penalize semantically recoverable outputs or reward plausible but incorrect ones.<n>We analyze 6,855 ground-truth/reconstruction pairs from ATM, ENIGMA, BrainVis, and DreamDiffusion.
• Score: 0.10499611180329804
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: EEG-to-image evaluation should distinguish visual fidelity from recoverable meaning. Yet EEG-derived reconstructions are blurry, distorted, and low-detail, causing SSIM, LPIPS, and CLIP to penalize semantically recoverable outputs or reward plausible but incorrect ones. We analyze 6,855 ground-truth/reconstruction pairs from ATM, ENIGMA, BrainVis, and DreamDiffusion using semantic probes, caption harshness and blind-spot rates, and controlled degradations. Pixel metrics show near-zero correlation with semantic consistency, while representation metrics conflate perceptual and semantic errors. We therefore introduce a BCI-aware framework in which four VLMs assess image pairs through structured questions, producing Tolerant Perceptual Alignment Scores (T-PAS) and Tolerant Semantic Alignment Scores (T-SAS). Their consensus is distilled into the BCI-Coherence Score (BCS), a compact evaluator achieving a T-PAS MAE of 0.079 (r = 0.700) and a T-SAS MAE of 0.082 (r = 0.850) on our data. Human validation shows highly reliable joint coherence judgments, with Cohen's kappa = 0.882 +/- 0.174 and Krippendorff's alpha = 0.882, supporting perceptual-semantic recoverability over generic visual similarity. Code and resources are available at https://sukt03.github.io/BCS/.

Related papers

• A Dynamic-Semantics Framework for Grounding Human Referring Expressions in Visual Perceptual Data [0.0]
  We present a framework that externalizes pact state into three explicit, inspectable sets of referent-object bindings.<n>We evaluate the framework on the Stanford Repeated Reference Game corpus.
  arXiv  Detail & Related papers  (2026-08-09T12:14:10Z)
• Support Operation Factorization: Compositional Readout of Frozen Vision Encoders under Controlled Interventions [87.95757610455173]
  We introduce an injectively aligned leave-one-cell-out protocol over support x operation grids and SO-OPF.<n>We separate two questions that aggregate scores conflate: whether the carrier composes held-out bindings when the grid is known, and whether that grid can be recovered from flat cell labels.
  arXiv  Detail & Related papers  (2026-08-06T15:38:55Z)
• Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models [11.172591458557832]
  Vision-language models expose a similarity interface that enables zero-shot retrieval but fails compositional constraints.<n>We propose factored inference, which separates evidence extraction from constraint execution.<n>We also introduce FACTOR-Bench, where LCSE achieves 85.5% accuracy versus 73.2% for the best fine-tuned baseline.
  arXiv  Detail & Related papers  (2026-07-25T05:39:51Z)
• Template Collapse and Information-Theoretic Limits in Camera rPPG Pulse Morphology Restoration [0.0]
  Consumer face camera remote photoplethys hallucination (r) enables passive cardiovascular monitoring.<n>But whether single-cycle waveform morphology encoding arterial stiffness is recoverable from this measurement has not been characterised.<n>We evaluated 16 encoder architectures spanning six families on 153 subjects across three datasets.
  arXiv  Detail & Related papers  (2026-06-02T15:50:07Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Symbolic Augmentation Closes a Canonical-Equivalence Blind Spot in Neural Fact-Checkers [1.7844382164707184]
  We introduce a five-class typed-quantity error taxonomy and a 1500-item benchmark.<n>A ModernBERT encoder fine-tuned on this benchmark reaches macro-F1 = 0.899, far above any off-the-shelf neural fact-checker.<n>We propose Symbolic Augmentation, a training-time framework that runs the modules of a symbolic verifier in reverse to generate label-preserving augmented training data.
  arXiv  Detail & Related papers  (2026-05-19T15:58:00Z)
• PHANTOM: Polymorphic Honeytoken Adaptation with Narrative-Tailored Organisational Mimicry [0.2538209532048867]
  Honeytokens are decoy digital assets planted to detect and unauthorised attribute access.<n>We introduce PHANTOM, a framework that generates contextually convincing honeytokens by encoding organisation-specific knowledge.<n>Results are reproduced without external API calls, making the pipeline fully deployable in air-gapped environments.
  arXiv  Detail & Related papers  (2026-05-04T17:14:13Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• KoALA: KL-L0 Adversarial Detector via Label Agreement [9.360893727306324]
  KoALA is a semantics-free adversarial detector for deep neural networks.<n>It detects adversarial attacks when class predictions from two complementary similarity metrics disagree.<n>KoALA achieves a precision of 0.94 and a recall of 0.81 on ResNet/CIFAR-10, and a precision of 0.66 and a recall of 0.85 on CLIP/Tiny-ImageNet.
  arXiv  Detail & Related papers  (2025-10-14T17:33:00Z)
• Group Evidence Matters: Tiling-based Semantic Gating for Dense Object Detection [0.0]
  This paper presents a detector-agnostic post-processing framework that converts overlap-induced redundancy into group evidence.<n>The framework requires no retraining and integrates with modern detectors.
  arXiv  Detail & Related papers  (2025-09-13T01:55:08Z)
• LipSim: A Provably Robust Perceptual Similarity Metric [56.03417732498859]
  We show the vulnerability of state-of-the-art perceptual similarity metrics based on an ensemble of ViT-based feature extractors to adversarial attacks. We then propose a framework to train a robust perceptual similarity metric called LipSim with provable guarantees. LipSim provides guarded areas around each data point and certificates for all perturbations within an $ell$ ball.
  arXiv  Detail & Related papers  (2023-10-27T16:59:51Z)
• Multi-View Correlation Consistency for Semi-Supervised Semantic Segmentation [59.34619548026885]
  Semi-supervised semantic segmentation needs rich and robust supervision on unlabeled data. We propose a view-coherent data augmentation strategy that guarantees pixel-pixel correspondence between different views. In a series of semi-supervised settings on two datasets, we report competitive accuracy compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-08-17T17:59:11Z)
• Introspective Deep Metric Learning for Image Retrieval [80.29866561553483]
  We argue that a good similarity model should consider the semantic discrepancies with caution to better deal with ambiguous images for more robust training. We propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively. The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling and attains state-of-the-art results on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets.
  arXiv  Detail & Related papers  (2022-05-09T17:51:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.