Brain-Grasp: Graph-based Saliency Priors for Improved fMRI-based Visual Brain Decoding

• URL: http://arxiv.org/abs/2604.10617v1
• Date: Sun, 12 Apr 2026 12:50:49 GMT
• Title: Brain-Grasp: Graph-based Saliency Priors for Improved fMRI-based Visual Brain Decoding
• Abstract summary: Recent progress in brain-guided image generation has improved the quality of fMRI-based reconstructions.<n>We propose a saliency-driven decoding framework that employs graph-informed saliency priors to translate structural cues from brain signals into spatial masks.
• Score: 1.130221069212388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in brain-guided image generation has improved the quality of fMRI-based reconstructions; however, fundamental challenges remain in preserving object-level structure and semantic fidelity. Many existing approaches overlook the spatial arrangement of salient objects, leading to conceptually inconsistent outputs. We propose a saliency-driven decoding framework that employs graph-informed saliency priors to translate structural cues from brain signals into spatial masks. These masks, together with semantic information extracted from embeddings, condition a diffusion model to guide image regeneration, helping preserve object conformity while maintaining natural scene composition. In contrast to pipelines that invoke multiple diffusion stages, our approach relies on a single frozen model, offering a more lightweight yet effective design. Experiments show that this strategy improves both conceptual alignment and structural similarity to the original stimuli, while also introducing a new direction for efficient, interpretable, and structurally grounded brain decoding.

Related papers

• Versatile Framework with Semantic and Structural guidance for Image Reconstruction from Brain Activity [17.625829377712492]
  We propose a two-stage image reconstruction framework, termed MindDiffuser.<n>In Stage 1, Contrastive Language-Image Pretraining (CLIP) text embeddings decoded from brain responses are input into Stable Diffusion, generating a preliminary image.<n>In Stage 2, we use decoded shallow CLIP visual features as supervisory signals, iteratively refining the feature vectors from Stage 1 via backpropagation to align structural information.
  arXiv  Detail & Related papers  (2026-05-28T09:20:10Z)
• Seeing the imagined: a latent functional alignment in visual imagery decoding from fMRI data [0.0]
  We study how a state-of-the-art perception decoder can be adapted to reconstruct imagined content from the Imagery-NSD benchmark.<n>We propose a latent functional alignment approach that maps imagery-evoked activity into the pretrained model's conditioning space.<n>To mitigate the limited amount of matched imagery-perception supervision, we introduce a retrieval-based augmentation strategy.
  arXiv  Detail & Related papers  (2026-04-15T16:31:57Z)
• SynMind: Reducing Semantic Hallucination in fMRI-Based Image Reconstruction [52.34513874272676]
  We argue that existing methods rely too heavily on entangled visual embeddings over explicit semantic identity.<n>We parse fMRI signals into rich, sentence-level semantic descriptions that mirror the hierarchical and compositional nature of human visual understanding.<n>We propose SynMind, a framework that integrates these explicit semantic encodings with visual priors to condition a pretrained diffusion model.
  arXiv  Detail & Related papers  (2026-01-25T14:31:23Z)
• On the Utility of Foundation Models for Fast MRI: Vision-Language-Guided Image Reconstruction [5.796028565806211]
  We propose a semantic distribution-guided reconstruction framework that uses a pre-trained vision-language foundation model.<n>A contrastive objective aligns the reconstructed representation with the target semantic distribution.<n>Experiments on knee and brain datasets demonstrate that semantic priors from images preserve fine anatomical structures and achieve superior perceptual quality.
  arXiv  Detail & Related papers  (2025-11-24T19:15:47Z)
• Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstruction [65.67001243986981]
  We propose MindHier, a coarse-to-fine fMRI-to-image reconstruction framework built on scale-wise autoregressive modeling.<n>MindHier achieves superior semantic fidelity, 4.67x faster inference, and more deterministic results than the diffusion-based baselines.
  arXiv  Detail & Related papers  (2025-10-25T15:40:07Z)
• "Principal Components" Enable A New Language of Images [79.45806370905775]
  We introduce a novel visual tokenization framework that embeds a provable PCA-like structure into the latent token space.<n>Our approach achieves state-of-the-art reconstruction performance and enables better interpretability to align with the human vision system.
  arXiv  Detail & Related papers  (2025-03-11T17:59:41Z)
• Brain-Streams: fMRI-to-Image Reconstruction with Multi-modal Guidance [3.74142789780782]
  We show how modern LDMs incorporate multi-modal guidance for structurally and semantically plausible image generations. Brain-Streams maps fMRI signals from brain regions to appropriate embeddings. We validate the reconstruction ability of Brain-Streams both quantitatively and qualitatively on a real fMRI dataset.
  arXiv  Detail & Related papers  (2024-09-18T16:19:57Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• NeuralDiffuser: Neuroscience-inspired Diffusion Guidance for fMRI Visual Reconstruction [25.987801733791986]
  We propose NeuralDiffuser, which incorporates primary visual feature guidance to provide detailed cues in the form of gradients.<n>This extension of the bottom-up process for diffusion models achieves both semantic coherence and detail fidelity when reconstructing visual stimuli.
  arXiv  Detail & Related papers  (2024-02-21T13:46:25Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion [7.597218661195779]
  We propose a two-stage image reconstruction model called MindDiffuser. In Stage 1, the VQ-VAE latent representations and the CLIP text embeddings decoded from fMRI are put into Stable Diffusion. In Stage 2, we utilize the CLIP visual feature decoded from fMRI as supervisory information, and continually adjust the two feature vectors decoded in Stage 1 through backpagation to align the structural information.
  arXiv  Detail & Related papers  (2023-08-08T13:28:34Z)
• High-resolution Face Swapping via Latent Semantics Disentanglement [50.23624681222619]
  We present a novel high-resolution hallucination face swapping method using the inherent prior knowledge of a pre-trained GAN model. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator. We extend our method to video face swapping by enforcing two-temporal constraints on the latent space and the image space.
  arXiv  Detail & Related papers  (2022-03-30T00:33:08Z)
• Limited-angle tomographic reconstruction of dense layered objects by dynamical machine learning [68.9515120904028]
  Limited-angle tomography of strongly scattering quasi-transparent objects is a challenging, highly ill-posed problem. Regularizing priors are necessary to reduce artifacts by improving the condition of such problems. We devised a recurrent neural network (RNN) architecture with a novel split-convolutional gated recurrent unit (SC-GRU) as the building block.
  arXiv  Detail & Related papers  (2020-07-21T11:48:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.