UniBrain: A Unified Model for Cross-Subject Brain Decoding

• URL: http://arxiv.org/abs/2412.19487v1
• Date: Fri, 27 Dec 2024 07:03:47 GMT
• Title: UniBrain: A Unified Model for Cross-Subject Brain Decoding
• Authors: Zicheng Wang, Zhen Zhao, Luping Zhou, Parashkev Nachev,
• Abstract summary: We present UniBrain, a unified brain decoding model that requires no subject-specific parameters.<n>Our approach includes a group-based extractor to handle variable fMRI signal lengths, a mutual assistance embedder to capture cross-subject commonalities, and a bilevel feature alignment scheme for extracting subject-invariant features.<n>We validate our UniBrain on the brain decoding benchmark, achieving comparable performance to current state-of-the-art subject-specific models with extremely fewer parameters.
• Score: 22.49964298783508
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Brain decoding aims to reconstruct original stimuli from fMRI signals, providing insights into interpreting mental content. Current approaches rely heavily on subject-specific models due to the complex brain processing mechanisms and the variations in fMRI signals across individuals. Therefore, these methods greatly limit the generalization of models and fail to capture cross-subject commonalities. To address this, we present UniBrain, a unified brain decoding model that requires no subject-specific parameters. Our approach includes a group-based extractor to handle variable fMRI signal lengths, a mutual assistance embedder to capture cross-subject commonalities, and a bilevel feature alignment scheme for extracting subject-invariant features. We validate our UniBrain on the brain decoding benchmark, achieving comparable performance to current state-of-the-art subject-specific models with extremely fewer parameters. We also propose a generalization benchmark to encourage the community to emphasize cross-subject commonalities for more general brain decoding. Our code is available at https://github.com/xiaoyao3302/UniBrain.

Related papers

• MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-Text Decoding [15.264864090398374]
  Decoding functional magnetic resonance imaging (fMRI) signals into text has been a key challenge in neuroscience. We propose MindLLM, a model designed for subject-agnostic and versatile fMRI-to-text decoding. We introduce Brain Instruction Tuning (BIT), a novel approach that enhances the model's ability to capture diverse semantic representations from fMRI signals.
  arXiv  Detail & Related papers  (2025-02-18T03:27:37Z)
• MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data [64.92867794764247]
  MindAligner is a framework for cross-subject brain decoding from limited fMRI data. Brain Transfer Matrix (BTM) projects the brain signals of an arbitrary new subject to one of the known subjects. Brain Functional Alignment module is proposed to perform soft cross-subject brain alignment under different visual stimuli.
  arXiv  Detail & Related papers  (2025-02-07T16:01:59Z)
• CodeBrain: Imputing Any Brain MRI via Modality- and Instance-Specific Codes [39.308423499912806]
  We propose CodeBrain, a pipeline for unified brain MRI imputation. In the first stage, CodeBrain reconstructs a target modality by learning a compact scalar-quantized code for each instance and modality. In the second stage, a projection encoder is trained to predict full-modality compact codes from any incomplete MRI samples.
  arXiv  Detail & Related papers  (2025-01-30T13:14:40Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signals [5.283718601431859]
  Invasive brain-computer interfaces with Electrocorticography (ECoG) have shown promise for high-performance speech decoding in medical applications. We developed the Du-IN model, which extracts contextual embeddings based on region-level tokens through discrete codex-guided mask modeling. Our model achieves state-of-the-art performance on the 61-word classification task, surpassing all baselines.
  arXiv  Detail & Related papers  (2024-05-19T06:00:36Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• UMBRAE: Unified Multimodal Brain Decoding [43.6339793925953]
  We propose UMBRAE, a unified multimodal decoding of brain signals. We introduce an efficient universal brain encoder for multimodal-brain alignment. We also introduce a cross-subject training strategy mapping subject-specific features to a common feature space.
  arXiv  Detail & Related papers  (2024-04-10T17:59:20Z)
• UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training [66.16134293168535]
  We propose a hierarchical knowledge-enhanced pre-training framework for the universal brain MRI diagnosis, termed as UniBrain. Specifically, UniBrain leverages a large-scale dataset of 24,770 imaging-report pairs from routine diagnostics.
  arXiv  Detail & Related papers  (2023-09-13T09:22:49Z)
• Patched Diffusion Models for Unsupervised Anomaly Detection in Brain MRI [55.78588835407174]
  We propose a method that reformulates the generation task of diffusion models as a patch-based estimation of healthy brain anatomy. We evaluate our approach on data of tumors and multiple sclerosis lesions and demonstrate a relative improvement of 25.1% compared to existing baselines.
  arXiv  Detail & Related papers  (2023-03-07T09:40:22Z)
• BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding [51.911473457195555]
  BrainCLIP is a task-agnostic fMRI-based brain decoding model. It bridges the modality gap between brain activity, image, and text. BrainCLIP can reconstruct visual stimuli with high semantic fidelity.
  arXiv  Detail & Related papers  (2023-02-25T03:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.