Voxel-Level Brain States Prediction Using Swin Transformer

• URL: http://arxiv.org/abs/2506.11455v1
• Date: Fri, 13 Jun 2025 04:14:38 GMT
• Title: Voxel-Level Brain States Prediction Using Swin Transformer
• Authors: Yifei Sun, Daniel Chahine, Qinghao Wen, Tianming Liu, Xiang Li, Yixuan Yuan, Fernando Calamante, Jinglei Lv,
• Abstract summary: We propose a novel architecture which employs a 4D Shifted Window (Swin) Transformer as encoder to efficiently learn-temporal information and a convolutional decoder to enable brain state prediction at the same spatial and temporal resolution as the input fMRI data.<n>Our model has shown high accuracy when predicting 7.2s resting-state brain activities based on the prior 23.04s fMRI time series.<n>This shows promising evidence that thetemporal organization of the human brain can be learned by a Swin Transformer model, at high resolution, which provides a potential for reducing fMRI scan time and the development of brain-computer interfaces
• Score: 65.9194533414066
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding brain dynamics is important for neuroscience and mental health. Functional magnetic resonance imaging (fMRI) enables the measurement of neural activities through blood-oxygen-level-dependent (BOLD) signals, which represent brain states. In this study, we aim to predict future human resting brain states with fMRI. Due to the 3D voxel-wise spatial organization and temporal dependencies of the fMRI data, we propose a novel architecture which employs a 4D Shifted Window (Swin) Transformer as encoder to efficiently learn spatio-temporal information and a convolutional decoder to enable brain state prediction at the same spatial and temporal resolution as the input fMRI data. We used 100 unrelated subjects from the Human Connectome Project (HCP) for model training and testing. Our novel model has shown high accuracy when predicting 7.2s resting-state brain activities based on the prior 23.04s fMRI time series. The predicted brain states highly resemble BOLD contrast and dynamics. This work shows promising evidence that the spatiotemporal organization of the human brain can be learned by a Swin Transformer model, at high resolution, which provides a potential for reducing the fMRI scan time and the development of brain-computer interfaces in the future.

Related papers

• A Survey on fMRI-based Brain Decoding for Reconstructing Multimodal Stimuli [26.07986165893441]
  Decoding brain signals to reconstruct stimuli drives progress in AI, disease treatment, and brain-computer interfaces.<n>Recent advancements in neuroimaging and image generation models have significantly improved fMRI-based decoding.<n>This survey systematically reviews recent progress in fMRI-based brain decoding, focusing on stimulus reconstruction from passive brain signals.
  arXiv  Detail & Related papers  (2025-03-20T09:23:07Z)
• Predicting Human Brain States with Transformer [45.25907962341717]
  We show that a self-attention-based model can accurately predict the brain states up to 5.04s with the previous 21.6s.<n>These promising initial results demonstrate the possibility of developing gen-erative models for fMRI data.
  arXiv  Detail & Related papers  (2024-12-11T00:18:39Z)
• BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation [6.5388528484686885]
  This study introduces a novel approach towards the creation of medical foundation models. Our method involves a novel two-stage pretraining approach using vision transformers. BrainFounder demonstrates a significant performance gain, surpassing the achievements of previous winning solutions.
  arXiv  Detail & Related papers  (2024-06-14T19:49:45Z)
• Brain3D: Generating 3D Objects from fMRI [76.41771117405973]
  We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject. We show that our model captures the distinct functionalities of each region of human vision system. Preliminary evaluations indicate that Brain3D can successfully identify the disordered brain regions in simulated scenarios.
  arXiv  Detail & Related papers  (2024-05-24T06:06:11Z)
• BrainODE: Dynamic Brain Signal Analysis via Graph-Aided Neural Ordinary Differential Equations [67.79256149583108]
  We propose a novel model called BrainODE to achieve continuous modeling of dynamic brain signals. By learning latent initial values and neural ODE functions from irregular time series, BrainODE effectively reconstructs brain signals at any time point.
  arXiv  Detail & Related papers  (2024-04-30T10:53:30Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• Towards a Foundation Model for Brain Age Prediction using coVariance Neural Networks [102.75954614946258]
  Increasing brain age with respect to chronological age can reflect increased vulnerability to neurodegeneration and cognitive decline. NeuroVNN is pre-trained as a regression model on healthy population to predict chronological age. NeuroVNN adds anatomical interpretability to brain age and has a scale-free' characteristic that allows its transference to datasets curated according to any arbitrary brain atlas.
  arXiv  Detail & Related papers  (2024-02-12T14:46:31Z)
• SwiFT: Swin 4D fMRI Transformer [17.95502427633986]
  We present SwiFTS (win 4D fMRI Transformer), a Swin Transformer architecture that can learn brain dynamics directly from volumes fMRI. We evaluate SwiFT using multiple large-scale resting-state fMRI datasets to predict sex age and cognitive intelligence.
  arXiv  Detail & Related papers  (2023-07-12T04:53:36Z)
• Explainable fMRI-based Brain Decoding via Spatial Temporal-pyramid Graph Convolutional Network [0.8399688944263843]
  Existing machine learning methods for fMRI-based brain decoding either suffer from low classification performance or poor explainability. We propose a biologically inspired architecture, Spatial Temporal-pyramid Graph Convolutional Network (STpGCN), to capture the spatial-temporal graph representation of functional brain activities. We conduct extensive experiments on fMRI data under 23 cognitive tasks from Human Connectome Project (HCP) S1200.
  arXiv  Detail & Related papers  (2022-10-08T12:14:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.