Towards Zero-Shot Task-Generalizable Learning on fMRI

• URL: http://arxiv.org/abs/2502.10662v1
• Date: Sat, 15 Feb 2025 03:59:49 GMT
• Title: Towards Zero-Shot Task-Generalizable Learning on fMRI
• Authors: Jiyao Wang, Nicha C. Dvornek, Peiyu Duan, Lawrence H. Staib, James S. Duncan,
• Abstract summary: We propose a supervised task-aware network TA-GAT to aggregate task-based fMRI acquired in different tasks to train a generalizable model. The proposed architecture can plug-and-play in any neural network architecture to incorporate the prior knowledge of fMRI tasks into capturing functional brain patterns.
• Score: 9.90774796922676
• License:
• Abstract: Functional MRI measuring BOLD signal is an increasingly important imaging modality in studying brain functions and neurological disorders. It can be acquired in either a resting-state or a task-based paradigm. Compared to resting-state fMRI, task-based fMRI is acquired while the subject is performing a specific task designed to enhance study-related brain activities. Consequently, it generally has more informative task-dependent signals. However, due to the variety of task designs, it is much more difficult than in resting state to aggregate task-based fMRI acquired in different tasks to train a generalizable model. To resolve this complication, we propose a supervised task-aware network TA-GAT that jointly learns a general-purpose encoder and task-specific contextual information. The encoder-generated embedding and the learned contextual information are then combined as input to multiple modules for performing downstream tasks. We believe that the proposed task-aware architecture can plug-and-play in any neural network architecture to incorporate the prior knowledge of fMRI tasks into capturing functional brain patterns.

Related papers

• BrainMAP: Learning Multiple Activation Pathways in Brain Networks [77.15180533984947]
  We introduce a novel framework BrainMAP to learn Multiple Activation Pathways in Brain networks. Our framework enables explanatory analyses of crucial brain regions involved in tasks.
  arXiv  Detail & Related papers  (2024-12-23T09:13:35Z)
• STNAGNN: Spatiotemporal Node Attention Graph Neural Network for Task-based fMRI Analysis [9.35032090865023]
  We show that using task-driven data structures is effective for autism analysis. We propose a GNN-based task-based architecture and validate its performance in an autism task.
  arXiv  Detail & Related papers  (2024-06-17T20:08:05Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• Uncovering cognitive taskonomy through transfer learning in masked autoencoder-based fMRI reconstruction [6.3348067441225915]
  We employ the masked autoencoder (MAE) model to reconstruct functional magnetic resonance imaging (fMRI) data. Our study suggests that the fMRI reconstruction with MAE model can uncover the latent representation.
  arXiv  Detail & Related papers  (2024-05-24T09:29:16Z)
• fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding [54.17776744076334]
  We propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training. Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving brain activity patterns. Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach.
  arXiv  Detail & Related papers  (2023-11-01T07:24:22Z)
• Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretraining [121.89793208683625]
  Medical artificial general intelligence (MAGI) enables one foundation model to solve different medical tasks. We propose a new paradigm called Medical-knedge-enhanced mulTimOdal pretRaining (MOTOR)
  arXiv  Detail & Related papers  (2023-04-26T01:26:19Z)
• BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding [51.911473457195555]
  BrainCLIP is a task-agnostic fMRI-based brain decoding model. It bridges the modality gap between brain activity, image, and text. BrainCLIP can reconstruct visual stimuli with high semantic fidelity.
  arXiv  Detail & Related papers  (2023-02-25T03:28:54Z)
• Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks [107.8565143456161]
  We investigate the information processing strategies adopted by simple artificial neural networks performing a variety of cognitive tasks. Results show that synergy increases as neural networks learn multiple diverse tasks. randomly turning off neurons during training through dropout increases network redundancy, corresponding to an increase in robustness.
  arXiv  Detail & Related papers  (2022-10-06T15:36:27Z)
• Learning Multitask Gaussian Bayesian Networks [11.745963019193955]
  Major depressive disorder (MDD) requires study of brain functional connectivity alterations for patients. The amount of data collected during an fMRI scan is too limited to provide sufficient information for individual analysis. We propose a multitask Gaussian Bayesian network framework capable for identifying individual disease-induced alterations for MDD patients.
  arXiv  Detail & Related papers  (2022-05-11T08:38:00Z)
• Attend and Decode: 4D fMRI Task State Decoding Using Attention Models [2.6954666679827137]
  We present a novel architecture called Brain Attend and Decode (BAnD) BAnD uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the Human Connectome Project-Young Adult dataset.
  arXiv  Detail & Related papers  (2020-04-10T21:29:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.