Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

• URL: http://arxiv.org/abs/2004.05234v2
• Date: Tue, 19 Jan 2021 06:35:49 GMT
• Title: Attend and Decode: 4D fMRI Task State Decoding Using Attention Models
• Authors: Sam Nguyen, Brenda Ng, Alan D. Kaplan and Priyadip Ray
• Abstract summary: We present a novel architecture called Brain Attend and Decode (BAnD) BAnD uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the Human Connectome Project-Young Adult dataset.
• Score: 2.6954666679827137
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Functional magnetic resonance imaging (fMRI) is a neuroimaging modality that captures the blood oxygen level in a subject's brain while the subject either rests or performs a variety of functional tasks under different conditions. Given fMRI data, the problem of inferring the task, known as task state decoding, is challenging due to the high dimensionality (hundreds of million sampling points per datum) and complex spatio-temporal blood flow patterns inherent in the data. In this work, we propose to tackle the fMRI task state decoding problem by casting it as a 4D spatio-temporal classification problem. We present a novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project-Young Adult (HCP-YA) dataset. We also investigate the transferability of BAnD's extracted features on unseen HCP tasks, either by freezing the spatial feature extraction layers and retraining the temporal model, or finetuning the entire model. The pre-trained features from BAnD are useful on similar tasks while finetuning them yields competitive results on unseen tasks/conditions.

Related papers

• Unifying Subsampling Pattern Variations for Compressed Sensing MRI with Neural Operators [72.79532467687427]
  Compressed Sensing MRI reconstructs images of the body's internal anatomy from undersampled and compressed measurements. Deep neural networks have shown great potential for reconstructing high-quality images from highly undersampled measurements. We propose a unified model that is robust to different subsampling patterns and image resolutions in CS-MRI.
  arXiv  Detail & Related papers  (2024-10-05T20:03:57Z)
• STNAGNN: Spatiotemporal Node Attention Graph Neural Network for Task-based fMRI Analysis [9.35032090865023]
  We show that using task-driven data structures is effective for autism analysis. We propose a GNN-based task-based architecture and validate its performance in an autism task.
  arXiv  Detail & Related papers  (2024-06-17T20:08:05Z)
• Brain3D: Generating 3D Objects from fMRI [76.41771117405973]
  We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject. We show that our model captures the distinct functionalities of each region of human vision system. Preliminary evaluations indicate that Brain3D can successfully identify the disordered brain regions in simulated scenarios.
  arXiv  Detail & Related papers  (2024-05-24T06:06:11Z)
• Learning Sequential Information in Task-based fMRI for Synthetic Data Augmentation [10.629487323161323]
  We propose an approach for generating synthetic fMRI sequences that can be used to create augmented training datasets in downstream learning. The synthetic images are evaluated from multiple perspectives including visualizations and an autism spectrum disorder (ASD) classification task.
  arXiv  Detail & Related papers  (2023-08-29T18:36:21Z)
• TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling [54.97005925277638]
  The identification of sensory cues associated with potential opportunities and dangers is frequently complicated by unrelated events that separate useful cues by long delays. It remains a challenging task for state-of-the-art spiking neural networks (SNNs) to establish long-term temporal dependency between distant cues. We propose a novel biologically inspired Two-Compartment Leaky Integrate-and-Fire spiking neuron model, dubbed TC-LIF.
  arXiv  Detail & Related papers  (2023-08-25T08:54:41Z)
• Long Short-term Memory with Two-Compartment Spiking Neuron [64.02161577259426]
  We propose a novel biologically inspired Long Short-Term Memory Leaky Integrate-and-Fire spiking neuron model, dubbed LSTM-LIF. Our experimental results, on a diverse range of temporal classification tasks, demonstrate superior temporal classification capability, rapid training convergence, strong network generalizability, and high energy efficiency of the proposed LSTM-LIF model. This work, therefore, opens up a myriad of opportunities for resolving challenging temporal processing tasks on emerging neuromorphic computing machines.
  arXiv  Detail & Related papers  (2023-07-14T08:51:03Z)
• SwiFT: Swin 4D fMRI Transformer [17.95502427633986]
  We present SwiFTS (win 4D fMRI Transformer), a Swin Transformer architecture that can learn brain dynamics directly from volumes fMRI. We evaluate SwiFT using multiple large-scale resting-state fMRI datasets to predict sex age and cognitive intelligence.
  arXiv  Detail & Related papers  (2023-07-12T04:53:36Z)
• Cross-Modality Neuroimage Synthesis: A Survey [71.27193056354741]
  Multi-modality imaging improves disease diagnosis and reveals distinct deviations in tissues with anatomical properties. The existence of completely aligned and paired multi-modality neuroimaging data has proved its effectiveness in brain research. An alternative solution is to explore unsupervised or weakly supervised learning methods to synthesize the absent neuroimaging data.
  arXiv  Detail & Related papers  (2022-02-14T19:29:08Z)
• Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay [67.50637511633212]
  A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day.
  arXiv  Detail & Related papers  (2021-12-09T07:11:14Z)
• Understanding Cognitive Fatigue from fMRI Scans with Self-supervised Learning [0.0]
  This paper proposes dividing state of cognitive fatigue into six different levels, ranging from no-fatigue to extreme fatigue conditions. We built a-temporal model that uses convolutional neural networks (CNN) for spatial feature extraction and a long short-term memory (LSTM) network for temporal modeling of 4D fMRI scans. This method establishes a state-of-the-art technique to analyze cognitive fatigue from fMRI data and beats previous approaches to solve this problem.
  arXiv  Detail & Related papers  (2021-06-28T22:38:51Z)
• Identification of brain states, transitions, and communities using functional MRI [0.5872014229110214]
  We propose a Bayesian model-based characterization of latent brain states and showcase a novel method based on posterior predictive discrepancy. Our results obtained through an analysis of task-fMRI data show appropriate lags between external task demands and change-points between brain states.
  arXiv  Detail & Related papers  (2021-01-26T08:10:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.