Explainable fMRI-based Brain Decoding via Spatial Temporal-pyramid Graph Convolutional Network

• URL: http://arxiv.org/abs/2210.05713v1
• Date: Sat, 8 Oct 2022 12:14:33 GMT
• Title: Explainable fMRI-based Brain Decoding via Spatial Temporal-pyramid Graph Convolutional Network
• Authors: Ziyuan Ye, Youzhi Qu, Zhichao Liang, Mo Wang, Quanying Liu
• Abstract summary: Existing machine learning methods for fMRI-based brain decoding either suffer from low classification performance or poor explainability. We propose a biologically inspired architecture, Spatial Temporal-pyramid Graph Convolutional Network (STpGCN), to capture the spatial-temporal graph representation of functional brain activities. We conduct extensive experiments on fMRI data under 23 cognitive tasks from Human Connectome Project (HCP) S1200.
• Score: 0.8399688944263843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Brain decoding, aiming to identify the brain states using neural activity, is important for cognitive neuroscience and neural engineering. However, existing machine learning methods for fMRI-based brain decoding either suffer from low classification performance or poor explainability. Here, we address this issue by proposing a biologically inspired architecture, Spatial Temporal-pyramid Graph Convolutional Network (STpGCN), to capture the spatial-temporal graph representation of functional brain activities. By designing multi-scale spatial-temporal pathways and bottom-up pathways that mimic the information process and temporal integration in the brain, STpGCN is capable of explicitly utilizing the multi-scale temporal dependency of brain activities via graph, thereby achieving high brain decoding performance. Additionally, we propose a sensitivity analysis method called BrainNetX to better explain the decoding results by automatically annotating task-related brain regions from the brain-network standpoint. We conduct extensive experiments on fMRI data under 23 cognitive tasks from Human Connectome Project (HCP) S1200. The results show that STpGCN significantly improves brain decoding performance compared to competing baseline models; BrainNetX successfully annotates task-relevant brain regions. Post hoc analysis based on these regions further validates that the hierarchical structure in STpGCN significantly contributes to the explainability, robustness and generalization of the model. Our methods not only provide insights into information representation in the brain under multiple cognitive tasks but also indicate a bright future for fMRI-based brain decoding.

Related papers

• Brain-like Functional Organization within Large Language Models [58.93629121400745]
  The human brain has long inspired the pursuit of artificial intelligence (AI) Recent neuroimaging studies provide compelling evidence of alignment between the computational representation of artificial neural networks (ANNs) and the neural responses of the human brain to stimuli. In this study, we bridge this gap by directly coupling sub-groups of artificial neurons with functional brain networks (FBNs) This framework links the AN sub-groups to FBNs, enabling the delineation of brain-like functional organization within large language models (LLMs)
  arXiv  Detail & Related papers  (2024-10-25T13:15:17Z)
• Latent Representation Learning for Multimodal Brain Activity Translation [14.511112110420271]
  We present the Spatiotemporal Alignment of Multimodal Brain Activity (SAMBA) framework, which bridges the spatial and temporal resolution gaps across modalities. SAMBA introduces a novel attention-based wavelet decomposition for spectral filtering of electrophysiological recordings. We show that the training of SAMBA, aside from achieving translation, also learns a rich representation of brain information processing.
  arXiv  Detail & Related papers  (2024-09-27T05:50:29Z)
• BrainMAE: A Region-aware Self-supervised Learning Framework for Brain Signals [11.030708270737964]
  We propose Brain Masked Auto-Encoder (BrainMAE) for learning representations directly from fMRI time-series data. BrainMAE consistently outperforms established baseline methods by significant margins in four distinct downstream tasks.
  arXiv  Detail & Related papers  (2024-06-24T19:16:24Z)
• Graph Neural Networks for Brain Graph Learning: A Survey [53.74244221027981]
  Graph neural networks (GNNs) have demonstrated a significant advantage in mining graph-structured data. GNNs to learn brain graph representations for brain disorder analysis has recently gained increasing attention. In this paper, we aim to bridge this gap by reviewing brain graph learning works that utilize GNNs.
  arXiv  Detail & Related papers  (2024-06-01T02:47:39Z)
• BrainODE: Dynamic Brain Signal Analysis via Graph-Aided Neural Ordinary Differential Equations [67.79256149583108]
  We propose a novel model called BrainODE to achieve continuous modeling of dynamic brain signals. By learning latent initial values and neural ODE functions from irregular time series, BrainODE effectively reconstructs brain signals at any time point.
  arXiv  Detail & Related papers  (2024-04-30T10:53:30Z)
• Brain Networks and Intelligence: A Graph Neural Network Based Approach to Resting State fMRI Data [2.193937336601403]
  We present a novel modeling architecture called BrainRGIN for predicting intelligence (fluid, crystallized, and total intelligence) using graph neural networks on rsfMRI derived connectivity matrices. Our approach incorporates a clustering-based embedding and graph isomorphism network in the graph convolutional layer to reflect the nature of the brain sub-network organization.
  arXiv  Detail & Related papers  (2023-11-06T20:58:07Z)
• Transformer-Based Hierarchical Clustering for Brain Network Analysis [13.239896897835191]
  We propose a novel interpretable transformer-based model for joint hierarchical cluster identification and brain network classification. With the help of hierarchical clustering, the model achieves increased accuracy and reduced runtime complexity while providing plausible insight into the functional organization of brain regions.
  arXiv  Detail & Related papers  (2023-05-06T22:14:13Z)
• Functional2Structural: Cross-Modality Brain Networks Representation Learning [55.24969686433101]
  Graph mining on brain networks may facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. We propose a novel graph learning framework, known as Deep Signed Brain Networks (DSBN), with a signed graph encoder. We validate our framework on clinical phenotype and neurodegenerative disease prediction tasks using two independent, publicly available datasets.
  arXiv  Detail & Related papers  (2022-05-06T03:45:36Z)
• Deep Reinforcement Learning Guided Graph Neural Networks for Brain Network Analysis [61.53545734991802]
  We propose a novel brain network representation framework, namely BN-GNN, which searches for the optimal GNN architecture for each brain network. Our proposed BN-GNN improves the performance of traditional GNNs on different brain network analysis tasks.
  arXiv  Detail & Related papers  (2022-03-18T07:05:27Z)
• Recurrent Brain Graph Mapper for Predicting Time-Dependent Brain Graph Evaluation Trajectory [0.0]
  Brain disorders can be detected by observing alterations in the brain's structural and functional connectivities. Recent studies aimed to predict the evolution of brain connectivities over time by proposing machine learning models. Here, we propose to use brain connectivities as a more efficient alternative for time-dependent brain disorder diagnosis.
  arXiv  Detail & Related papers  (2021-10-06T09:25:55Z)
• Towards a Neural Model for Serial Order in Frontal Cortex: a Brain Theory from Memory Development to Higher-Level Cognition [53.816853325427424]
  We propose that the immature prefrontal cortex (PFC) use its primary functionality of detecting hierarchical patterns in temporal signals. Our hypothesis is that the PFC detects the hierarchical structure in temporal sequences in the form of ordinal patterns and use them to index information hierarchically in different parts of the brain. By doing so, it gives the tools to the language-ready brain for manipulating abstract knowledge and planning temporally ordered information.
  arXiv  Detail & Related papers  (2020-05-22T14:29:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.