Brain Structure-Function Fusing Representation Learning using Adversarial Decomposed-VAE for Analyzing MCI

• URL: http://arxiv.org/abs/2305.14404v1
• Date: Tue, 23 May 2023 11:19:02 GMT
• Title: Brain Structure-Function Fusing Representation Learning using Adversarial Decomposed-VAE for Analyzing MCI
• Authors: Qiankun Zuo, Baiying Lei, Ning Zhong, Yi Pan, Shuqiang Wang
• Abstract summary: A novel brain structure-function fusing-representation learning (BSFL) model is proposed to learn fused representation from fMRI imaging. The proposed model achieves better performance than other competitive methods in predicting and analyzing mild cognitive impairment (MCI) The model could be a potential tool for reconstructing unified brain networks and predicting abnormal connections during the degenerative processes in MCI.
• Score: 17.757114703434027
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Integrating the brain structural and functional connectivity features is of great significance in both exploring brain science and analyzing cognitive impairment clinically. However, it remains a challenge to effectively fuse structural and functional features in exploring the brain network. In this paper, a novel brain structure-function fusing-representation learning (BSFL) model is proposed to effectively learn fused representation from diffusion tensor imaging (DTI) and resting-state functional magnetic resonance imaging (fMRI) for mild cognitive impairment (MCI) analysis. Specifically, the decomposition-fusion framework is developed to first decompose the feature space into the union of the uniform and the unique spaces for each modality, and then adaptively fuse the decomposed features to learn MCI-related representation. Moreover, a knowledge-aware transformer module is designed to automatically capture local and global connectivity features throughout the brain. Also, a uniform-unique contrastive loss is further devised to make the decomposition more effective and enhance the complementarity of structural and functional features. The extensive experiments demonstrate that the proposed model achieves better performance than other competitive methods in predicting and analyzing MCI. More importantly, the proposed model could be a potential tool for reconstructing unified brain networks and predicting abnormal connections during the degenerative processes in MCI.

Related papers

• Integrated Brain Connectivity Analysis with fMRI, DTI, and sMRI Powered by Interpretable Graph Neural Networks [17.063133885403154]
  We integrate functional magnetic resonance imaging, diffusion tensor imaging, and structural MRI into a cohesive framework. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating a holistic amalgamation of multimodal imaging data. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth.
  arXiv  Detail & Related papers  (2024-08-26T13:16:42Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation [56.34634121544929]
  In this study, we first construct the brain-effective network via the dynamic causal model. We then introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE) This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks.
  arXiv  Detail & Related papers  (2024-05-21T20:37:07Z)
• Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activity [60.983327742457995]
  Reconstructing the viewed images from human brain activity bridges human and computer vision through the Brain-Computer Interface. We devise Psychometry, an omnifit model for reconstructing images from functional Magnetic Resonance Imaging (fMRI) obtained from different subjects.
  arXiv  Detail & Related papers  (2024-03-29T07:16:34Z)
• fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding [54.17776744076334]
  We propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training. Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving brain activity patterns. Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach.
  arXiv  Detail & Related papers  (2023-11-01T07:24:22Z)
• Alzheimer's Disease Prediction via Brain Structural-Functional Deep Fusing Network [5.945843237682432]
  Cross-modal transformer generative adversarial network (CT-GAN) is proposed to fuse functional and structural information. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively.
  arXiv  Detail & Related papers  (2023-09-28T07:06:42Z)
• Multi-task Collaborative Pre-training and Individual-adaptive-tokens Fine-tuning: A Unified Framework for Brain Representation Learning [3.1453938549636185]
  We propose a unified framework that combines Collaborative pre-training and Individual--Tokens fine-tuning. The proposed MCIAT achieves state-of-the-art diagnosis performance on the ADHD-200 dataset.
  arXiv  Detail & Related papers  (2023-06-20T08:38:17Z)
• Fusing Structural and Functional Connectivities using Disentangled VAE for Detecting MCI [9.916963496386089]
  A novel hierarchical structural-functional connectivity fusing (HSCF) model is proposed to construct brain structural-functional connectivity matrices. Results from a wide range of tests performed on the public Alzheimer's Disease Neuroimaging Initiative database show that the proposed model performs better than competing approaches.
  arXiv  Detail & Related papers  (2023-06-16T05:22:25Z)
• Cross-Modal Transformer GAN: A Brain Structure-Function Deep Fusing Framework for Alzheimer's Disease [5.608783790624866]
  Cross-modal fusion of different types of neuroimaging data has shown great promise for predicting the progression of Alzheimer's Disease(AD) In this work, a novel cross-modal transformer generative adversarial network(CT-GAN) is proposed to fuse functional information contained in resting-state functional magnetic resonance imaging (rs-fMRI) and structural information contained in Diffusion Imaging (DTI) The proposed model can not only improve classification performance but also detect the AD-related brain connectivity effectively.
  arXiv  Detail & Related papers  (2022-06-20T11:38:55Z)
• Functional2Structural: Cross-Modality Brain Networks Representation Learning [55.24969686433101]
  Graph mining on brain networks may facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. We propose a novel graph learning framework, known as Deep Signed Brain Networks (DSBN), with a signed graph encoder. We validate our framework on clinical phenotype and neurodegenerative disease prediction tasks using two independent, publicly available datasets.
  arXiv  Detail & Related papers  (2022-05-06T03:45:36Z)
• Influence Estimation and Maximization via Neural Mean-Field Dynamics [60.91291234832546]
  We propose a novel learning framework using neural mean-field (NMF) dynamics for inference and estimation problems. Our framework can simultaneously learn the structure of the diffusion network and the evolution of node infection probabilities.
  arXiv  Detail & Related papers  (2021-06-03T00:02:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.