Learning 3D Medical Image Models From Brain Functional Connectivity Network Supervision For Mental Disorder Diagnosis

• URL: http://arxiv.org/abs/2503.04205v1
• Date: Thu, 06 Mar 2025 08:30:33 GMT
• Title: Learning 3D Medical Image Models From Brain Functional Connectivity Network Supervision For Mental Disorder Diagnosis
• Authors: Xingcan Hu, Wei Wang, Li Xiao,
• Abstract summary: In MRI-based mental disorder diagnosis, most previous studies focus on functional connectivity network (FCN) derived from functional MRI (fMRI)<n>We propose CINP (Contrastive Image-Network Pre-training), a framework that employs contrastive learning between sMRI and FCN.
• Score: 5.198001918667427
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In MRI-based mental disorder diagnosis, most previous studies focus on functional connectivity network (FCN) derived from functional MRI (fMRI). However, the small size of annotated fMRI datasets restricts its wide application. Meanwhile, structural MRIs (sMRIs), such as 3D T1-weighted (T1w) MRI, which are commonly used and readily accessible in clinical settings, are often overlooked. To integrate the complementary information from both function and structure for improved diagnostic accuracy, we propose CINP (Contrastive Image-Network Pre-training), a framework that employs contrastive learning between sMRI and FCN. During pre-training, we incorporate masked image modeling and network-image matching to enhance visual representation learning and modality alignment. Since the CINP facilitates knowledge transfer from FCN to sMRI, we introduce network prompting. It utilizes only sMRI from suspected patients and a small amount of FCNs from different patient classes for diagnosing mental disorders, which is practical in real-world clinical scenario. The competitive performance on three mental disorder diagnosis tasks demonstrate the effectiveness of the CINP in integrating multimodal MRI information, as well as the potential of incorporating sMRI into clinical diagnosis using network prompting.

Related papers

• 4D Multimodal Co-attention Fusion Network with Latent Contrastive Alignment for Alzheimer's Diagnosis [24.771496672135395]
  We propose M2M-AlignNet: a geometry-aware co-attention network with latent alignment for early Alzheimer's diagnosis. At the core of our approach is a multi-patch-to-multi-patch (M2M) contrastive loss function that quantifies and reduces representational discrepancies. We conduct extensive experiments to confirm the effectiveness of our method and highlight the correspondance between fMRI and sMRI as AD biomarkers.
  arXiv  Detail & Related papers  (2025-04-23T15:18:55Z)
• Copula-Linked Parallel ICA: A Method for Coupling Structural and Functional MRI brain Networks [0.5277756703318045]
  Prior studies fusing functional MRI (fMRI) and structural MRI (sMRI) have shown the benefits of this approach. We developed a novel fusion method, by combining deep learning frameworks, copulas and independent component analysis (ICA), named copula linked parallel ICA (CLiP-ICA) CLiP-ICA effectively captures both strongly and weakly linked sMRI and fMRI networks, including the cerebellum, sensorimotor, visual, cognitive control, and default mode networks.
  arXiv  Detail & Related papers  (2024-10-14T01:35:41Z)
• Towards General Text-guided Image Synthesis for Customized Multimodal Brain MRI Generation [51.28453192441364]
  Multimodal brain magnetic resonance (MR) imaging is indispensable in neuroscience and neurology. Current MR image synthesis approaches are typically trained on independent datasets for specific tasks. We present TUMSyn, a Text-guided Universal MR image Synthesis model, which can flexibly generate brain MR images.
  arXiv  Detail & Related papers  (2024-09-25T11:14:47Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding [54.17776744076334]
  We propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training. Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving brain activity patterns. Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach.
  arXiv  Detail & Related papers  (2023-11-01T07:24:22Z)
• Multi-Dimension-Embedding-Aware Modality Fusion Transformer for Psychiatric Disorder Clasification [13.529183496842819]
  We construct a deep learning architecture that takes as input 2D time series of rs-fMRI and 3D volumes T1w. We show that our proposed MFFormer performs better than that using a single modality or multi-modality MRI on schizophrenia and bipolar disorder diagnosis.
  arXiv  Detail & Related papers  (2023-10-04T10:02:04Z)
• Leveraging Brain Modularity Prior for Interpretable Representation Learning of fMRI [38.236414924531196]
  Resting-state functional magnetic resonance imaging (rs-fMRI) can reflect spontaneous neural activities in brain. Previous studies propose to extract fMRI representations through diverse machine/deep learning methods for subsequent analysis. This paper proposes a Brain Modularity-constrained dynamic Representation learning (BMR) framework for interpretable fMRI analysis.
  arXiv  Detail & Related papers  (2023-06-24T23:45:47Z)
• Video4MRI: An Empirical Study on Brain Magnetic Resonance Image Analytics with CNN-based Video Classification Frameworks [60.42012344842292]
  3D CNN-based models dominate the field of magnetic resonance image (MRI) analytics. In this paper, four datasets of Alzheimer's and Parkinson's disease recognition are utilized in experiments. In terms of efficiency, the video framework performs better than 3D-CNN models by 5% - 11% with 50% - 66% less trainable parameters.
  arXiv  Detail & Related papers  (2023-02-24T15:26:31Z)
• Deep Labeling of fMRI Brain Networks Using Cloud Based Processing [0.0]
  Resting state fMRI is used in neurosurgical pre-planning to visualize functional regions and assess regional activity. We propose an end-to-end reproducible pipeline which incorporates image processing of rs-fMRI in a cloud-based workflow. We use deep learning to automate the classification of Resting State Networks (RSNs)
  arXiv  Detail & Related papers  (2022-09-16T23:59:42Z)
• Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction [68.80715727288514]
  We show how to unfold an iterative MGDUN algorithm into a novel model-guided deep unfolding network by taking the MRI observation matrix. In this paper, we propose a novel Model-Guided interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction.
  arXiv  Detail & Related papers  (2022-09-15T03:58:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.