Brain Foundation Models with Hypergraph Dynamic Adapter for Brain Disease Analysis

• URL: http://arxiv.org/abs/2505.00627v1
• Date: Thu, 01 May 2025 16:06:17 GMT
• Title: Brain Foundation Models with Hypergraph Dynamic Adapter for Brain Disease Analysis
• Authors: Zhongying Deng, Haoyu Wang, Ziyan Huang, Lipei Zhang, Angelica I. Aviles-Rivero, Chaoyu Liu, Junjun He, Zoe Kourtzi, Carola-Bibiane Schönlieb,
• Abstract summary: Brain diseases, such as Alzheimer's disease and brain tumors, present profound challenges due to their complexity and societal impact.<n>Recent advancements in brain foundation models have shown significant promise in addressing a range of brain-related tasks.<n>We propose SAM-Brain3D, a brain-specific foundation model trained on over 66,000 brain image-label pairs.
• Score: 18.02038938366483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Brain diseases, such as Alzheimer's disease and brain tumors, present profound challenges due to their complexity and societal impact. Recent advancements in brain foundation models have shown significant promise in addressing a range of brain-related tasks. However, current brain foundation models are limited by task and data homogeneity, restricted generalization beyond segmentation or classification, and inefficient adaptation to diverse clinical tasks. In this work, we propose SAM-Brain3D, a brain-specific foundation model trained on over 66,000 brain image-label pairs across 14 MRI sub-modalities, and Hypergraph Dynamic Adapter (HyDA), a lightweight adapter for efficient and effective downstream adaptation. SAM-Brain3D captures detailed brain-specific anatomical and modality priors for segmenting diverse brain targets and broader downstream tasks. HyDA leverages hypergraphs to fuse complementary multi-modal data and dynamically generate patient-specific convolutional kernels for multi-scale feature fusion and personalized patient-wise adaptation. Together, our framework excels across a broad spectrum of brain disease segmentation and classification tasks. Extensive experiments demonstrate that our method consistently outperforms existing state-of-the-art approaches, offering a new paradigm for brain disease analysis through multi-modal, multi-scale, and dynamic foundation modeling.

Related papers

• BrainNet-MoE: Brain-Inspired Mixture-of-Experts Learning for Neurological Disease Identification [31.45078414913088]
  The Lewy body dementia (LBD) is the second most common neurodegenerative dementia after Alzheimer's disease (AD) Our work represents a pioneering effort in modeling system-level artificial neural network called BrainNet-MoE for brain modeling and diagnosing.
  arXiv  Detail & Related papers  (2025-03-05T22:19:49Z)
• A Differentiable Approach to Multi-scale Brain Modeling [3.5874544981360987]
  We present a multi-scale differentiable brain modeling workflow utilizing BrainPy, a unique differentiable brain simulator. At the single-neuron level, we implement differentiable neuron models and employ gradient methods to optimize their fit to electrophysiological data. On the network level, we incorporate connectomic data to construct biologically constrained network models.
  arXiv  Detail & Related papers  (2024-06-28T07:41:31Z)
• BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation [6.5388528484686885]
  This study introduces a novel approach towards the creation of medical foundation models. Our method involves a novel two-stage pretraining approach using vision transformers. BrainFounder demonstrates a significant performance gain, surpassing the achievements of previous winning solutions.
  arXiv  Detail & Related papers  (2024-06-14T19:49:45Z)
• A Foundation Model for Brain Lesion Segmentation with Mixture of Modality Experts [3.208907282505264]
  We propose a universal foundation model for 3D brain lesion segmentation. We formulate a novel Mixture of Modality Experts (MoME) framework with multiple expert networks attending to different imaging modalities. Our model outperforms state-of-the-art universal models and provides promising generalization to unseen datasets.
  arXiv  Detail & Related papers  (2024-05-16T16:49:20Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• Modality-Aware and Shift Mixer for Multi-modal Brain Tumor Segmentation [12.094890186803958]
  We present a novel Modality Aware and Shift Mixer that integrates intra-modality and inter-modality dependencies of multi-modal images for effective and robust brain tumor segmentation. Specifically, we introduce a Modality-Aware module according to neuroimaging studies for modeling the specific modality pair relationships at low levels, and a Modality-Shift module with specific mosaic patterns is developed to explore the complex relationships across modalities at high levels via the self-attention.
  arXiv  Detail & Related papers  (2024-03-04T14:21:51Z)
• UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training [66.16134293168535]
  We propose a hierarchical knowledge-enhanced pre-training framework for the universal brain MRI diagnosis, termed as UniBrain. Specifically, UniBrain leverages a large-scale dataset of 24,770 imaging-report pairs from routine diagnostics.
  arXiv  Detail & Related papers  (2023-09-13T09:22:49Z)
• Incomplete Multimodal Learning for Complex Brain Disorders Prediction [65.95783479249745]
  We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative cohort.
  arXiv  Detail & Related papers  (2023-05-25T16:29:16Z)
• Patched Diffusion Models for Unsupervised Anomaly Detection in Brain MRI [55.78588835407174]
  We propose a method that reformulates the generation task of diffusion models as a patch-based estimation of healthy brain anatomy. We evaluate our approach on data of tumors and multiple sclerosis lesions and demonstrate a relative improvement of 25.1% compared to existing baselines.
  arXiv  Detail & Related papers  (2023-03-07T09:40:22Z)
• Multimodal foundation models are better simulators of the human brain [65.10501322822881]
  We present a newly-designed multimodal foundation model pre-trained on 15 million image-text pairs. We find that both visual and lingual encoders trained multimodally are more brain-like compared with unimodal ones.
  arXiv  Detail & Related papers  (2022-08-17T12:36:26Z)
• Cross-Modality Deep Feature Learning for Brain Tumor Segmentation [158.8192041981564]
  This paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance.
  arXiv  Detail & Related papers  (2022-01-07T07:46:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.