UniBrain: Universal Brain MRI Diagnosis with Hierarchical
Knowledge-enhanced Pre-training
- URL: http://arxiv.org/abs/2309.06828v1
- Date: Wed, 13 Sep 2023 09:22:49 GMT
- Title: UniBrain: Universal Brain MRI Diagnosis with Hierarchical
Knowledge-enhanced Pre-training
- Authors: Jiayu Lei, Lisong Dai, Haoyun Jiang, Chaoyi Wu, Xiaoman Zhang, Yao
Zhang, Jiangchao Yao, Weidi Xie, Yanyong Zhang, Yuehua Li, Ya Zhang, Yanfeng
Wang
- Abstract summary: We propose a hierarchical knowledge-enhanced pre-training framework for the universal brain MRI diagnosis, termed as UniBrain.
Specifically, UniBrain leverages a large-scale dataset of 24,770 imaging-report pairs from routine diagnostics.
- Score: 66.16134293168535
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Magnetic resonance imaging~(MRI) have played a crucial role in brain disease
diagnosis, with which a range of computer-aided artificial intelligence methods
have been proposed. However, the early explorations usually focus on the
limited types of brain diseases in one study and train the model on the data in
a small scale, yielding the bottleneck of generalization. Towards a more
effective and scalable paradigm, we propose a hierarchical knowledge-enhanced
pre-training framework for the universal brain MRI diagnosis, termed as
UniBrain. Specifically, UniBrain leverages a large-scale dataset of 24,770
imaging-report pairs from routine diagnostics. Different from previous
pre-training techniques for the unitary vision or textual feature, or with the
brute-force alignment between vision and language information, we leverage the
unique characteristic of report information in different granularity to build a
hierarchical alignment mechanism, which strengthens the efficiency in feature
learning. Our UniBrain is validated on three real world datasets with severe
class imbalance and the public BraTS2019 dataset. It not only consistently
outperforms all state-of-the-art diagnostic methods by a large margin and
provides a superior grounding performance but also shows comparable performance
compared to expert radiologists on certain disease types.
Related papers
- Knowledge-Guided Prompt Learning for Lifespan Brain MR Image Segmentation [53.70131202548981]
We present a two-step segmentation framework employing Knowledge-Guided Prompt Learning (KGPL) for brain MRI.
Specifically, we first pre-train segmentation models on large-scale datasets with sub-optimal labels.
The introduction of knowledge-wise prompts captures semantic relationships between anatomical variability and biological processes.
arXiv Detail & Related papers (2024-07-31T04:32:43Z) - Enhanced Deep Learning Methodologies and MRI Selection Techniques for Dementia Diagnosis in the Elderly Population [5.103059984821972]
We introduce a novel methodology for the classification of demented and non-demented elderly patients using 3D brain Magnetic Resonance Imaging (MRI) scans.
Our approach features a unique technique for selectively processing MRI slices, focusing on the most relevant brain regions and excluding less informative sections.
This methodology is complemented by a confidence-based classification committee composed of three custom deep learning models.
arXiv Detail & Related papers (2024-07-24T14:48:40Z) - BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation [6.5388528484686885]
This study introduces a novel approach towards the creation of medical foundation models.
Our method involves a novel two-stage pretraining approach using vision transformers.
BrainFounder demonstrates a significant performance gain, surpassing the achievements of previous winning solutions.
arXiv Detail & Related papers (2024-06-14T19:49:45Z) - Aligning brain functions boosts the decoding of visual semantics in
novel subjects [3.226564454654026]
We propose to boost brain decoding by aligning brain responses to videos and static images across subjects.
Our method improves out-of-subject decoding performance by up to 75%.
It also outperforms classical single-subject approaches when fewer than 100 minutes of data is available for the tested subject.
arXiv Detail & Related papers (2023-12-11T15:55:20Z) - fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for
Multi-Subject Brain Activity Decoding [54.17776744076334]
We propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training.
Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving brain activity patterns.
Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach.
arXiv Detail & Related papers (2023-11-01T07:24:22Z) - Connectional-Style-Guided Contextual Representation Learning for Brain
Disease Diagnosis [12.172262618438173]
We propose a connectional style contextual representation learning model (CS-CRL) to capture the intrinsic pattern of the brain.
CS-CRL achieves superior accuracy in multiple brain disease diagnosis tasks across six datasets and three diseases.
arXiv Detail & Related papers (2023-06-08T15:39:27Z) - Incomplete Multimodal Learning for Complex Brain Disorders Prediction [65.95783479249745]
We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks.
We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative cohort.
arXiv Detail & Related papers (2023-05-25T16:29:16Z) - Hierarchical Graph Convolutional Network Built by Multiscale Atlases for
Brain Disorder Diagnosis Using Functional Connectivity [48.75665245214903]
We propose a novel framework to perform multiscale FCN analysis for brain disorder diagnosis.
We first use a set of well-defined multiscale atlases to compute multiscale FCNs.
Then, we utilize biologically meaningful brain hierarchical relationships among the regions in multiscale atlases to perform nodal pooling.
arXiv Detail & Related papers (2022-09-22T04:17:57Z) - BrainFormer: A Hybrid CNN-Transformer Model for Brain fMRI Data
Classification [31.83866719445596]
BrainFormer is a general hybrid Transformer architecture for brain disease classification with single fMRI volume.
BrainFormer is constructed by modeling the local cues within each voxel with 3D convolutions.
We evaluate BrainFormer on five independently acquired datasets including ABIDE, ADNI, MPILMBB, ADHD-200 and ECHO.
arXiv Detail & Related papers (2022-08-05T07:54:10Z) - Cross-Modality Deep Feature Learning for Brain Tumor Segmentation [158.8192041981564]
This paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data.
The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale.
Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance.
arXiv Detail & Related papers (2022-01-07T07:46:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.