CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with
Modality-Correlated Cross-Attention for Brain Tumor Segmentation
- URL: http://arxiv.org/abs/2207.07370v1
- Date: Fri, 15 Jul 2022 09:35:29 GMT
- Title: CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with
Modality-Correlated Cross-Attention for Brain Tumor Segmentation
- Authors: Jianwei Lin, Jiatai Lin, Cheng Lu, Hao Chen, Huan Lin, Bingchao Zhao,
Zhenwei Shi, Bingjiang Qiu, Xipeng Pan, Zeyan Xu, Biao Huang, Changhong
Liang, Guoqiang Han, Zaiyi Liu, Chu Han
- Abstract summary: Brain tumor segmentation in magnetic resonance image (MRI) is crucial for brain tumor diagnosis, cancer management and research purposes.
With the great success of the ten-year BraTS challenges, a lot of outstanding BTS models have been proposed to tackle the difficulties of BTS in different technical aspects.
We propose a clinical knowledge-driven brain tumor segmentation model, called CKD-TransBTS.
- Score: 37.39921484146194
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Brain tumor segmentation (BTS) in magnetic resonance image (MRI) is crucial
for brain tumor diagnosis, cancer management and research purposes. With the
great success of the ten-year BraTS challenges as well as the advances of CNN
and Transformer algorithms, a lot of outstanding BTS models have been proposed
to tackle the difficulties of BTS in different technical aspects. However,
existing studies hardly consider how to fuse the multi-modality images in a
reasonable manner. In this paper, we leverage the clinical knowledge of how
radiologists diagnose brain tumors from multiple MRI modalities and propose a
clinical knowledge-driven brain tumor segmentation model, called CKD-TransBTS.
Instead of directly concatenating all the modalities, we re-organize the input
modalities by separating them into two groups according to the imaging
principle of MRI. A dual-branch hybrid encoder with the proposed
modality-correlated cross-attention block (MCCA) is designed to extract the
multi-modality image features. The proposed model inherits the strengths from
both Transformer and CNN with the local feature representation ability for
precise lesion boundaries and long-range feature extraction for 3D volumetric
images. To bridge the gap between Transformer and CNN features, we propose a
Trans&CNN Feature Calibration block (TCFC) in the decoder. We compare the
proposed model with five CNN-based models and six transformer-based models on
the BraTS 2021 challenge dataset. Extensive experiments demonstrate that the
proposed model achieves state-of-the-art brain tumor segmentation performance
compared with all the competitors.
Related papers
- multiPI-TransBTS: A Multi-Path Learning Framework for Brain Tumor Image Segmentation Based on Multi-Physical Information [1.7359724605901228]
Brain Tumor distances (BraTS) plays a critical role in clinical diagnosis, treatment planning, and monitoring the progression of brain tumors.
Due to the variability in tumor appearance, size, and intensity across different MRI modalities, automated segmentation remains a challenging task.
We propose a novel Transformer-based framework, multiPI-TransBTS, which integrates multi-physical information to enhance segmentation accuracy.
arXiv Detail & Related papers (2024-09-18T17:35:19Z) - Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI [58.809276442508256]
We propose a hybrid network via the combination of convolution neural network (CNN) and transformer layers.
The experimental results on private and public DCE-MRI datasets demonstrate that the proposed hybrid network superior performance than the state-of-the-art methods.
arXiv Detail & Related papers (2024-08-11T15:46:00Z) - Enhancing Incomplete Multi-modal Brain Tumor Segmentation with Intra-modal Asymmetry and Inter-modal Dependency [31.047259264831947]
A common problem in practice is the unavailability of some modalities due to varying scanning protocols and patient conditions.
Previous methods have attempted to address this by fusing accessible multi-modal features, leveraging attention mechanisms, and synthesizing missing modalities.
We propose a novel approach that enhances the deep learning-based brain tumor segmentation model from two perspectives.
arXiv Detail & Related papers (2024-06-14T16:54:53Z) - SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion
Classification Using 3D Multi-Phase Imaging [59.78761085714715]
This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification.
The proposed framework has been validated through comprehensive experiments on two clinical datasets.
To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
arXiv Detail & Related papers (2024-02-27T06:32:56Z) - Cross-modality Guidance-aided Multi-modal Learning with Dual Attention
for MRI Brain Tumor Grading [47.50733518140625]
Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly.
We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
arXiv Detail & Related papers (2024-01-17T07:54:49Z) - The Brain Tumor Segmentation (BraTS) Challenge 2023: Brain MR Image Synthesis for Tumor Segmentation (BraSyn) [9.082208613256295]
We present the establishment of the Brain MR Image Synthesis Benchmark (BraSyn) in conjunction with the Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2023.
The primary objective of this challenge is to evaluate image synthesis methods that can realistically generate missing MRI modalities when multiple available images are provided.
arXiv Detail & Related papers (2023-05-15T20:49:58Z) - Patched Diffusion Models for Unsupervised Anomaly Detection in Brain MRI [55.78588835407174]
We propose a method that reformulates the generation task of diffusion models as a patch-based estimation of healthy brain anatomy.
We evaluate our approach on data of tumors and multiple sclerosis lesions and demonstrate a relative improvement of 25.1% compared to existing baselines.
arXiv Detail & Related papers (2023-03-07T09:40:22Z) - ME-Net: Multi-Encoder Net Framework for Brain Tumor Segmentation [6.643336433892116]
We propose a model for brain tumor segmentation with multiple encoders.
Four encoders correspond to the four modalities of the MRI image, perform one-to-one feature extraction, and then merge the feature maps of the four modalities into the decoder.
We also introduced a new loss function named "Categorical Dice", and set different weights for different segmented regions at the same time, which solved the problem of voxel imbalance.
arXiv Detail & Related papers (2022-03-21T14:42:05Z) - Cross-Modality Deep Feature Learning for Brain Tumor Segmentation [158.8192041981564]
This paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data.
The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale.
Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance.
arXiv Detail & Related papers (2022-01-07T07:46:01Z) - Soft Tissue Sarcoma Co-Segmentation in Combined MRI and PET/CT Data [2.2515303891664358]
Tumor segmentation in multimodal medical images has seen a growing trend towards deep learning based methods.
We propose a simultaneous co-segmentation method, which enables multimodal feature learning through modality-specific encoder and decoder branches.
We demonstrate the effectiveness of our approach on public soft tissue sarcoma data, which comprises MRI (T1 and T2 sequence) and PET/CT scans.
arXiv Detail & Related papers (2020-08-28T09:15:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.