Multi-modal Evidential Fusion Network for Trusted PET/CT Tumor Segmentation

• URL: http://arxiv.org/abs/2406.18327v1
• Date: Wed, 26 Jun 2024 13:14:24 GMT
• Title: Multi-modal Evidential Fusion Network for Trusted PET/CT Tumor Segmentation
• Authors: Yuxuan Qi, Li Lin, Jiajun Wang, Jingya Zhang, Bin Zhang,
• Abstract summary: The quality of PET and CT images varies widely in clinical settings, which leads to uncertainty in the modality information extracted by networks. This paper proposes a novel Multi-modal Evidential Fusion Network (MEFN) comprising a Cross-Modal Feature Learning (CFL) module and a Multi-modal Trusted Fusion (MTF) module. Our model can provide radiologists with credible uncertainty of the segmentation results for their decision in accepting or rejecting the automatic segmentation results.
• Score: 5.839660501978193
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate segmentation of tumors in PET/CT images is important in computer-aided diagnosis and treatment of cancer. The key issue of such a segmentation problem lies in the effective integration of complementary information from PET and CT images. However, the quality of PET and CT images varies widely in clinical settings, which leads to uncertainty in the modality information extracted by networks. To take the uncertainty into account in multi-modal information fusion, this paper proposes a novel Multi-modal Evidential Fusion Network (MEFN) comprising a Cross-Modal Feature Learning (CFL) module and a Multi-modal Trusted Fusion (MTF) module. The CFL module reduces the domain gap upon modality conversion and highlights common tumor features, thereby alleviating the needs of the segmentation module to handle modality specificity. The MTF module utilizes mutual attention mechanisms and an uncertainty calibrator to fuse modality features based on modality uncertainty and then fuse the segmentation results under the guidance of Dempster-Shafer Theory. Besides, a new uncertainty perceptual loss is introduced to force the model focusing on uncertain features and hence improve its ability to extract trusted modality information. Extensive comparative experiments are conducted on two publicly available PET/CT datasets to evaluate the performance of our proposed method whose results demonstrate that our MEFN significantly outperforms state-of-the-art methods with improvements of 2.15% and 3.23% in DSC scores on the AutoPET dataset and the Hecktor dataset, respectively. More importantly, our model can provide radiologists with credible uncertainty of the segmentation results for their decision in accepting or rejecting the automatic segmentation results, which is particularly important for clinical applications. Our code will be available at https://github.com/QPaws/MEFN.

Related papers

• H2ASeg: Hierarchical Adaptive Interaction and Weighting Network for Tumor Segmentation in PET/CT Images [6.753315684414596]
  Positron emission tomography (PET) combined with computed tomography (CT) imaging is routinely used in cancer diagnosis and prognosis. Traditional multi-modal segmentation solutions rely on concatenation operations for modality fusion. We propose a Hierarchical Adaptive Interaction and Weighting Network termed H2ASeg to explore intrinsic cross-modal correlations.
  arXiv  Detail & Related papers  (2024-03-27T08:28:14Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction [62.29541106695824]
  This paper presents a coarse-to-fine PET reconstruction framework that consists of a coarse prediction module (CPM) and an iterative refinement module (IRM) By delegating most of the computational overhead to the CPM, the overall sampling speed of our method can be significantly improved. Two additional strategies, i.e., an auxiliary guidance strategy and a contrastive diffusion strategy, are proposed and integrated into the reconstruction process.
  arXiv  Detail & Related papers  (2023-08-20T04:10:36Z)
• Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images [55.83984261827332]
  In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network. We develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module. Our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches.
  arXiv  Detail & Related papers  (2022-12-01T07:32:56Z)
• ISA-Net: Improved spatial attention network for PET-CT tumor segmentation [22.48294544919023]
  We propose a deep learning segmentation method based on multimodal positron emission tomography-computed tomography (PET-CT) We design an improved spatial attention network(ISA-Net) to increase the accuracy of PET or CT in detecting tumors. We validated the proposed ISA-Net method on two clinical datasets, a soft tissue sarcoma(STS) and a head and neck tumor(HECKTOR) dataset.
  arXiv  Detail & Related papers  (2022-11-04T04:15:13Z)
• Cross-Modality Deep Feature Learning for Brain Tumor Segmentation [158.8192041981564]
  This paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance.
  arXiv  Detail & Related papers  (2022-01-07T07:46:01Z)
• Deep PET/CT fusion with Dempster-Shafer theory for lymphoma segmentation [17.623576885481747]
  Lymphoma detection and segmentation from PET/CT volumes are crucial for surgical indication and radiotherapy. We propose an lymphoma segmentation model using an UNet with an evidential PET/CT fusion layer. Our method get accurate segmentation results with Dice score of 0.726, without any user interaction.
  arXiv  Detail & Related papers  (2021-08-11T19:24:40Z)
• Cross-Modality Brain Tumor Segmentation via Bidirectional Global-to-Local Unsupervised Domain Adaptation [61.01704175938995]
  In this paper, we propose a novel Bidirectional Global-to-Local (BiGL) adaptation framework under a UDA scheme. Specifically, a bidirectional image synthesis and segmentation module is proposed to segment the brain tumor. The proposed method outperforms several state-of-the-art unsupervised domain adaptation methods by a large margin.
  arXiv  Detail & Related papers  (2021-05-17T10:11:45Z)
• Evidential segmentation of 3D PET/CT images [20.65495780362289]
  A segmentation method based on belief functions is proposed to segment lymphomas in 3D PET/CT images. The architecture is composed of a feature extraction module and an evidential segmentation (ES) module. The method was evaluated on a database of 173 patients with diffuse large b-cell lymphoma.
  arXiv  Detail & Related papers  (2021-04-27T16:06:27Z)
• Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation [11.622615048002567]
  Multimodal spatial attention module (MSAM) learns to emphasize regions related to tumors. MSAM can be applied to common backbone architectures and trained end-to-end.
  arXiv  Detail & Related papers  (2020-07-29T10:27:22Z)
• Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion [71.87627318863612]
  We propose a novel multimodal segmentation framework which is robust to the absence of imaging modalities. Our network uses feature disentanglement to decompose the input modalities into the modality-specific appearance code. We validate our method on the important yet challenging multimodal brain tumor segmentation task with the BRATS challenge dataset.
  arXiv  Detail & Related papers  (2020-02-22T14:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.