Unsupervised Tumor-Aware Distillation for Multi-Modal Brain Image Translation

• URL: http://arxiv.org/abs/2403.20168v2
• Date: Wed, 24 Apr 2024 09:31:11 GMT
• Title: Unsupervised Tumor-Aware Distillation for Multi-Modal Brain Image Translation
• Authors: Chuan Huang, Jia Wei, Rui Li,
• Abstract summary: Unsupervised multi-modal brain image translation has been extensively studied. Existing methods suffer from the problem of brain tumor deformation during translation. We propose an unsupervised tumor-aware distillation teacher-student network called UTAD-Net.
• Score: 8.380597715285237
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modal brain images from MRI scans are widely used in clinical diagnosis to provide complementary information from different modalities. However, obtaining fully paired multi-modal images in practice is challenging due to various factors, such as time, cost, and artifacts, resulting in modality-missing brain images. To address this problem, unsupervised multi-modal brain image translation has been extensively studied. Existing methods suffer from the problem of brain tumor deformation during translation, as they fail to focus on the tumor areas when translating the whole images. In this paper, we propose an unsupervised tumor-aware distillation teacher-student network called UTAD-Net, which is capable of perceiving and translating tumor areas precisely. Specifically, our model consists of two parts: a teacher network and a student network. The teacher network learns an end-to-end mapping from source to target modality using unpaired images and corresponding tumor masks first. Then, the translation knowledge is distilled into the student network, enabling it to generate more realistic tumor areas and whole images without masks. Experiments show that our model achieves competitive performance on both quantitative and qualitative evaluations of image quality compared with state-of-the-art methods. Furthermore, we demonstrate the effectiveness of the generated images on downstream segmentation tasks. Our code is available at https://github.com/scut-HC/UTAD-Net.

Related papers

• Generative Adversarial Networks for Brain Images Synthesis: A Review [2.609784101826762]
  In medical imaging, image synthesis is the estimation process of one image (sequence, modality) from another image (sequence, modality) generative adversarial network (GAN) as one of the most popular generative-based deep learning methods. We summarized the recent developments of GANs for cross-modality brain image synthesis including CT to PET, CT to MRI, MRI to PET, and vice versa.
  arXiv  Detail & Related papers  (2023-05-16T17:28:06Z)
• Learning to Learn Unlearned Feature for Brain Tumor Segmentation [13.402170359958752]
  We propose a fine-tuning algorithm for brain tumor segmentation that needs only a few data samples and helps networks not to forget the original tasks. We show a transfer learning method from high grade glioma to brain metastasis, and demonstrate that the proposed algorithm achieves balanced parameters for both glioma and brain metastasis domains within a few steps.
  arXiv  Detail & Related papers  (2023-05-13T05:26:25Z)
• Brain Tumor Segmentation from MRI Images using Deep Learning Techniques [3.1498833540989413]
  A public MRI dataset contains 3064 TI-weighted images from 233 patients with three variants of brain tumor, viz. meningioma, glioma, and pituitary tumor. The dataset files were converted and preprocessed before indulging into the methodology which employs implementation and training of some well-known image segmentation deep learning models. The experimental findings showed that among all the applied approaches, the recurrent residual U-Net which uses Adam reaches a Mean Intersection Over Union of 0.8665 and outperforms other compared state-of-the-art deep learning models.
  arXiv  Detail & Related papers  (2023-04-29T13:33:21Z)
• Unsupervised Brain Tumor Segmentation with Image-based Prompts [12.525656002678856]
  We propose an approach to unsupervised brain tumor segmentation by designing image-based prompts that allow indication of brain tumors. Instead of directly training a model for brain tumor segmentation with a large amount of annotated data, we seek to train a model that can answer the question: is a voxel in the input image associated with tumor-like hyper-/hypo-intensity? Since the hand-crafted designs may be too simplistic to represent all kinds of real tumors, the trained model may overfit the simplistic hand-crafted task rather than actually answer the question of abnormality.
  arXiv  Detail & Related papers  (2023-04-04T02:28:25Z)
• Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training [62.215025958347105]
  We propose a self-supervised learning paradigm with multi-modal masked autoencoders. We learn cross-modal domain knowledge by reconstructing missing pixels and tokens from randomly masked images and texts.
  arXiv  Detail & Related papers  (2022-09-15T07:26:43Z)
• Cross-Modality Deep Feature Learning for Brain Tumor Segmentation [158.8192041981564]
  This paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance.
  arXiv  Detail & Related papers  (2022-01-07T07:46:01Z)
• Triplet Contrastive Learning for Brain Tumor Classification [99.07846518148494]
  We present a novel approach of directly learning deep embeddings for brain tumor types, which can be used for downstream tasks such as classification. We evaluate our method on an extensive brain tumor dataset which consists of 27 different tumor classes, out of which 13 are defined as rare.
  arXiv  Detail & Related papers  (2021-08-08T11:26:34Z)
• A Multi-Stage Attentive Transfer Learning Framework for Improving COVID-19 Diagnosis [49.3704402041314]
  We propose a multi-stage attentive transfer learning framework for improving COVID-19 diagnosis. Our proposed framework consists of three stages to train accurate diagnosis models through learning knowledge from multiple source tasks and data of different domains. Importantly, we propose a novel self-supervised learning method to learn multi-scale representations for lung CT images.
  arXiv  Detail & Related papers  (2021-01-14T01:39:19Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)
• Interpretation of 3D CNNs for Brain MRI Data Classification [56.895060189929055]
  We extend the previous findings in gender differences from diffusion-tensor imaging on T1 brain MRI scans. We provide the voxel-wise 3D CNN interpretation comparing the results of three interpretation methods.
  arXiv  Detail & Related papers  (2020-06-20T17:56:46Z)
• Weakly Supervised PET Tumor Detection Using Class Response [3.947298454012977]
  We present a novel approach to locate different type of lesions in positron emission tomography (PET) images using only a class label at the image-level. The advantage of our proposed method consists of detecting the whole tumor volume in 3D images, using only two 2D images of PET image, and showing a very promising results.
  arXiv  Detail & Related papers  (2020-03-18T17:06:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.