TarGAN: Target-Aware Generative Adversarial Networks for Multi-modality Medical Image Translation

• URL: http://arxiv.org/abs/2105.08993v1
• Date: Wed, 19 May 2021 08:45:33 GMT
• Title: TarGAN: Target-Aware Generative Adversarial Networks for Multi-modality Medical Image Translation
• Authors: Junxiao Chen, Jia Wei, and Rui Li
• Abstract summary: We propose a novel target-aware generative adversarial network called TarGAN. TarGAN is capable of learning multi-modality medical image translation without relying on paired data. Experiments on both quantitative measures and qualitative evaluations demonstrate that TarGAN outperforms the state-of-the-art methods in all cases.
• Score: 4.333115837538408
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Paired multi-modality medical images, can provide complementary information to help physicians make more reasonable decisions than single modality medical images. But they are difficult to generate due to multiple factors in practice (e.g., time, cost, radiation dose). To address these problems, multi-modality medical image translation has aroused increasing research interest recently. However, the existing works mainly focus on translation effect of a whole image instead of a critical target area or Region of Interest (ROI), e.g., organ and so on. This leads to poor-quality translation of the localized target area which becomes blurry, deformed or even with extra unreasonable textures. In this paper, we propose a novel target-aware generative adversarial network called TarGAN, which is a generic multi-modality medical image translation model capable of (1) learning multi-modality medical image translation without relying on paired data, (2) enhancing quality of target area generation with the help of target area labels. The generator of TarGAN jointly learns mapping at two levels simultaneously - whole image translation mapping and target area translation mapping. These two mappings are interrelated through a proposed crossing loss. The experiments on both quantitative measures and qualitative evaluations demonstrate that TarGAN outperforms the state-of-the-art methods in all cases. Subsequent segmentation task is conducted to demonstrate effectiveness of synthetic images generated by TarGAN in a real-world application. Our code is available at https://github.com/2165998/TarGAN.

Related papers

• AV-GAN: Attention-Based Varifocal Generative Adversarial Network for Uneven Medical Image Translation [25.665817446819386]
  We develop an Attention-Based Key Region Selection Module, which can attend to regions with higher translation difficulty. We then develop a Varifocal Module to translate these regions at multiple resolutions. Experimental results show that our proposed AV-GAN outperforms existing image translation methods.
  arXiv  Detail & Related papers  (2024-04-16T16:43:36Z)
• Cascaded Multi-path Shortcut Diffusion Model for Medical Image Translation [26.67518950976257]
  We propose a Cascade Multi-path Shortcut Diffusion Model (CMDM) for high-quality medical image translation and uncertainty estimation. Our experimental results found that CMDM can produce high-quality translations comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-04-06T03:02:47Z)
• C^2M-DoT: Cross-modal consistent multi-view medical report generation with domain transfer network [67.97926983664676]
  We propose a cross-modal consistent multi-view medical report generation with a domain transfer network (C2M-DoT) C2M-DoT substantially outperforms state-of-the-art baselines in all metrics.
  arXiv  Detail & Related papers  (2023-10-09T02:31:36Z)
• Multi-Level Global Context Cross Consistency Model for Semi-Supervised Ultrasound Image Segmentation with Diffusion Model [0.0]
  We propose a framework that uses images generated by a Latent Diffusion Model (LDM) as unlabeled images for semi-supervised learning. Our approach enables the effective transfer of probability distribution knowledge to the segmentation network, resulting in improved segmentation accuracy.
  arXiv  Detail & Related papers  (2023-05-16T14:08:24Z)
• Guided Image-to-Image Translation by Discriminator-Generator Communication [71.86347329356244]
  The goal of Image-to-image (I2I) translation is to transfer an image from a source domain to a target domain. One major branch of this research is to formulate I2I translation based on Generative Adversarial Network (GAN)
  arXiv  Detail & Related papers  (2023-03-07T02:29:36Z)
• AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation [50.21065317817769]
  We propose an AlignTransformer framework, which includes the Align Hierarchical Attention (AHA) and the Multi-Grained Transformer (MGT) modules. Experiments on the public IU-Xray and MIMIC-CXR datasets show that the AlignTransformer can achieve results competitive with state-of-the-art methods on the two datasets.
  arXiv  Detail & Related papers  (2022-03-18T13:43:53Z)
• Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network [73.5062435623908]
  We propose a new I2I translation method that generates a new model in the target domain via a series of model transformations. By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain.
  arXiv  Detail & Related papers  (2020-10-12T13:51:40Z)
• Discriminative Cross-Modal Data Augmentation for Medical Imaging Applications [24.06277026586584]
  Deep learning methods have shown great success in medical image analysis, they require a number of medical images to train. Due to data privacy concerns and unavailability of medical annotators, it is oftentimes very difficult to obtain a lot of labeled medical images for model training. We propose a discriminative unpaired image-to-image translation model which translates images in source modality into images in target modality.
  arXiv  Detail & Related papers  (2020-10-07T15:07:00Z)
• MI^2GAN: Generative Adversarial Network for Medical Image Domain Adaptation using Mutual Information Constraint [47.07869311690419]
  We propose a novel GAN to maintain image-contents during cross-domain I2I translation. Particularly, we disentangle the content features from domain information for both the source and translated images. The proposed MI$2$GAN is evaluated on two tasks---polyp segmentation using colonoscopic images and the segmentation of optic disc and cup in fundus images.
  arXiv  Detail & Related papers  (2020-07-22T03:19:54Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.