CyTran: A Cycle-Consistent Transformer with Multi-Level Consistency for Non-Contrast to Contrast CT Translation

• URL: http://arxiv.org/abs/2110.06400v3
• Date: Wed, 5 Apr 2023 09:36:33 GMT
• Title: CyTran: A Cycle-Consistent Transformer with Multi-Level Consistency for Non-Contrast to Contrast CT Translation
• Authors: Nicolae-Catalin Ristea, Andreea-Iuliana Miron, Olivian Savencu, Mariana-Iuliana Georgescu, Nicolae Verga, Fahad Shahbaz Khan, Radu Tudor Ionescu
• Abstract summary: We propose a novel approach to translate unpaired contrast computed tomography (CT) scans to non-contrast CT scans. Our approach is based on cycle-consistent generative adversarial convolutional transformers, for short, CyTran. Our empirical results show that CyTran outperforms all competing methods.
• Score: 56.622832383316215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel approach to translate unpaired contrast computed tomography (CT) scans to non-contrast CT scans and the other way around. Solving this task has two important applications: (i) to automatically generate contrast CT scans for patients for whom injecting contrast substance is not an option, and (ii) to enhance the alignment between contrast and non-contrast CT by reducing the differences induced by the contrast substance before registration. Our approach is based on cycle-consistent generative adversarial convolutional transformers, for short, CyTran. Our neural model can be trained on unpaired images, due to the integration of a multi-level cycle-consistency loss. Aside from the standard cycle-consistency loss applied at the image level, we propose to apply additional cycle-consistency losses between intermediate feature representations, which enforces the model to be cycle-consistent at multiple representations levels, leading to superior results. To deal with high-resolution images, we design a hybrid architecture based on convolutional and multi-head attention layers. In addition, we introduce a novel data set, Coltea-Lung-CT-100W, containing 100 3D triphasic lung CT scans (with a total of 37,290 images) collected from 100 female patients (there is one examination per patient). Each scan contains three phases (non-contrast, early portal venous, and late arterial), allowing us to perform experiments to compare our novel approach with state-of-the-art methods for image style transfer. Our empirical results show that CyTran outperforms all competing methods. Moreover, we show that CyTran can be employed as a preliminary step to improve a state-of-the-art medical image alignment method. We release our novel model and data set as open source at https://github.com/ristea/cycle-transformer.

Related papers

• Improving Cone-Beam CT Image Quality with Knowledge Distillation-Enhanced Diffusion Model in Imbalanced Data Settings [6.157230849293829]
  Daily cone-beam CT (CBCT) imaging, pivotal for therapy adjustment, falls short in tissue density accuracy. We maximize CBCT data during therapy, complemented by sparse paired fan-beam CTs. Our approach shows promise in generating high-quality CT images from CBCT scans in RT.
  arXiv  Detail & Related papers  (2024-09-19T07:56:06Z)
• Gadolinium dose reduction for brain MRI using conditional deep learning [66.99830668082234]
  Two main challenges for these approaches are the accurate prediction of contrast enhancement and the synthesis of realistic images. We address both challenges by utilizing the contrast signal encoded in the subtraction images of pre-contrast and post-contrast image pairs. We demonstrate the effectiveness of our approach on synthetic and real datasets using various scanners, field strengths, and contrast agents.
  arXiv  Detail & Related papers  (2024-03-06T08:35:29Z)
• Enhanced Sharp-GAN For Histopathology Image Synthesis [63.845552349914186]
  Histopathology image synthesis aims to address the data shortage issue in training deep learning approaches for accurate cancer detection. We propose a novel approach that enhances the quality of synthetic images by using nuclei topology and contour regularization. The proposed approach outperforms Sharp-GAN in all four image quality metrics on two datasets.
  arXiv  Detail & Related papers  (2023-01-24T17:54:01Z)
• Anatomically constrained CT image translation for heterogeneous blood vessel segmentation [3.88838725116957]
  Anatomical structures in contrast-enhanced CT (ceCT) images can be challenging to segment due to variability in contrast medium diffusion. To limit the radiation dose, generative models could be used to synthesize one modality, instead of acquiring it. CycleGAN has attracted particular attention because it alleviates the need for paired data. We present an extension of CycleGAN to generate high fidelity images, with good structural consistency.
  arXiv  Detail & Related papers  (2022-10-04T16:14:49Z)
• Self-supervised 3D anatomy segmentation using self-distilled masked image transformer (SMIT) [2.7298989068857487]
  Self-supervised learning has demonstrated success in medical image segmentation using convolutional networks. We show our approach is more accurate and requires fewer fine tuning datasets than other pretext tasks.
  arXiv  Detail & Related papers  (2022-05-20T17:55:14Z)
• Harmonizing Pathological and Normal Pixels for Pseudo-healthy Synthesis [68.5287824124996]
  We present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. We apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T08:41:17Z)
• Incremental Cross-view Mutual Distillation for Self-supervised Medical CT Synthesis [88.39466012709205]
  This paper builds a novel medical slice to increase the between-slice resolution. Considering that the ground-truth intermediate medical slices are always absent in clinical practice, we introduce the incremental cross-view mutual distillation strategy. Our method outperforms state-of-the-art algorithms by clear margins.
  arXiv  Detail & Related papers  (2021-12-20T03:38:37Z)
• Flow-based Deformation Guidance for Unpaired Multi-Contrast MRI Image-to-Image Translation [7.8333615755210175]
  In this paper, we introduce a novel approach to unpaired image-to-image translation based on the invertible architecture. We utilize the temporal information between consecutive slices to provide more constraints to the optimization for transforming one domain to another in unpaired medical images.
  arXiv  Detail & Related papers  (2020-12-03T09:10:22Z)
• Co-Heterogeneous and Adaptive Segmentation from Multi-Source and Multi-Phase CT Imaging Data: A Study on Pathological Liver and Lesion Segmentation [48.504790189796836]
  We present a novel segmentation strategy, co-heterogenous and adaptive segmentation (CHASe) We propose a versatile framework that fuses appearance based semi-supervision, mask based adversarial domain adaptation, and pseudo-labeling. CHASe can further improve pathological liver mask Dice-Sorensen coefficients by ranges of $4.2% sim 9.4%$.
  arXiv  Detail & Related papers  (2020-05-27T06:58:39Z)
• A Deep learning Approach to Generate Contrast-Enhanced Computerised Tomography Angiography without the Use of Intravenous Contrast Agents [2.2840399926157806]
  We trained a 2-D Cycle Generative Adversarial Network for this non-contrast to contrast (NC2C) transformation task. This pipeline is able to differentiate between visually incoherent soft tissue regions in non-contrast CT images.
  arXiv  Detail & Related papers  (2020-03-02T22:20:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.