A Vessel-Segmentation-Based CycleGAN for Unpaired Multi-modal Retinal Image Synthesis

• URL: http://arxiv.org/abs/2306.02901v1
• Date: Mon, 5 Jun 2023 14:06:43 GMT
• Title: A Vessel-Segmentation-Based CycleGAN for Unpaired Multi-modal Retinal Image Synthesis
• Authors: Aline Sindel, Andreas Maier, Vincent Christlein
• Abstract summary: Unpaired image-to-image translation of retinal images can efficiently increase the training dataset for deep-learning-based retinal registration methods. Our method integrates a vessel segmentation network into the image-to-image translation task by extending the CycleGAN framework.
• Score: 11.225641274591101
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unpaired image-to-image translation of retinal images can efficiently increase the training dataset for deep-learning-based multi-modal retinal registration methods. Our method integrates a vessel segmentation network into the image-to-image translation task by extending the CycleGAN framework. The segmentation network is inserted prior to a UNet vision transformer generator network and serves as a shared representation between both domains. We reformulate the original identity loss to learn the direct mapping between the vessel segmentation and the real image. Additionally, we add a segmentation loss term to ensure shared vessel locations between fake and real images. In the experiments, our method shows a visually realistic look and preserves the vessel structures, which is a prerequisite for generating multi-modal training data for image registration.

Related papers

• Distillation Learning Guided by Image Reconstruction for One-Shot Medical Image Segmentation [12.33442990188044]
  One-shot medical image segmentation (MIS) methods often struggle with registration errors and low-quality synthetic images. We introduce a novel one-shot MIS framework based on knowledge distillation. It allows the network to directly'see' real images through a distillation process guided by image reconstruction.
  arXiv  Detail & Related papers  (2024-08-07T08:17:34Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• Unsupervised Segmentation of Fetal Brain MRI using Deep Learning Cascaded Registration [2.494736313545503]
  Traditional deep learning-based automatic segmentation requires extensive training data with ground-truth labels. We propose a novel method based on multi-atlas segmentation, that accurately segments multiple tissues without relying on labeled data for training. Our method employs a cascaded deep learning network for 3D image registration, which computes small, incremental deformations to the moving image to align it precisely with the fixed image.
  arXiv  Detail & Related papers  (2023-07-07T13:17:12Z)
• VesselMorph: Domain-Generalized Retinal Vessel Segmentation via Shape-Aware Representation [12.194439938007672]
  Domain shift is an inherent property of medical images and has become a major obstacle for large-scale deployment of learning-based algorithms. We propose a method named VesselMorph which generalizes the 2D retinal vessel segmentation task by synthesizing a shape-aware representation. VesselMorph achieves superior generalization performance compared with competing methods in different domain shift scenarios.
  arXiv  Detail & Related papers  (2023-07-01T06:02:22Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• Cross-Modal Fusion Distillation for Fine-Grained Sketch-Based Image Retrieval [55.21569389894215]
  We propose a cross-attention framework for Vision Transformers (XModalViT) that fuses modality-specific information instead of discarding them. Our framework first maps paired datapoints from the individual photo and sketch modalities to fused representations that unify information from both modalities. We then decouple the input space of the aforementioned modality fusion network into independent encoders of the individual modalities via contrastive and relational cross-modal knowledge distillation.
  arXiv  Detail & Related papers  (2022-10-19T11:50:14Z)
• Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation [4.43142018105102]
  We propose a novel translation-based unsupervised deformable image registration approach to convert the multi-modal registration problem to a mono-modal one. Our approach incorporates a discriminator-free translation network to facilitate the training of the registration network and a patchwise contrastive loss to encourage the translation network to preserve object shapes.
  arXiv  Detail & Related papers  (2022-04-28T17:18:21Z)
• Unsupervised Domain Adaptation with Contrastive Learning for OCT Segmentation [49.59567529191423]
  We propose a novel semi-supervised learning framework for segmentation of volumetric images from new unlabeled domains. We jointly use supervised and contrastive learning, also introducing a contrastive pairing scheme that leverages similarity between nearby slices in 3D.
  arXiv  Detail & Related papers  (2022-03-07T19:02:26Z)
• Image-to-image Mapping with Many Domains by Sparse Attribute Transfer [71.28847881318013]
  Unsupervised image-to-image translation consists of learning a pair of mappings between two domains without known pairwise correspondences between points. Current convention is to approach this task with cycle-consistent GANs. We propose an alternate approach that directly restricts the generator to performing a simple sparse transformation in a latent layer.
  arXiv  Detail & Related papers  (2020-06-23T19:52:23Z)
• Image Embedded Segmentation: Uniting Supervised and Unsupervised Objectives for Segmenting Histopathological Images [0.0]
  This paper presents a new regularization method to train a fully convolutional network for semantic tissue segmentation. It relies on the benefit of unsupervised learning, in the form of image reconstruction, for network training. Our experiments demonstrate that it leads to better segmentation results in these datasets, compared to its counterparts.
  arXiv  Detail & Related papers  (2020-01-30T08:09:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.