Mask-conditioned latent diffusion for generating gastrointestinal polyp images

• URL: http://arxiv.org/abs/2304.05233v1
• Date: Tue, 11 Apr 2023 14:11:17 GMT
• Title: Mask-conditioned latent diffusion for generating gastrointestinal polyp images
• Authors: Roman Mach\'a\v{c}ek, Leila Mozaffari, Zahra Sepasdar, Sravanthi Parasa, P{\aa}l Halvorsen, Michael A. Riegler, Vajira Thambawita
• Abstract summary: This study proposes a conditional DPM framework to generate synthetic GI polyp images conditioned on given segmentation masks. Our system can generate an unlimited number of high-fidelity synthetic polyp images with the corresponding ground truth masks of polyps. Results show that the best micro-imagewise IOU of 0.7751 was achieved from DeepLabv3+ when the training data consists of both real data and synthetic data.
• Score: 2.027538200191349
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In order to take advantage of AI solutions in endoscopy diagnostics, we must overcome the issue of limited annotations. These limitations are caused by the high privacy concerns in the medical field and the requirement of getting aid from experts for the time-consuming and costly medical data annotation process. In computer vision, image synthesis has made a significant contribution in recent years as a result of the progress of generative adversarial networks (GANs) and diffusion probabilistic models (DPM). Novel DPMs have outperformed GANs in text, image, and video generation tasks. Therefore, this study proposes a conditional DPM framework to generate synthetic GI polyp images conditioned on given generated segmentation masks. Our experimental results show that our system can generate an unlimited number of high-fidelity synthetic polyp images with the corresponding ground truth masks of polyps. To test the usefulness of the generated data, we trained binary image segmentation models to study the effect of using synthetic data. Results show that the best micro-imagewise IOU of 0.7751 was achieved from DeepLabv3+ when the training data consists of both real data and synthetic data. However, the results reflect that achieving good segmentation performance with synthetic data heavily depends on model architectures.

Related papers

• Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis [55.959002385347645]
  Scaling by training on large datasets has been shown to enhance the quality and fidelity of image generation and manipulation with diffusion models. Latent Drifting enables diffusion models to be conditioned for medical images fitted for the complex task of counterfactual image generation. Our results demonstrate significant performance gains in various scenarios when combined with different fine-tuning schemes.
  arXiv  Detail & Related papers  (2024-12-30T01:59:34Z)
• MRGen: Diffusion-based Controllable Data Engine for MRI Segmentation towards Unannotated Modalities [59.61465292965639]
  This paper investigates a new paradigm for leveraging generative models in medical applications. We propose a diffusion-based data engine, termed MRGen, which enables generation conditioned on text prompts and masks.
  arXiv  Detail & Related papers  (2024-12-04T16:34:22Z)
• Medical Imaging Complexity and its Effects on GAN Performance [1.776717121506676]
  Medical image synthesis via generative adversarial networks (GANs) emerged as a powerful method for synthetically generating photo-realistic images. We experimentally establish benchmarks that measure the relationship between a sample dataset size and the fidelity of the generated images. We conduct experiments with two state-of-the-art GANs, StyleGAN 3 and SPADE-GAN, trained on multiple medical imaging datasets with variable sample sizes.
  arXiv  Detail & Related papers  (2024-10-23T15:28:25Z)
• Evaluating Utility of Memory Efficient Medical Image Generation: A Study on Lung Nodule Segmentation [0.0]
  This work proposes a memory-efficient patch-wise denoising diffusion probabilistic model (DDPM) for generating synthetic medical images. Our approach generates high-utility synthetic images with nodule segmentation while efficiently managing memory constraints. We evaluate the method in two scenarios: training a segmentation model exclusively on synthetic data, and augmenting real-world training data with synthetic images.
  arXiv  Detail & Related papers  (2024-10-16T13:20:57Z)
• MediSyn: A Generalist Text-Guided Latent Diffusion Model For Diverse Medical Image Synthesis [4.541407789437896]
  MediSyn is a text-guided latent diffusion model capable of generating synthetic images from 6 medical specialties and 10 image types. A direct comparison of the synthetic images against the real images confirms that our model synthesizes novel images and, crucially, may preserve patient privacy. Our findings highlight the immense potential for generalist image generative models to accelerate algorithmic research and development in medicine.
  arXiv  Detail & Related papers  (2024-05-16T04:28:44Z)
• A Domain Translation Framework with an Adversarial Denoising Diffusion Model to Generate Synthetic Datasets of Echocardiography Images [0.5999777817331317]
  We introduce a framework to create echocardiography images suitable to be used for clinical research purposes. For several domain translation operations, the results verified that such generative model was able to synthesize high quality image samples.
  arXiv  Detail & Related papers  (2024-03-07T15:58:03Z)
• UAV-Sim: NeRF-based Synthetic Data Generation for UAV-based Perception [62.71374902455154]
  We leverage recent advancements in neural rendering to improve static and dynamic novelview UAV-based image rendering. We demonstrate a considerable performance boost when a state-of-the-art detection model is optimized primarily on hybrid sets of real and synthetic data.
  arXiv  Detail & Related papers  (2023-10-25T00:20:37Z)
• DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model [3.890243179348094]
  Large-scale, big-variant, high-quality data are crucial for developing robust and successful deep-learning models for medical applications. This paper proposes a novel approach by developing controllable diffusion models for medical image synthesis, called DiffBoost. We leverage recent diffusion probabilistic models to generate realistic and diverse synthetic medical image data.
  arXiv  Detail & Related papers  (2023-10-19T16:18:02Z)
• ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models [69.9178140563928]
  Colonoscopy analysis is essential for assisting clinical diagnosis and treatment. The scarcity of annotated data limits the effectiveness and generalization of existing methods. We propose an Adaptive Refinement Semantic Diffusion Model (ArSDM) to generate colonoscopy images that benefit the downstream tasks.
  arXiv  Detail & Related papers  (2023-09-03T07:55:46Z)
• OADAT: Experimental and Synthetic Clinical Optoacoustic Data for Standardized Image Processing [62.993663757843464]
  Optoacoustic (OA) imaging is based on excitation of biological tissues with nanosecond-duration laser pulses followed by detection of ultrasound waves generated via light-absorption-mediated thermoelastic expansion. OA imaging features a powerful combination between rich optical contrast and high resolution in deep tissues. No standardized datasets generated with different types of experimental set-up and associated processing methods are available to facilitate advances in broader applications of OA in clinical settings.
  arXiv  Detail & Related papers  (2022-06-17T08:11:26Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.