Beware of diffusion models for synthesizing medical images -- A comparison with GANs in terms of memorizing brain MRI and chest x-ray images

• URL: http://arxiv.org/abs/2305.07644v3
• Date: Sun, 7 Jul 2024 19:09:39 GMT
• Title: Beware of diffusion models for synthesizing medical images -- A comparison with GANs in terms of memorizing brain MRI and chest x-ray images
• Authors: Muhammad Usman Akbar, Wuhao Wang, Anders Eklund,
• Abstract summary: We train StyleGAN and a diffusion model, using BRATS20, BRATS21 and a chest x-ray pneumonia dataset, to synthesize brain MRI and chest x-ray images. Our results show that diffusion models are more likely to memorize the training images, compared to StyleGAN, especially for small datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models were initially developed for text-to-image generation and are now being utilized to generate high quality synthetic images. Preceded by GANs, diffusion models have shown impressive results using various evaluation metrics. However, commonly used metrics such as FID and IS are not suitable for determining whether diffusion models are simply reproducing the training images. Here we train StyleGAN and a diffusion model, using BRATS20, BRATS21 and a chest x-ray pneumonia dataset, to synthesize brain MRI and chest x-ray images, and measure the correlation between the synthetic images and all training images. Our results show that diffusion models are more likely to memorize the training images, compared to StyleGAN, especially for small datasets and when using 2D slices from 3D volumes. Researchers should be careful when using diffusion models (and to some extent GANs) for medical imaging, if the final goal is to share the synthetic images.

Related papers

• DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception [66.88792390480343]
  We propose DEEM, a simple and effective approach that utilizes the generative feedback of diffusion models to align the semantic distributions of the image encoder. DEEM exhibits enhanced robustness and a superior capacity to alleviate hallucinations while utilizing fewer trainable parameters, less pre-training data, and a smaller base model size.
  arXiv  Detail & Related papers  (2024-05-24T05:46:04Z)
• Training Class-Imbalanced Diffusion Model Via Overlap Optimization [55.96820607533968]
  Diffusion models trained on real-world datasets often yield inferior fidelity for tail classes. Deep generative models, including diffusion models, are biased towards classes with abundant training images. We propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes.
  arXiv  Detail & Related papers  (2024-02-16T16:47:21Z)
• Optimizing Sampling Patterns for Compressed Sensing MRI with Diffusion Generative Models [75.52575380824051]
  We present a learning method to optimize sub-sampling patterns for compressed sensing multi-coil MRI. We use a single-step reconstruction based on the posterior mean estimate given by the diffusion model and the MRI measurement process. Our method requires as few as five training images to learn effective sampling patterns.
  arXiv  Detail & Related papers  (2023-06-05T22:09:06Z)
• Brain tumor segmentation using synthetic MR images -- A comparison of GANs and diffusion models [0.0]
  Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images. We show that segmentation networks trained on synthetic images reach Dice scores that are 80% - 90% of Dice scores when training with real images. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required.
  arXiv  Detail & Related papers  (2023-06-05T15:56:30Z)
• DIRE for Diffusion-Generated Image Detection [128.95822613047298]
  We propose a novel representation called DIffusion Reconstruction Error (DIRE) DIRE measures the error between an input image and its reconstruction counterpart by a pre-trained diffusion model. It provides a hint that DIRE can serve as a bridge to distinguish generated and real images.
  arXiv  Detail & Related papers  (2023-03-16T13:15:03Z)
• Extracting Training Data from Diffusion Models [77.11719063152027]
  We show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from state-of-the-art models. We train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy.
  arXiv  Detail & Related papers  (2023-01-30T18:53:09Z)
• Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models [53.03978584040557]
  We study image retrieval frameworks that enable us to compare generated images with training samples and detect when content has been replicated. Applying our frameworks to diffusion models trained on multiple datasets including Oxford flowers, Celeb-A, ImageNet, and LAION, we discuss how factors such as training set size impact rates of content replication.
  arXiv  Detail & Related papers  (2022-12-07T18:58:02Z)
• Medical Diffusion -- Denoising Diffusion Probabilistic Models for 3D Medical Image Generation [0.6486409713123691]
  We show that diffusion probabilistic models can synthesize high quality medical imaging data. We provide quantitative measurements of their performance through a reader study with two medical experts. We demonstrate that synthetic images can be used in a self-supervised pre-training and improve the performance of breast segmentation models when data is scarce.
  arXiv  Detail & Related papers  (2022-11-07T08:37:48Z)
• Spot the fake lungs: Generating Synthetic Medical Images using Neural Diffusion Models [1.0957528713294873]
  We use a pre-trained DALLE2 model to generate lungs X-Ray and CT images from an input text prompt. We train a stable diffusion model with 3165 X-Ray images and generate synthetic images. Results demonstrate that images generated with the diffusion model can translate characteristics that are otherwise very specific to certain medical conditions.
  arXiv  Detail & Related papers  (2022-11-02T06:02:55Z)
• Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models [7.745729132928934]
  We propose to use an emerging deep learning framework called diffusion and score-matching models. Our results show that the diffusion and score-matching models generate better synthetic CT images than the CNN and GAN models. Our study suggests that diffusion and score-matching models are powerful to generate high quality images conditioned on an image obtained using a complementary imaging modality.
  arXiv  Detail & Related papers  (2022-09-24T23:50:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.