Generating Realistic Counterfactuals for Retinal Fundus and OCT Images using Diffusion Models

• URL: http://arxiv.org/abs/2311.11629v2
• Date: Mon, 4 Dec 2023 17:01:20 GMT
• Title: Generating Realistic Counterfactuals for Retinal Fundus and OCT Images using Diffusion Models
• Authors: Indu Ilanchezian, Valentyn Boreiko, Laura K\"uhlewein, Ziwei Huang, Murat Se\c{c}kin Ayhan, Matthias Hein, Lisa Koch, Philipp Berens
• Abstract summary: Counterfactual reasoning is often used in clinical settings to explain decisions or weigh alternatives. Here, we demonstrate that using a diffusion model in combination with an adversarially robust classifier trained on retinal disease classification tasks enables the generation of highly realistic counterfactuals. In a user study, domain experts found the counterfactuals generated using our method significantly more realistic than counterfactuals generated from a previous method, and even indistinguishable from real images.
• Score: 36.81751569090276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Counterfactual reasoning is often used in clinical settings to explain decisions or weigh alternatives. Therefore, for imaging based specialties such as ophthalmology, it would be beneficial to be able to create counterfactual images, illustrating answers to questions like "If the subject had had diabetic retinopathy, how would the fundus image have looked?". Here, we demonstrate that using a diffusion model in combination with an adversarially robust classifier trained on retinal disease classification tasks enables the generation of highly realistic counterfactuals of retinal fundus images and optical coherence tomography (OCT) B-scans. The key to the realism of counterfactuals is that these classifiers encode salient features indicative for each disease class and can steer the diffusion model to depict disease signs or remove disease-related lesions in a realistic way. In a user study, domain experts also found the counterfactuals generated using our method significantly more realistic than counterfactuals generated from a previous method, and even indistinguishable from real images.

Related papers

• StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model [62.25424831998405]
  StealthDiffusion is a framework that modifies AI-generated images into high-quality, imperceptible adversarial examples. It is effective in both white-box and black-box settings, transforming AI-generated images into high-quality adversarial forgeries.
  arXiv  Detail & Related papers  (2024-08-11T01:22:29Z)
• Mask-guided cross-image attention for zero-shot in-silico histopathologic image generation with a diffusion model [0.10910416614141322]
  Diffusion models are the state-of-the-art solution for generating in-silico images. Appearance transfer diffusion models are designed for natural images. In computational pathology, specifically in oncology, it is not straightforward to define which objects in an image should be classified as foreground and background. We contribute to the applicability of appearance transfer models to diffusion-stained images by modifying the appearance transfer guidance to alternate between class-specific AdaIN feature statistics matchings.
  arXiv  Detail & Related papers  (2024-07-16T12:36:26Z)
• Fundus-Enhanced Disease-Aware Distillation Model for Retinal Disease Classification from OCT Images [6.72159216082989]
  We propose a fundus-enhanced disease-aware distillation model for retinal disease classification from OCT images. Our framework enhances the OCT model during training by utilizing unpaired fundus images. Our proposed approach outperforms single-modal, multi-modal, and state-of-the-art distillation methods for retinal disease classification.
  arXiv  Detail & Related papers  (2023-08-01T05:13:02Z)
• Generation of Structurally Realistic Retinal Fundus Images with Diffusion Models [1.9346186297861747]
  We generate artery/vein masks to create the vascular structure, which we then condition to produce retinal fundus images. The proposed method can generate high-quality images with more realistic vascular structures.
  arXiv  Detail & Related papers  (2023-05-11T14:09:05Z)
• Improving Classification of Retinal Fundus Image Using Flow Dynamics Optimized Deep Learning Methods [0.0]
  Diabetic Retinopathy (DR) refers to a barrier that takes place in diabetes mellitus damaging the blood vessel network present in the retina. It can take some time to perform a DR diagnosis using color fundus pictures because experienced clinicians are required to identify the tumors in the imagery used to identify the illness.
  arXiv  Detail & Related papers  (2023-04-29T16:11:34Z)
• Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains [3.8137985834223502]
  Building generative models for medical images that faithfully depict clinical context may help alleviate the paucity of healthcare datasets. We explore the sub-components of the Stable Diffusion pipeline to fine-tune the model to generate medical images. Our best-performing model improves upon the stable diffusion baseline and can be conditioned to insert a realistic-looking abnormality on a synthetic radiology image.
  arXiv  Detail & Related papers  (2022-10-09T01:43:08Z)
• Assessing glaucoma in retinal fundus photographs using Deep Feature Consistent Variational Autoencoders [63.391402501241195]
  glaucoma is challenging to detect since it remains asymptomatic until the symptoms are severe. Early identification of glaucoma is generally made based on functional, structural, and clinical assessments. Deep learning methods have partially solved this dilemma by bypassing the marker identification stage and analyzing high-level information directly to classify the data.
  arXiv  Detail & Related papers  (2021-10-04T16:06:49Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• An Interpretable Multiple-Instance Approach for the Detection of referable Diabetic Retinopathy from Fundus Images [72.94446225783697]
  We propose a machine learning system for the detection of referable Diabetic Retinopathy in fundus images. By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy. We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-03-02T13:14:15Z)
• Modeling and Enhancing Low-quality Retinal Fundus Images [167.02325845822276]
  Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. We propose a clinically oriented fundus enhancement network (cofe-Net) to suppress global degradation factors. Experiments on both synthetic and real images demonstrate that our algorithm effectively corrects low-quality fundus images without losing retinal details.
  arXiv  Detail & Related papers  (2020-05-12T08:01:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.