Disentangling representations of retinal images with generative models

• URL: http://arxiv.org/abs/2402.19186v3
• Date: Mon, 23 Jun 2025 10:48:12 GMT
• Title: Disentangling representations of retinal images with generative models
• Authors: Sarah Müller, Lisa M. Koch, Hendrik P. A. Lensch, Philipp Berens,
• Abstract summary: We introduce a population model for retinal fundus images that effectively disentangles patient attributes from camera effects.<n>We show that our models encode desired information in disentangled subspaces and enable controllable image generation based on the learned subspaces.
• Score: 12.235975451036047
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retinal fundus images play a crucial role in the early detection of eye diseases. However, the impact of technical factors on these images can pose challenges for reliable AI applications in ophthalmology. For example, large fundus cohorts are often confounded by factors like camera type, bearing the risk of learning shortcuts rather than the causal relationships behind the image generation process. Here, we introduce a population model for retinal fundus images that effectively disentangles patient attributes from camera effects, enabling controllable and highly realistic image generation. To achieve this, we propose a disentanglement loss based on distance correlation. Through qualitative and quantitative analyses, we show that our models encode desired information in disentangled subspaces and enable controllable image generation based on the learned subspaces, demonstrating the effectiveness of our disentanglement loss. The project's code is publicly available: https://github.com/berenslab/disentangling-retinal-images.

Related papers

• Exploring the Effectiveness of Deep Features from Domain-Specific Foundation Models in Retinal Image Synthesis [0.0]
  The adoption of neural network models in medical imaging has been constrained by strict privacy regulations, limited data availability, high acquisition costs, and demographic biases.<n>In this study, we in-vestigated whether a distance-based loss function based on deep activation layers of a large foundational model trained on large corpus of domain data, colour fundus imaging, offers advantages over a perceptual loss and edge-detection based loss functions.
  arXiv  Detail & Related papers  (2025-06-13T13:09:11Z)
• PixCell: A generative foundation model for digital histopathology images [49.00921097924924]
  We introduce PixCell, the first diffusion-based generative foundation model for histopathology.<n>We train PixCell on PanCan-30M, a vast, diverse dataset derived from 69,184 H&E-stained whole slide images covering various cancer types.
  arXiv  Detail & Related papers  (2025-06-05T15:14:32Z)
• Causal Disentanglement for Robust Long-tail Medical Image Generation [80.15257897500578]
  We propose a novel medical image generation framework, which generates independent pathological and structural features. We leverage a diffusion model guided by pathological findings to model pathological features, enabling the generation of diverse counterfactual images.
  arXiv  Detail & Related papers  (2025-04-20T01:54:18Z)
• Enhancing Retinal Disease Classification from OCTA Images via Active Learning Techniques [0.8035416719640156]
  Eye diseases are common in older Americans and can lead to decreased vision and blindness. Recent advancements in imaging technologies allow clinicians to capture high-quality images of the retinal blood vessels via Optical Coherence Tomography Angiography ( OCTA) OCTA provides detailed vascular imaging as compared to the solely structural information obtained by common OCT imaging.
  arXiv  Detail & Related papers  (2024-07-21T23:24:49Z)
• Mask-guided cross-image attention for zero-shot in-silico histopathologic image generation with a diffusion model [0.10910416614141322]
  Diffusion models are the state-of-the-art solution for generating in-silico images.<n>Appearance transfer diffusion models are designed for natural images.<n>In computational pathology, specifically in oncology, it is not straightforward to define which objects in an image should be classified as foreground and background.<n>We contribute to the applicability of appearance transfer models to diffusion-stained images by modifying the appearance transfer guidance to alternate between class-specific AdaIN feature statistics matchings.
  arXiv  Detail & Related papers  (2024-07-16T12:36:26Z)
• How to Trace Latent Generative Model Generated Images without Artificial Watermark? [88.04880564539836]
  Concerns have arisen regarding potential misuse related to images generated by latent generative models. We propose a latent inversion based method called LatentTracer to trace the generated images of the inspected model. Our experiments show that our method can distinguish the images generated by the inspected model and other images with a high accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-05-22T05:33:47Z)
• Enhance Eye Disease Detection using Learnable Probabilistic Discrete Latents in Machine Learning Architectures [1.6000489723889526]
  Ocular diseases, including diabetic retinopathy and glaucoma, present a significant public health challenge. Deep learning models have emerged as powerful tools for analysing medical images, such as retina imaging. Challenges persist in model relibability and uncertainty estimation, which are critical for clinical decision-making.
  arXiv  Detail & Related papers  (2024-01-21T04:14:54Z)
• InceptionCaps: A Performant Glaucoma Classification Model for Data-scarce Environment [0.0]
  glaucoma is an irreversible ocular disease and is the second leading cause of visual disability worldwide. This work reviews existing state of the art models and proposes InceptionCaps, a novel capsule network (CapsNet) based deep learning model having pre-trained InceptionV3 as its convolution base, for automatic glaucoma classification. InceptionCaps achieved an accuracy of 0.956, specificity of 0.96, and AUC of 0.9556, which surpasses several state-of-the-art deep learning model performances on the RIM-ONE v2 dataset.
  arXiv  Detail & Related papers  (2023-11-24T11:58:11Z)
• Generating Realistic Counterfactuals for Retinal Fundus and OCT Images using Diffusion Models [36.81751569090276]
  Counterfactual reasoning is often used in clinical settings to explain decisions or weigh alternatives. Here, we demonstrate that using a diffusion model in combination with an adversarially robust classifier trained on retinal disease classification tasks enables the generation of highly realistic counterfactuals. In a user study, domain experts found the counterfactuals generated using our method significantly more realistic than counterfactuals generated from a previous method, and even indistinguishable from real images.
  arXiv  Detail & Related papers  (2023-11-20T09:28:04Z)
• Gravity Network for end-to-end small lesion detection [50.38534263407915]
  This paper introduces a novel one-stage end-to-end detector specifically designed to detect small lesions in medical images. Precise localization of small lesions presents challenges due to their appearance and the diverse contextual backgrounds in which they are found. We refer to this new architecture as GravityNet, and the novel anchors as gravity points since they appear to be "attracted" by the lesions.
  arXiv  Detail & Related papers  (2023-09-22T14:02:22Z)
• Phenotype-preserving metric design for high-content image reconstruction by generative inpainting [0.0]
  We evaluate the state-of-the-art inpainting methods for image restoration in a high-content fluorescence microscopy dataset of cultured cells. We show that architectures like DeepFill V2 and Edge Connect can faithfully restore microscopy images upon fine-tuning with relatively little data.
  arXiv  Detail & Related papers  (2023-07-26T18:13:16Z)
• ROCT-Net: A new ensemble deep convolutional model with improved spatial resolution learning for detecting common diseases from retinal OCT images [0.0]
  This paper presents a new enhanced deep ensemble convolutional neural network for detecting retinal diseases from OCT images. Our model generates rich and multi-resolution features by employing the learning architectures of two robust convolutional models. Our experiments on two datasets and comparing our model with some other well-known deep convolutional neural networks have proven that our architecture can increase the classification accuracy up to 5%.
  arXiv  Detail & Related papers  (2022-03-03T17:51:01Z)
• Data-driven generation of plausible tissue geometries for realistic photoacoustic image synthesis [53.65837038435433]
  Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties. We propose a novel approach to PAT data simulation, which we refer to as "learning to simulate" We leverage the concept of Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data to generate plausible tissue geometries.
  arXiv  Detail & Related papers  (2021-03-29T11:30:18Z)
• Multi-Disease Detection in Retinal Imaging based on Ensembling Heterogeneous Deep Learning Models [0.0]
  We propose an innovative multi-disease detection pipeline for retinal imaging. Our pipeline includes state-of-the-art strategies like transfer learning, class weighting, real-time image augmentation and Focal loss utilization.
  arXiv  Detail & Related papers  (2021-03-26T18:02:17Z)
• An Interpretable Multiple-Instance Approach for the Detection of referable Diabetic Retinopathy from Fundus Images [72.94446225783697]
  We propose a machine learning system for the detection of referable Diabetic Retinopathy in fundus images. By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy. We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-03-02T13:14:15Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)
• Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models [103.64435911083432]
  We present a novel contrastive learning strategy called it Proactive Pseudo-Intervention (PPI) PPI leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability.
  arXiv  Detail & Related papers  (2020-12-06T20:30:26Z)
• Modeling and Enhancing Low-quality Retinal Fundus Images [167.02325845822276]
  Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. We propose a clinically oriented fundus enhancement network (cofe-Net) to suppress global degradation factors. Experiments on both synthetic and real images demonstrate that our algorithm effectively corrects low-quality fundus images without losing retinal details.
  arXiv  Detail & Related papers  (2020-05-12T08:01:16Z)
• Retinopathy of Prematurity Stage Diagnosis Using Object Segmentation and Convolutional Neural Networks [68.96150598294072]
  Retinopathy of Prematurity (ROP) is an eye disorder primarily affecting premature infants with lower weights. It causes proliferation of vessels in the retina and could result in vision loss and, eventually, retinal detachment, leading to blindness. In recent years, there has been a significant effort to automate the diagnosis using deep learning. This paper builds upon the success of previous models and develops a novel architecture, which combines object segmentation and convolutional neural networks (CNN) Our proposed system first trains an object segmentation model to identify the demarcation line at a pixel level and adds the resulting mask as an additional "color" channel in
  arXiv  Detail & Related papers  (2020-04-03T14:07:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.