Latent Diffusion Models with Image-Derived Annotations for Enhanced AI-Assisted Cancer Diagnosis in Histopathology

• URL: http://arxiv.org/abs/2312.09792v1
• Date: Fri, 15 Dec 2023 13:48:55 GMT
• Title: Latent Diffusion Models with Image-Derived Annotations for Enhanced AI-Assisted Cancer Diagnosis in Histopathology
• Authors: Pedro Osorio and Guillermo Jimenez-Perez and Javier Montalt-Tordera and Jens Hooge and Guillem Duran-Ballester and Shivam Singh and Moritz Radbruch and Ute Bach and Sabrina Schroeder and Krystyna Siudak and Julia Vienenkoetter and Bettina Lawrenz and Sadegh Mohammadi
• Abstract summary: This work proposes a method that constructs structured textual prompts from automatically extracted image features. We show that including image-derived features in the prompt, as opposed to only healthy and cancerous labels, improves the Fr'echet Inception Distance (FID) from 178.8 to 90.2. We also show that pathologists find it challenging to detect synthetic images, with a median sensitivity/specificity of 0.55/0.55.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Artificial Intelligence (AI) based image analysis has an immense potential to support diagnostic histopathology, including cancer diagnostics. However, developing supervised AI methods requires large-scale annotated datasets. A potentially powerful solution is to augment training data with synthetic data. Latent diffusion models, which can generate high-quality, diverse synthetic images, are promising. However, the most common implementations rely on detailed textual descriptions, which are not generally available in this domain. This work proposes a method that constructs structured textual prompts from automatically extracted image features. We experiment with the PCam dataset, composed of tissue patches only loosely annotated as healthy or cancerous. We show that including image-derived features in the prompt, as opposed to only healthy and cancerous labels, improves the Fr\'echet Inception Distance (FID) from 178.8 to 90.2. We also show that pathologists find it challenging to detect synthetic images, with a median sensitivity/specificity of 0.55/0.55. Finally, we show that synthetic data effectively trains AI models.

Related papers

• A Sanity Check for AI-generated Image Detection [49.08585395873425]
  We present a sanity check on whether the task of AI-generated image detection has been solved. To quantify the generalization of existing methods, we evaluate 9 off-the-shelf AI-generated image detectors on Chameleon dataset. We propose AIDE (AI-generated Image DEtector with Hybrid Features), which leverages multiple experts to simultaneously extract visual artifacts and noise patterns.
  arXiv  Detail & Related papers  (2024-06-27T17:59:49Z)
• CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers [66.15847237150909]
  We introduce a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism. We validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms.
  arXiv  Detail & Related papers  (2024-03-21T15:13:36Z)
• Could We Generate Cytology Images from Histopathology Images? An Empirical Study [1.791005104399795]
  In this study, we have explored traditional image-to-image transfer models like CycleGAN, and Neural Style Transfer. In this study, we have explored traditional image-to-image transfer models like CycleGAN, and Neural Style Transfer.
  arXiv  Detail & Related papers  (2024-03-16T10:43:12Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• Augmenting medical image classifiers with synthetic data from latent diffusion models [12.077733447347592]
  We show that latent diffusion models can scalably generate images of skin disease. We generate and analyze a new dataset of 458,920 synthetic images produced using several generation strategies.
  arXiv  Detail & Related papers  (2023-08-23T22:34:49Z)
• Between Generating Noise and Generating Images: Noise in the Correct Frequency Improves the Quality of Synthetic Histopathology Images for Digital Pathology [0.0]
  Synthetic images can augment existing datasets, to improve and validate AI algorithms. We show that introducing random single-pixel noise with the appropriate spatial frequency into a semantic mask can dramatically improve the quality of the synthetic images. Our work suggests a simple and powerful approach for generating synthetic data on demand to unbias limited datasets.
  arXiv  Detail & Related papers  (2023-02-13T17:49:24Z)
• RoentGen: Vision-Language Foundation Model for Chest X-ray Generation [7.618389245539657]
  We develop a strategy to overcome the large natural-medical distributional shift by adapting a pre-trained latent diffusion model on a corpus of publicly available chest x-rays. We investigate the model's ability to generate high-fidelity, diverse synthetic CXR conditioned on text prompts. We present evidence that the resulting model (RoentGen) is able to create visually convincing, diverse synthetic CXR images.
  arXiv  Detail & Related papers  (2022-11-23T06:58:09Z)
• SYNTA: A novel approach for deep learning-based image analysis in muscle histopathology using photo-realistic synthetic data [2.1616289178832666]
  We introduce SYNTA (synthetic data) as a novel approach for the generation of synthetic, photo-realistic, and highly complex biomedical images as training data. We demonstrate that it is possible to perform robust and expert-level segmentation tasks on previously unseen real-world data, without the need for manual annotations.
  arXiv  Detail & Related papers  (2022-07-29T12:50:32Z)
• WSSS4LUAD: Grand Challenge on Weakly-supervised Tissue Semantic Segmentation for Lung Adenocarcinoma [51.50991881342181]
  This challenge includes 10,091 patch-level annotations and over 130 million labeled pixels. First place team achieved mIoU of 0.8413 (tumor: 0.8389, stroma: 0.7931, normal: 0.8919)
  arXiv  Detail & Related papers  (2022-04-13T15:27:05Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.