Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks

• URL: http://arxiv.org/abs/2403.19880v1
• Date: Thu, 28 Mar 2024 23:26:45 GMT
• Title: Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks
• Authors: Pooria Ashrafian, Milad Yazdani, Moein Heidari, Dena Shahriari, Ilker Hacihaliloglu,
• Abstract summary: This paper utilizes recent vision-language models to produce diverse and realistic synthetic echocardiography image data. We show that the rich contextual information present in the synthesized data potentially enhances the accuracy and interpretability of downstream tasks.
• Score: 4.1942958779358674
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: High-quality, large-scale data is essential for robust deep learning models in medical applications, particularly ultrasound image analysis. Diffusion models facilitate high-fidelity medical image generation, reducing the costs associated with acquiring and annotating new images. This paper utilizes recent vision-language models to produce diverse and realistic synthetic echocardiography image data, preserving key features of the original images guided by textual and semantic label maps. Specifically, we investigate three potential avenues: unconditional generation, generation guided by text, and a hybrid approach incorporating both textual and semantic supervision. We show that the rich contextual information present in the synthesized data potentially enhances the accuracy and interpretability of downstream tasks, such as echocardiography segmentation and classification with improved metrics and faster convergence. Our implementation with checkpoints, prompts, and the created synthetic dataset will be publicly available at \href{https://github.com/Pooria90/DiffEcho}{GitHub}.

Related papers

• Transesophageal Echocardiography Generation using Anatomical Models [0.5679566039341877]
  We develop a pipeline to generate synthetic TEE images and corresponding semantic labels. For the pipeline's unpaired image-to-image (I2I) translation section, we explore two generative methods. We achieve a dice score improvement of up to 10% when we augment datasets with our synthetic images.
  arXiv  Detail & Related papers  (2024-10-09T11:20:28Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images [22.455833806331384]
  This paper introduces an innovative methodology for producing high-quality 3D lung CT images guided by textual information. Current state-of-the-art approaches are limited to low-resolution outputs and underutilize radiology reports' abundant information.
  arXiv  Detail & Related papers  (2023-10-05T14:16:22Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• Text-guided Foundation Model Adaptation for Pathological Image Classification [40.45252665455015]
  We propose to connect image and text Embeddings (CITE) to enhance pathological image classification. CITE injects text insights gained from language models pre-trained with a broad range of biomedical texts, leading to adapt foundation models towards pathological image understanding.
  arXiv  Detail & Related papers  (2023-07-27T14:44:56Z)
• Generalizable Synthetic Image Detection via Language-guided Contrastive Learning [22.4158195581231]
  malevolent use of synthetic images, such as the dissemination of fake news or the creation of fake profiles, raises significant concerns regarding the authenticity of images. We propose a simple yet very effective synthetic image detection method via a language-guided contrastive learning and a new formulation of the detection problem. It is shown that our proposed LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved generalizability to unseen image generation models.
  arXiv  Detail & Related papers  (2023-05-23T08:13:27Z)
• RoentGen: Vision-Language Foundation Model for Chest X-ray Generation [7.618389245539657]
  We develop a strategy to overcome the large natural-medical distributional shift by adapting a pre-trained latent diffusion model on a corpus of publicly available chest x-rays. We investigate the model's ability to generate high-fidelity, diverse synthetic CXR conditioned on text prompts. We present evidence that the resulting model (RoentGen) is able to create visually convincing, diverse synthetic CXR images.
  arXiv  Detail & Related papers  (2022-11-23T06:58:09Z)
• Re-Imagen: Retrieval-Augmented Text-to-Image Generator [58.60472701831404]
  Retrieval-Augmented Text-to-Image Generator (Re-Imagen) Retrieval-Augmented Text-to-Image Generator (Re-Imagen)
  arXiv  Detail & Related papers  (2022-09-29T00:57:28Z)
• Semantic segmentation of multispectral photoacoustic images using deep learning [53.65837038435433]
  Photoacoustic imaging has the potential to revolutionise healthcare. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. We present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images.
  arXiv  Detail & Related papers  (2021-05-20T09:33:55Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.