Polyp-Gen: Realistic and Diverse Polyp Image Generation for Endoscopic Dataset Expansion

• URL: http://arxiv.org/abs/2501.16679v2
• Date: Wed, 29 Jan 2025 16:04:56 GMT
• Title: Polyp-Gen: Realistic and Diverse Polyp Image Generation for Endoscopic Dataset Expansion
• Authors: Shengyuan Liu, Zhen Chen, Qiushi Yang, Weihao Yu, Di Dong, Jiancong Hu, Yixuan Yuan,
• Abstract summary: We present Polyp-Gen, the first full-automatic diffusion-based endoscopic image generation framework. Specifically, we devise a spatial-aware diffusion training scheme with a lesion-guided loss to enhance the structural context of polyp boundary regions. To capture medical priors for the localization of potential polyp areas, we introduce a hierarchical retrieval-based sampling strategy.
• Score: 35.74618077230043
• License:
• Abstract: Automated diagnostic systems (ADS) have shown significant potential in the early detection of polyps during endoscopic examinations, thereby reducing the incidence of colorectal cancer. However, due to high annotation costs and strict privacy concerns, acquiring high-quality endoscopic images poses a considerable challenge in the development of ADS. Despite recent advancements in generating synthetic images for dataset expansion, existing endoscopic image generation algorithms failed to accurately generate the details of polyp boundary regions and typically required medical priors to specify plausible locations and shapes of polyps, which limited the realism and diversity of the generated images. To address these limitations, we present Polyp-Gen, the first full-automatic diffusion-based endoscopic image generation framework. Specifically, we devise a spatial-aware diffusion training scheme with a lesion-guided loss to enhance the structural context of polyp boundary regions. Moreover, to capture medical priors for the localization of potential polyp areas, we introduce a hierarchical retrieval-based sampling strategy to match similar fine-grained spatial features. In this way, our Polyp-Gen can generate realistic and diverse endoscopic images for building reliable ADS. Extensive experiments demonstrate the state-of-the-art generation quality, and the synthetic images can improve the downstream polyp detection task. Additionally, our Polyp-Gen has shown remarkable zero-shot generalizability on other datasets. The source code is available at https://github.com/CUHK-AIM-Group/Polyp-Gen.

Related papers

• EndoFinder: Online Image Retrieval for Explainable Colorectal Polyp Diagnosis [10.83700068295662]
  EndoFinder is a content-based image retrieval framework. It finds the 'digital twin' polyp in the reference database given a newly detected polyp. The clinical semantics of the new polyp can be inferred referring to the matched ones.
  arXiv  Detail & Related papers  (2024-07-16T05:40:17Z)
• ECC-PolypDet: Enhanced CenterNet with Contrastive Learning for Automatic Polyp Detection [88.4359020192429]
  Existing methods either involve computationally expensive context aggregation or lack prior modeling of polyps, resulting in poor performance in challenging cases. In this paper, we propose the Enhanced CenterNet with Contrastive Learning (ECC-PolypDet), a two-stage training & end-to-end inference framework. Box-assisted Contrastive Learning (BCL) during training to minimize the intra-class difference and maximize the inter-class difference between foreground polyps and backgrounds, enabling our model to capture concealed polyps. In the fine-tuning stage, we introduce the IoU-guided Sample Re-weighting
  arXiv  Detail & Related papers  (2024-01-10T07:03:41Z)
• DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [55.48770333927732]
  We propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection. It consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion's denoising network, and a feature-space pre-trained feature extractor. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-12-11T18:38:28Z)
• RetSeg: Retention-based Colorectal Polyps Segmentation Network [0.0]
  Vision Transformers (ViTs) have revolutionized medical imaging analysis. ViTs exhibit contextual awareness in processing visual data, culminating in robust and precise predictions. We introduce RetSeg, an encoder-decoder network featuring multi-head retention blocks.
  arXiv  Detail & Related papers  (2023-10-09T06:43:38Z)
• Simple U-net Based Synthetic Polyp Image Generation: Polyp to Negative and Negative to Polyp [5.943584566266871]
  Synthetic polyp generation is a good alternative to overcome the privacy problem of medical data. We propose a deep learning-based polyp image generation framework that generates synthetic polyp images similar to real ones.
  arXiv  Detail & Related papers  (2023-02-20T08:47:44Z)
• Lesion-aware Dynamic Kernel for Polyp Segmentation [49.63274623103663]
  We propose a lesion-aware dynamic network (LDNet) for polyp segmentation. It is a traditional u-shape encoder-decoder structure incorporated with a dynamic kernel generation and updating scheme. This simple but effective scheme endows our model with powerful segmentation performance and generalization capability.
  arXiv  Detail & Related papers  (2023-01-12T09:53:57Z)
• PolypConnect: Image inpainting for generating realistic gastrointestinal tract images with polyps [1.7915968197912802]
  Early identification of a polyp in the lower gastrointestinal (GI) tract can lead to prevention of life-threatening colorectal cancer. CAD systems to detect polyps can improve detection accuracy and efficiency and save the time of the domain experts called endoscopists. We propose the PolypConnect pipeline, which can convert non-polyp images into polyp images to increase the size of training datasets for training.
  arXiv  Detail & Related papers  (2022-05-30T20:20:19Z)
• Stepwise Feature Fusion: Local Guides Global [14.394421688712052]
  We propose a new State-Of-The-Art model for medical image segmentation, the SSFormer, which uses a pyramid Transformer encoder to improve the generalization ability of models. Our proposed Progressive Locality Decoder can be adapted to the pyramid Transformer backbone to emphasize local features and attention dispersion.
  arXiv  Detail & Related papers  (2022-03-07T10:36:38Z)
• Generative Residual Attention Network for Disease Detection [51.60842580044539]
  We present a novel approach for disease generation in X-rays using a conditional generative adversarial learning. We generate a corresponding radiology image in a target domain while preserving the identity of the patient. We then use the generated X-ray image in the target domain to augment our training to improve the detection performance.
  arXiv  Detail & Related papers  (2021-10-25T14:15:57Z)
• Self-Supervised U-Net for Segmenting Flat and Sessile Polyps [63.62764375279861]
  Development of colorectal polyps is one of the earliest signs of cancer. Early detection and resection of polyps can greatly increase survival rate to 90%. Computer-Aided Diagnosis systems(CADx) has been proposed that detect polyps by processing the colonoscopic videos.
  arXiv  Detail & Related papers  (2021-10-17T09:31:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.