AgentPolyp: Accurate Polyp Segmentation via Image Enhancement Agent

• URL: http://arxiv.org/abs/2504.10978v1
• Date: Tue, 15 Apr 2025 08:39:35 GMT
• Title: AgentPolyp: Accurate Polyp Segmentation via Image Enhancement Agent
• Authors: Pu Wang, Zhihua Zhang, Dianjie Lu, Guijuan Zhang, Youshan Zhang, Zhuoran Zheng,
• Abstract summary: AgentPolyp is a novel framework integrating CLIP-based semantic guidance and dynamic image enhancement with a lightweight neural network for segmentation.<n>The framework supports plug-and-play extensions for various enhancement algorithms and segmentation networks, meeting deployment requirements for endoscopic devices.
• Score: 29.891645824604684
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Since human and environmental factors interfere, captured polyp images usually suffer from issues such as dim lighting, blur, and overexposure, which pose challenges for downstream polyp segmentation tasks. To address the challenges of noise-induced degradation in polyp images, we present AgentPolyp, a novel framework integrating CLIP-based semantic guidance and dynamic image enhancement with a lightweight neural network for segmentation. The agent first evaluates image quality using CLIP-driven semantic analysis (e.g., identifying ``low-contrast polyps with vascular textures") and adapts reinforcement learning strategies to dynamically apply multi-modal enhancement operations (e.g., denoising, contrast adjustment). A quality assessment feedback loop optimizes pixel-level enhancement and segmentation focus in a collaborative manner, ensuring robust preprocessing before neural network segmentation. This modular architecture supports plug-and-play extensions for various enhancement algorithms and segmentation networks, meeting deployment requirements for endoscopic devices.

Related papers

• Polyp-SES: Automatic Polyp Segmentation with Self-Enriched Semantic Model [7.424888086388194]
  We propose an innovative method named Automatic Polyp with Self-Enriched Semantic Model'' to address limitations in existing approaches. First, we extract a sequence of features from an input image and decode high-level features to generate an initial segmentation mask. Using the proposed self-enriched semantic module, we query potential semantics and augment deep features with additional semantics, thereby aiding the model in understanding context more effectively.
  arXiv  Detail & Related papers  (2024-10-02T03:34:23Z)
• ECC-PolypDet: Enhanced CenterNet with Contrastive Learning for Automatic Polyp Detection [88.4359020192429]
  Existing methods either involve computationally expensive context aggregation or lack prior modeling of polyps, resulting in poor performance in challenging cases. In this paper, we propose the Enhanced CenterNet with Contrastive Learning (ECC-PolypDet), a two-stage training & end-to-end inference framework. Box-assisted Contrastive Learning (BCL) during training to minimize the intra-class difference and maximize the inter-class difference between foreground polyps and backgrounds, enabling our model to capture concealed polyps. In the fine-tuning stage, we introduce the IoU-guided Sample Re-weighting
  arXiv  Detail & Related papers  (2024-01-10T07:03:41Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training [51.16994853817024]
  This work focuses on designing an effective pre-training framework for 3D radiology images. We introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations. The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-07-31T17:59:42Z)
• Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning [9.391271552098878]
  We propose a practical yet lightweight architectural unit, Pyramid Pixel Context Adaption (PPCA) module. PPCA exploits multi-scale pixel context information to recalibrate pixel position in a pixel-independent manner. We show that PPCANet outperforms state-of-the-art attention-based networks and recent deep neural networks.
  arXiv  Detail & Related papers  (2023-03-03T13:36:55Z)
• Lesion-aware Dynamic Kernel for Polyp Segmentation [49.63274623103663]
  We propose a lesion-aware dynamic network (LDNet) for polyp segmentation. It is a traditional u-shape encoder-decoder structure incorporated with a dynamic kernel generation and updating scheme. This simple but effective scheme endows our model with powerful segmentation performance and generalization capability.
  arXiv  Detail & Related papers  (2023-01-12T09:53:57Z)
• Adaptive Context Selection for Polyp Segmentation [99.9959901908053]
  We propose an adaptive context selection based encoder-decoder framework which is composed of Local Context Attention (LCA) module, Global Context Module (GCM) and Adaptive Selection Module (ASM) LCA modules deliver local context features from encoder layers to decoder layers, enhancing the attention to the hard region which is determined by the prediction map of previous layer. GCM aims to further explore the global context features and send to the decoder layers. ASM is used for adaptive selection and aggregation of context features through channel-wise attention.
  arXiv  Detail & Related papers  (2023-01-12T04:06:44Z)
• Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers [124.01928050651466]
  We propose a new type of polyp segmentation method, named Polyp-PVT. The proposed model, named Polyp-PVT, effectively suppresses noises in the features and significantly improves their expressive capabilities.
  arXiv  Detail & Related papers  (2021-08-16T07:09:06Z)
• Enhanced U-Net: A Feature Enhancement Network for Polyp Segmentation [17.8181080354116]
  We propose a feature enhancement network for accurate polyp segmentation in colonoscopy images. Specifically, the proposed network enhances the semantic information using the novel Semantic Feature Enhance Module (SFEM) The proposed approach is evaluated on five colonoscopy datasets and demonstrates superior performance compared to other state-of-the-art models.
  arXiv  Detail & Related papers  (2021-05-03T16:46:26Z)
• Max-Fusion U-Net for Multi-Modal Pathology Segmentation with Attention and Dynamic Resampling [13.542898009730804]
  The performance of relevant algorithms is significantly affected by the proper fusion of the multi-modal information. We present the Max-Fusion U-Net that achieves improved pathology segmentation performance. We evaluate our methods using the Myocardial pathology segmentation (MyoPS) combining the multi-sequence CMR dataset.
  arXiv  Detail & Related papers  (2020-09-05T17:24:23Z)
• The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
  We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors. We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS. To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
  arXiv  Detail & Related papers  (2020-05-16T08:17:44Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.