EndoFinder: Online Lesion Retrieval for Explainable Colorectal Polyp Diagnosis Leveraging Latent Scene Representations

• URL: http://arxiv.org/abs/2507.17323v1
• Date: Wed, 23 Jul 2025 08:45:19 GMT
• Title: EndoFinder: Online Lesion Retrieval for Explainable Colorectal Polyp Diagnosis Leveraging Latent Scene Representations
• Authors: Ruijie Yang, Yan Zhu, Peiyao Fu, Yizhe Zhang, Zhihua Wang, Quanlin Li, Pinghong Zhou, Xian Yang, Shuo Wang,
• Abstract summary: Colorectal cancer (CRC) remains a leading cause of cancer-related mortality, underscoring the importance of timely polyp detection and diagnosis.<n>We propose EndoFinder, an online polyp retrieval framework that leverages multi-view scene representations for explainable and scalable CRC diagnosis.
• Score: 10.83700068295662
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Colorectal cancer (CRC) remains a leading cause of cancer-related mortality, underscoring the importance of timely polyp detection and diagnosis. While deep learning models have improved optical-assisted diagnostics, they often demand extensive labeled datasets and yield "black-box" outputs with limited interpretability. In this paper, we propose EndoFinder, an online polyp retrieval framework that leverages multi-view scene representations for explainable and scalable CRC diagnosis. First, we develop a Polyp-aware Image Encoder by combining contrastive learning and a reconstruction task, guided by polyp segmentation masks. This self-supervised approach captures robust features without relying on large-scale annotated data. Next, we treat each polyp as a three-dimensional "scene" and introduce a Scene Representation Transformer, which fuses multiple views of the polyp into a single latent representation. By discretizing this representation through a hashing layer, EndoFinder enables real-time retrieval from a compiled database of historical polyp cases, where diagnostic information serves as interpretable references for new queries. We evaluate EndoFinder on both public and newly collected polyp datasets for re-identification and pathology classification. Results show that EndoFinder outperforms existing methods in accuracy while providing transparent, retrieval-based insights for clinical decision-making. By contributing a novel dataset and a scalable, explainable framework, our work addresses key challenges in polyp diagnosis and offers a promising direction for more efficient AI-driven colonoscopy workflows. The source code is available at https://github.com/ku262/EndoFinder-Scene.

Related papers

• PolypDB: A Curated Multi-Center Dataset for Development of AI Algorithms in Colonoscopy [32.24135806984274]
  PolypDB is a large scale publicly available dataset that contains 3934 still polyp images and their corresponding ground truth from real colonoscopy videos.<n>PolypDB comprises images from five modalities: Blue Light Imaging (BLI), Flexible Imaging Color Enhancement (FICE), Linked Color Imaging (LCI), Narrow Band Imaging (NBI) and White Light Imaging (WLI) from three medical centers in Norway, Sweden, and Vietnam.
  arXiv  Detail & Related papers  (2024-08-19T05:36:01Z)
• EndoFinder: Online Image Retrieval for Explainable Colorectal Polyp Diagnosis [10.83700068295662]
  EndoFinder is a content-based image retrieval framework. It finds the 'digital twin' polyp in the reference database given a newly detected polyp. The clinical semantics of the new polyp can be inferred referring to the matched ones.
  arXiv  Detail & Related papers  (2024-07-16T05:40:17Z)
• A Survey on Deep Learning for Polyp Segmentation: Techniques, Challenges and Future Trends [39.29697088643365]
  Early detection and assessment of polyps play a crucial role in the prevention and treatment of colorectal cancer (CRC)<n>In the past, people often relied on manually extracted lower-level features such as color, texture, and shape.<n>With the advent of deep learning, more and more outstanding medical image segmentation algorithms have emerged.
  arXiv  Detail & Related papers  (2023-11-30T09:14:37Z)
• Validating polyp and instrument segmentation methods in colonoscopy through Medico 2020 and MedAI 2021 Challenges [58.32937972322058]
  "Medico automatic polyp segmentation (Medico 2020)" and "MedAI: Transparency in Medical Image (MedAI 2021)" competitions. We present a comprehensive summary and analyze each contribution, highlight the strength of the best-performing methods, and discuss the possibility of clinical translations of such methods into the clinic.
  arXiv  Detail & Related papers  (2023-07-30T16:08:45Z)
• Lesion-aware Dynamic Kernel for Polyp Segmentation [49.63274623103663]
  We propose a lesion-aware dynamic network (LDNet) for polyp segmentation. It is a traditional u-shape encoder-decoder structure incorporated with a dynamic kernel generation and updating scheme. This simple but effective scheme endows our model with powerful segmentation performance and generalization capability.
  arXiv  Detail & Related papers  (2023-01-12T09:53:57Z)
• Stepwise Feature Fusion: Local Guides Global [14.394421688712052]
  We propose a new State-Of-The-Art model for medical image segmentation, the SSFormer, which uses a pyramid Transformer encoder to improve the generalization ability of models. Our proposed Progressive Locality Decoder can be adapted to the pyramid Transformer backbone to emphasize local features and attention dispersion.
  arXiv  Detail & Related papers  (2022-03-07T10:36:38Z)
• Self-Supervised U-Net for Segmenting Flat and Sessile Polyps [63.62764375279861]
  Development of colorectal polyps is one of the earliest signs of cancer. Early detection and resection of polyps can greatly increase survival rate to 90%. Computer-Aided Diagnosis systems(CADx) has been proposed that detect polyps by processing the colonoscopic videos.
  arXiv  Detail & Related papers  (2021-10-17T09:31:20Z)
• Automatic Polyp Segmentation via Multi-scale Subtraction Network [100.94922587360871]
  In clinical practice, precise polyp segmentation provides important information in the early detection of colorectal cancer. Most existing methods are based on U-shape structure and use element-wise addition or concatenation to fuse different level features progressively in decoder. We propose a multi-scale subtraction network (MSNet) to segment polyp from colonoscopy image.
  arXiv  Detail & Related papers  (2021-08-11T07:54:07Z)
• Colorectal Polyp Classification from White-light Colonoscopy Images via Domain Alignment [57.419727894848485]
  A computer-aided diagnosis system is required to assist accurate diagnosis from colonoscopy images. Most previous studies at-tempt to develop models for polyp differentiation using Narrow-Band Imaging (NBI) or other enhanced images. We propose a novel framework based on a teacher-student architecture for the accurate colorectal polyp classification.
  arXiv  Detail & Related papers  (2021-08-05T09:31:46Z)
• PraNet: Parallel Reverse Attention Network for Polyp Segmentation [155.93344756264824]
  We propose a parallel reverse attention network (PraNet) for accurate polyp segmentation in colonoscopy images. We first aggregate the features in high-level layers using a parallel partial decoder (PPD) In addition, we mine the boundary cues using a reverse attention (RA) module, which is able to establish the relationship between areas and boundary cues.
  arXiv  Detail & Related papers  (2020-06-13T08:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.