Using Multi-Instance Learning to Identify Unique Polyps in Colon Capsule Endoscopy Images

• URL: http://arxiv.org/abs/2601.14771v1
• Date: Wed, 21 Jan 2026 08:50:31 GMT
• Title: Using Multi-Instance Learning to Identify Unique Polyps in Colon Capsule Endoscopy Images
• Abstract summary: This paper formulates this problem as a multi-instance learning (MIL) task, where a query polyp image is compared with a target bag of images to determine uniqueness.<n>We employ a multi-instance verification (MIV) framework that incorporates attention mechanisms, such as variance-excited multi-head attention (VEMA) and distance-based attention (DBA) to enhance the model's ability to extract meaningful representations.
• Score: 1.3825641968010818
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identifying unique polyps in colon capsule endoscopy (CCE) images is a critical yet challenging task for medical personnel due to the large volume of images, the cognitive load it creates for clinicians, and the ambiguity in labeling specific frames. This paper formulates this problem as a multi-instance learning (MIL) task, where a query polyp image is compared with a target bag of images to determine uniqueness. We employ a multi-instance verification (MIV) framework that incorporates attention mechanisms, such as variance-excited multi-head attention (VEMA) and distance-based attention (DBA), to enhance the model's ability to extract meaningful representations. Additionally, we investigate the impact of self-supervised learning using SimCLR to generate robust embeddings. Experimental results on a dataset of 1912 polyps from 754 patients demonstrate that attention mechanisms significantly improve performance, with DBA L1 achieving the highest test accuracy of 86.26\% and a test AUC of 0.928 using a ConvNeXt backbone with SimCLR pretraining. This study underscores the potential of MIL and self-supervised learning in advancing automated analysis of Colon Capsule Endoscopy images, with implications for broader medical imaging applications.

Related papers

• Context-Aware Asymmetric Ensembling for Interpretable Retinopathy of Prematurity Screening via Active Query and Vascular Attention [1.8420107091891775]
  Retinopathy of Prematurity (ROP) is among the major causes of preventable childhood blindness.<n>Current deep learning models depend heavily on large private datasets and passive multimodal fusion.<n>We propose the Context-Aware Asymmetric Ensemble Model (CAA Ensemble) that simulates clinical reasoning through two specialized streams.
  arXiv  Detail & Related papers  (2026-02-05T02:06:26Z)
• MedROV: Towards Real-Time Open-Vocabulary Detection Across Diverse Medical Imaging Modalities [89.81463562506637]
  We introduce MedROV, the first Real-time Open Vocabulary detection model for medical imaging.<n>By leveraging contrastive learning and cross-modal representations, MedROV effectively detects both known and novel structures.
  arXiv  Detail & Related papers  (2025-11-25T18:59:53Z)
• Self-supervised Representation Learning with Local Aggregation for Image-based Profiling [84.52554180480037]
  Image-based cell profiling aims to create informative representations of cell images.<n>Recent developments in non-contrastive Self-Supervised Learning have inspired this paper.<n>We introduce specialized data augmentation and representation post-processing methods tailored to cell images.
  arXiv  Detail & Related papers  (2025-06-17T07:25:57Z)
• Enhanced Anomaly Detection for Capsule Endoscopy Using Ensemble Learning Strategies [0.8824955686704116]
  This work introduces an ensemble strategy to address the challenge in anomaly detection tasks in video capsule endoscopies.<n>We propose using various loss functions, drawn from the anomaly detection field, to train each network.<n>We achieve an AUC score of 76.86% on the Kvasir-Capsule and an AUC score of 76.98% on the Galar dataset.
  arXiv  Detail & Related papers  (2025-04-08T13:39:39Z)
• A foundation model for generalizable disease diagnosis in chest X-ray images [40.9095393430871]
  We introduce CXRBase, a foundational model designed to learn versatile representations from unlabelled CXR images. CXRBase is trained on a substantial dataset of 1.04 million unlabelled CXR images. It is fine-tuned with labeled data to enhance its performance in disease detection.
  arXiv  Detail & Related papers  (2024-10-11T14:41:27Z)
• A Simple Framework Uniting Visual In-context Learning with Masked Image Modeling to Improve Ultrasound Segmentation [0.6223528900192875]
  Visual in-context learning (ICL) is a new and exciting area of research in computer vision. We propose a new simple visual ICL method called SimICL, combining visual ICL pairing images with masked image modeling (MIM) designed for self-supervised learning. SimICL achieved a remarkably high Dice coeffient (DC) of 0.96 and Jaccard Index (IoU) of 0.92, surpassing state-of-the-art segmentation and visual ICL models.
  arXiv  Detail & Related papers  (2024-02-22T05:34:22Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• Improving Vision Anomaly Detection with the Guidance of Language Modality [64.53005837237754]
  This paper tackles the challenges for vision modality from a multimodal point of view. We propose Cross-modal Guidance (CMG) to tackle the redundant information issue and sparse space issue. To learn a more compact latent space for the vision anomaly detector, CMLE learns a correlation structure matrix from the language modality.
  arXiv  Detail & Related papers  (2023-10-04T13:44:56Z)
• Learning Through Guidance: Knowledge Distillation for Endoscopic Image Classification [40.366659911178964]
  Endoscopy plays a major role in identifying any underlying abnormalities within the gastrointestinal (GI) tract. Deep learning, specifically Convolution Neural Networks (CNNs) which are designed to perform automatic feature learning without any prior feature engineering, has recently reported great benefits for GI endoscopy image analysis. We investigate three KD-based learning frameworks, response-based, feature-based, and relation-based mechanisms, and introduce a novel multi-head attention-based feature fusion mechanism to support relation-based learning.
  arXiv  Detail & Related papers  (2023-08-17T02:02:11Z)
• Longitudinal Multimodal Transformer Integrating Imaging and Latent Clinical Signatures From Routine EHRs for Pulmonary Nodule Classification [4.002181247287472]
  We propose a transformer-based multimodal strategy to integrate repeat imaging with longitudinal clinical signatures from EHRs for solitary pulmonary nodule (SPN) classification. We perform unsupervised disentanglement of latent clinical signatures and leverage time-distance scaled self-attention to jointly learn from clinical signatures expressions and chest computed tomography (CT) scans.
  arXiv  Detail & Related papers  (2023-04-06T03:03:07Z)
• SEMPAI: a Self-Enhancing Multi-Photon Artificial Intelligence for prior-informed assessment of muscle function and pathology [48.54269377408277]
  We introduce the Self-Enhancing Multi-Photon Artificial Intelligence (SEMPAI), that integrates hypothesis-driven priors in a data-driven Deep Learning approach. SEMPAI performs joint learning of several tasks to enable prediction for small datasets. SEMPAI outperforms state-of-the-art biomarkers in six of seven predictive tasks, including those with scarce data.
  arXiv  Detail & Related papers  (2022-10-28T17:03:04Z)
• Intelligent Masking: Deep Q-Learning for Context Encoding in Medical Image Analysis [48.02011627390706]
  We develop a novel self-supervised approach that occludes targeted regions to improve the pre-training procedure. We show that training the agent against the prediction model can significantly improve the semantic features extracted for downstream classification tasks.
  arXiv  Detail & Related papers  (2022-03-25T19:05:06Z)
• Multi-Modal Multi-Instance Learning for Retinal Disease Recognition [10.294738095942812]
  We aim to build a deep neural network that recognizes multiple vision-threatening diseases for the given case. As both data acquisition and manual labeling are extremely expensive in the medical domain, the network has to be relatively lightweight.
  arXiv  Detail & Related papers  (2021-09-25T08:16:47Z)
• Colorectal Polyp Classification from White-light Colonoscopy Images via Domain Alignment [57.419727894848485]
  A computer-aided diagnosis system is required to assist accurate diagnosis from colonoscopy images. Most previous studies at-tempt to develop models for polyp differentiation using Narrow-Band Imaging (NBI) or other enhanced images. We propose a novel framework based on a teacher-student architecture for the accurate colorectal polyp classification.
  arXiv  Detail & Related papers  (2021-08-05T09:31:46Z)
• Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks [38.39901070720532]
  Current image-based survival models that limit to key patches or clusters derived from Whole Slide Images (WSIs) We propose Deep Attention Multiple Instance Survival Learning (DeepAttnMISL) by introducing both siamese MI-FCN and attention-based MIL pooling. We evaluated our methods on two large cancer whole slide images datasets and our results suggest that the proposed approach is more effective and suitable for large datasets.
  arXiv  Detail & Related papers  (2020-09-23T14:31:15Z)
• Additive Angular Margin for Few Shot Learning to Classify Clinical Endoscopy Images [42.74958357195011]
  We propose a few-shot learning approach that requires less training data and can be used to predict label classes of test samples from an unseen dataset. We compare our approach to the several established methods on a large cohort of multi-center, multi-organ, and multi-modal endoscopy data.
  arXiv  Detail & Related papers  (2020-03-23T00:20:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.