CapCLIP: A Vision-Language Representation Alignment Approach for Wireless Capsule Endoscopy Analysis

• URL: http://arxiv.org/abs/2605.08493v1
• Date: Fri, 08 May 2026 21:14:56 GMT
• Title: CapCLIP: A Vision-Language Representation Alignment Approach for Wireless Capsule Endoscopy Analysis
• Abstract summary: Wireless capsule endoscopy (WCE) enables non-invasive visual assessment of the small bowel.<n>Existing learning-based approaches for WCE are predominantly vision-only, often confined to narrow pathology sets.<n>This study introduces CapCLIP, a domain-specific vision-language representation learning framework for WCE.
• Score: 0.4551615447454768
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Wireless capsule endoscopy (WCE) enables non-invasive visual assessment of the small bowel, but its clinical utility is constrained by the large volume of frames generated per examination and the difficulty of recognising subtle abnormalities under highly variable imaging conditions. Existing learning-based approaches for WCE are predominantly vision-only, often confined to narrow pathology sets, and show limited transfer across datasets and centres. To address these limitations, this study introduces CapCLIP, a domain-specific vision-language representation learning framework for WCE. CapCLIP aligns capsule endoscopy frames with clinically grounded textual descriptions derived from standardised nomenclature and pathology-aware caption templates, thereby learning embeddings that are both semantically informed and transferable. The proposed framework is evaluated against relevant open-source vision and vision-language foundation models under strict zero-shot conditions using unseen WCE datasets. Evaluation covers three downstream tasks: K-nearest neighbour classification, CLIP-style image-text classification, and text-to-image retrieval. Across these settings, CapCLIP consistently outperforms the compared baselines, with particularly strong gains in zero-shot image-text classification and cross-modal retrieval on out-of-distribution datasets. The results indicate that language-guided representation learning can improve both generalisation and semantic interpretability in WCE analysis. These findings position CapCLIP as a step toward foundation models tailored to capsule endoscopy and support the use of language-grounded WCE analysis.

Related papers

• Towards Clinically Faithful Medical Image Captioning via Enhanced Vision-Language Alignment [10.863931022237757]
  We propose a framework that strengthens clinical alignment by separating and enhancing training-time alignment and inference-time alignment.<n>We build a medical image captioning pipeline that integrates single/dual vision encoders based on BioMedCLIP and SigLIP2, a Q-Former, and a LLaMA-based decoder.<n>Experiments show that complementary visual representations with a multi-encoder design and concept-level auxiliary learning help preserve clinically meaningful information.
  arXiv  Detail & Related papers  (2026-08-20T09:25:30Z)
• EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment [15.371433625713232]
  EndoVLM is a novel vision-language FM pre-trained on over 348K endoscopic examinations.<n>An Anatomy-Guided Sparse Pooling mechanism utilizes textual descriptions as queries to drive sparse attention.<n>A Progressive Semantic-Aware Alignment strategy models clinical taxonomy (anatomy and pathological status) via structured soft targets.<n>A Semantic-Concentrated Masked Autoencoder is applied exclusively to these semantic-rich frames, integrating low-level visual precision with robust high-level semantic representation.
  arXiv  Detail & Related papers  (2026-08-05T05:56:41Z)
• A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports [58.14965296904734]
  EndoCLIP is a vision-language foundation model trained on 125,756 lesion-level image-text pairs.<n>It outperforms general-purpose and biomedical vision-language encoders in both zero-shot and linear-probe settings.
  arXiv  Detail & Related papers  (2026-07-30T16:24:36Z)
• SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation [5.487826115955642]
  Vision-language pre-training serves as a cornerstone for medical multimodal representation learning.<n>We propose textbfSCALPEL, a textbfSemantic textbfCross-modal textbfAlignment framework via textbfLLM-textbfPowered textbfEncoder textbfLearning.
  arXiv  Detail & Related papers  (2026-07-29T13:12:32Z)
• MedSAD-CLIP: Supervised CLIP with Token-Patch Cross-Attention for Medical Anomaly Detection and Segmentation [1.055294253870608]
  Medical anomaly detection (MAD) and segmentation play a critical role in assisting clinical diagnosis.<n>Recent CLIP-based studies are promising for anomaly detection in zero-/few-shot settings.<n>We study supervised adaptation of CLIP for MAD under a realistic clinical setting.
  arXiv  Detail & Related papers  (2026-03-18T03:37:44Z)
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation [61.350584471060756]
  Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images.<n>We propose Self-Supervised Anatomical Consistency Learning (SS-ACL) to align generated reports with corresponding anatomical regions.<n>SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy.
  arXiv  Detail & Related papers  (2025-09-30T08:59:06Z)
• CLIP-IT: CLIP-based Pairing for Histology Images Classification [14.397842743945155]
  Multimodal learning has shown promise in medical imaging, combining complementary modalities like images and text.<n>We introduce CLIP-IT, a novel framework that relies on rich unpaired text reports.<n>We show that CLIP-IT consistently improves classification accuracy over both unimodal and multimodal CLIP-based baselines.
  arXiv  Detail & Related papers  (2025-04-22T18:14:43Z)
• Visual Prompt Engineering for Vision Language Models in Radiology [0.17183214167143138]
  Contrastive Language-Image Pretraining (CLIP) offers a promising solution by enabling zero-shot classification through multimodal large-scale pretraining.<n>While CLIP effectively captures global image content, radiology requires a more localized focus on specific pathology regions to enhance both interpretability and diagnostic accuracy.<n>We explore the potential of incorporating visual cues into zero-shot classification, embedding visual markers, such as arrows, bounding boxes, and circles, directly into radiological images to guide model attention.
  arXiv  Detail & Related papers  (2024-08-28T13:53:27Z)
• An Empirical Study of CLIP for Text-based Person Search [51.94743973155648]
  Text-based Person Search (TBPS) aims to retrieve the person images using natural language descriptions. Contrastive Language Image Pretraining (CLIP), a universal large cross-modal vision-language pre-training model, has remarkably performed over various cross-modal downstream tasks. This paper makes the first attempt to conduct a comprehensive empirical study of CLIP for TBPS tasks.
  arXiv  Detail & Related papers  (2023-08-19T15:08:10Z)
• Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime [70.04389979779195]
  This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space. We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains. Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
  arXiv  Detail & Related papers  (2023-03-30T18:20:00Z)
• Non-Contrastive Learning Meets Language-Image Pre-Training [145.6671909437841]
  We study the validity of non-contrastive language-image pre-training (nCLIP) We introduce xCLIP, a multi-tasking framework combining CLIP and nCLIP, and show that nCLIP aids CLIP in enhancing feature semantics.
  arXiv  Detail & Related papers  (2022-10-17T17:57:46Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.