CataractSAM-2: A Domain-Adapted Model for Anterior Segment Surgery Segmentation and Scalable Ground-Truth Annotation

• URL: http://arxiv.org/abs/2603.21566v1
• Date: Mon, 23 Mar 2026 04:40:35 GMT
• Title: CataractSAM-2: A Domain-Adapted Model for Anterior Segment Surgery Segmentation and Scalable Ground-Truth Annotation
• Abstract summary: We present CataractSAM-2, a domain-adapted extension of Meta's Segment Anything Model 2.<n>CataractSAM-2 is designed for real-time semantic segmentation of cataract ophthalmic surgery videos with high accuracy.<n>We introduce an interactive annotation framework that combines sparse prompts with video-based mask propagation.
• Score: 2.5106929624430383
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present CataractSAM-2, a domain-adapted extension of Meta's Segment Anything Model 2, designed for real-time semantic segmentation of cataract ophthalmic surgery videos with high accuracy. Positioned at the intersection of computer vision and medical robotics, CataractSAM-2 enables precise intraoperative perception crucial for robotic-assisted and computer-guided surgical systems. Furthermore, to alleviate the burden of manual labeling, we introduce an interactive annotation framework that combines sparse prompts with video-based mask propagation. This tool significantly reduces annotation time and facilitates the scalable creation of high-quality ground-truth masks, accelerating dataset development for ocular anterior segment surgeries. We also demonstrate the model's strong zero-shot generalization to glaucoma trabeculectomy procedures, confirming its cross-procedural utility and potential for broader surgical applications. The trained model and annotation toolkit are released as open-source resources, establishing CataractSAM-2 as a foundation for expanding anterior ophthalmic surgical datasets and advancing real-time AI-driven solutions in medical robotics, as well as surgical video understanding.

Related papers

• SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy [24.007662271891277]
  We introduce surgical affordance prediction, which identifies actionable regions for fundamental surgical actions from visual data.<n>A novel adaptive feature fusion framework is proposed that leverages the complementary strengths of a self-supervised vision transformer encoder.<n>A scene-guided attention decoder is proposed to focus on critical surgical areas while suppressing background distractions.
  arXiv  Detail & Related papers  (2026-07-05T16:12:01Z)
• Data-Efficient Learning for Generalizable Surgical Video Understanding [0.0]
  This research aims to bridge gap between deep learning-based surgical video analysis in research and its real-world clinical environments.<n>I benchmarked state-of-the-art neural network architectures to identify the most effective designs for each task.<n>We developed semi-driven frameworks that improve model performance across tasks by leveraging large amounts of unlabeled surgical video.
  arXiv  Detail & Related papers  (2025-08-13T22:00:23Z)
• SurgVidLM: Towards Multi-grained Surgical Video Understanding with Large Language Model [67.8359850515282]
  SurgVidLM is the first video language model designed to address both full and fine-grained surgical video comprehension.<n>We show that SurgVidLM significantly outperforms state-of-the-art Vid-LLMs of comparable parameter scale in both full and fine-grained video understanding tasks.
  arXiv  Detail & Related papers  (2025-06-22T02:16:18Z)
• Surgical Foundation Model Leveraging Compression and Entropy Maximization for Image-Guided Surgical Assistance [50.486523249499115]
  Real-time video understanding is critical to guide procedures in minimally invasive surgery (MIS)<n>We propose Compress-to-Explore (C2E), a novel self-supervised framework to learn compact, informative representations from surgical videos.<n>C2E uses entropy-maximizing decoders to compress images while preserving clinically relevant details, improving encoder performance without labeled data.
  arXiv  Detail & Related papers  (2025-05-16T14:02:24Z)
• Landmark-Free Preoperative-to-Intraoperative Registration in Laparoscopic Liver Resection [50.388465935739376]
  Liver registration by overlaying preoperative 3D models onto intraoperative 2D frames can assist surgeons in perceiving the spatial anatomy of the liver clearly for a higher surgical success rate.<n>Existing registration methods rely heavily on anatomical landmark-based, which encounter two major limitations.<n>We propose a landmark-free preoperative-to-intraoperative registration framework utilizing effective self-supervised learning.
  arXiv  Detail & Related papers  (2025-04-21T14:55:57Z)
• EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery [52.992415247012296]
  We introduce EndoChat to address various dialogue paradigms and subtasks in surgical scene understanding.<n>Our model achieves state-of-the-art performance across five dialogue paradigms and eight surgical scene understanding tasks.
  arXiv  Detail & Related papers  (2025-01-20T09:12:06Z)
• Monocular Microscope to CT Registration using Pose Estimation of the Incus for Augmented Reality Cochlear Implant Surgery [3.8909273404657556]
  We develop a method that permits direct 2D-to-3D registration of the view microscope video to the pre-operative Computed Tomography (CT) scan without the need for external tracking equipment. Our results demonstrate the accuracy with an average rotation error of less than 25 degrees and a translation error of less than 2 mm, 3 mm, and 0.55% for the x, y, and z axes, respectively.
  arXiv  Detail & Related papers  (2024-03-12T00:26:08Z)
• Surgical Temporal Action-aware Network with Sequence Regularization for Phase Recognition [28.52533700429284]
  We propose a Surgical Temporal Action-aware Network with sequence Regularization, named STAR-Net, to recognize surgical phases more accurately from input videos. MS-STA module integrates visual features with spatial and temporal knowledge of surgical actions at the cost of 2D networks. Our STAR-Net with MS-STA and DSR can exploit visual features of surgical actions with effective regularization, thereby leading to the superior performance of surgical phase recognition.
  arXiv  Detail & Related papers  (2023-11-21T13:43:16Z)
• Phase-Specific Augmented Reality Guidance for Microscopic Cataract Surgery Using Long-Short Spatiotemporal Aggregation Transformer [14.568834378003707]
  Phaemulsification cataract surgery (PCS) is a routine procedure using a surgical microscope. PCS guidance systems extract valuable information from surgical microscopic videos to enhance proficiency. Existing PCS guidance systems suffer from non-phasespecific guidance, leading to redundant visual information. We propose a novel phase-specific augmented reality (AR) guidance system, which offers tailored AR information corresponding to the recognized surgical phase.
  arXiv  Detail & Related papers  (2023-09-11T02:56:56Z)
• GLSFormer : Gated - Long, Short Sequence Transformer for Step Recognition in Surgical Videos [57.93194315839009]
  We propose a vision transformer-based approach to learn temporal features directly from sequence-level patches. We extensively evaluate our approach on two cataract surgery video datasets, Cataract-101 and D99, and demonstrate superior performance compared to various state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-20T17:57:04Z)
• Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical Procedures [70.69948035469467]
  We take advantage of the latest computer vision methodologies for generating 3D graphs from camera views. We then introduce the Multimodal Semantic Graph Scene (MSSG) which aims at providing unified symbolic and semantic representation of surgical procedures.
  arXiv  Detail & Related papers  (2021-06-09T14:35:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.