ReGround-Surg: Reliability-Guided Anchor Grounding for Referring Surgical Video Segmentation

• URL: http://arxiv.org/abs/2608.24671v1
• Date: Tue, 25 Aug 2026 15:08:24 GMT
• Title: ReGround-Surg: Reliability-Guided Anchor Grounding for Referring Surgical Video Segmentation
• Abstract summary: Referring surgical video segmentation requires segmenting a target instrument or tissue region across video frames according to a natural language expression.<n>Recent Segment Anything Model 2 (SAM2) based two-stage methods (e.g., ReSurgSAM2) first ground the referred target in an initial or selected frame, then propagate the selected mask via tracking.<n>We propose Re-Ground-Surg, a lightweight reliability-guided anchor grounding framework to improve SAM2-based referring surgical video segmentation.
• Score: 19.691062682670378
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Referring surgical video segmentation requires segmenting a target instrument or tissue region across video frames according to a natural language expression. Recent Segment Anything Model 2 (SAM2) based two-stage methods (e.g., ReSurgSAM2) first ground the referred target in an initial or selected frame, then propagate the selected mask via tracking. Although effective, their performance is highly sensitive to the quality of the initial grounded mask: once an incorrect anchor is selected, subsequent tracking tends to propagate the error. This issue is especially challenging in surgical videos due to visually similar instruments, occlusion, and complex tissue-tool interactions. To address this issue, we propose ReGround-Surg, a lightweight reliability-guided anchor grounding framework to improve SAM2-based referring surgical video segmentation. It first predicts a text-conditioned spatial reliability map from the referring expression and current-frame visual features. The map is then reused in two complementary branches: a Gated Side Adapter enhances expression-relevant visual regions before text-to-vision fusion, while a Reliability-Weighted Vision-to-Text Attention module suppresses off-target visual evidence during prompt-token aggregation. Experiments on Ref-EndoVis17 and Ref-EndoVis18 show consistent improvements over state-of-the-art methods across three evaluation splits with negligible speed reduction. Code is publicly available at https://github.com/JiaxinWen1/ReGround-Surg.

Related papers

• Prompting Diffusion Models for Zero-Shot Instance Segmentation [74.37083707535778]
  We present Prompt2Seg, a spatial conditioning framework for diffusion-based segmentation.<n>Our approach takes spatial prompts, represented as 2D Gaussians or confidence maps, as explicit input signals, training the model to respond directly to user intent.<n>We evaluate on seven datasets ranging from standard benchmarks to more challenging domains, including paintings, egocentric views, and X-ray data.
  arXiv  Detail & Related papers  (2026-06-21T20:30:41Z)
• Weakly-Supervised Referring Video Object Segmentation through Text Supervision [35.7765195118224]
  Referring video object segmentation (RVOS) aims to segment the target instance in a video, referred by a text expression.<n>We propose a novel weakly-supervised RVOS method, namely WSRVOS, to train the model with only text expressions.<n>Experiments on four publicly available RVOS datasets, including A2D Sentences, J-HMDB Sentences, Ref-YouTube-VOS, and Ref-DAVIS17, demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2026-04-20T04:38:45Z)
• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation [81.0871900167463]
  We introduce GeoSAM2, a prompt-controllable framework for 3D part segmentation.<n>Given a textureless object, we render normal and point maps from predefined viewpoints.<n>We accept simple 2D prompts - clicks or boxes - to guide part selection.<n>The predicted masks are back-projected to the object and aggregated across views.
  arXiv  Detail & Related papers  (2025-08-19T17:58:51Z)
• OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval [59.377821673653436]
  Composed Image Retrieval (CIR) is capable of expressing users' intricate retrieval requirements flexibly.<n>CIR remains in its nascent stages due to two limitations: 1) inhomogeneity between dominant and noisy portions in visual data is ignored, leading to query feature degradation.<n>This work presents a focus mapping-based feature extractor, which consists of two modules: dominant portion segmentation and dual focus mapping.
  arXiv  Detail & Related papers  (2025-07-08T03:27:46Z)
• Decoupled Seg Tokens Make Stronger Reasoning Video Segmenter and Grounder [5.57393627015653]
  Video segmenter and grounder approaches, exemplified by Sa2VA, directly fuse features within segmentation models.<n>This often results in an undesirable entanglement of dynamic visual information and static semantics, thereby degrading segmentation accuracy.<n>We propose DeSa2VA, a decoupling-enhanced prompting scheme integrating text pre-training and a linear decoupling module to address the information processing limitations inherent in SAM-2.
  arXiv  Detail & Related papers  (2025-06-28T13:30:36Z)
• ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations [33.74746234704817]
  Video object segmentation (RVOS) aims to segment target objects throughout a video based on a text description.<n>This is challenging as it involves deep vision-level understanding, pixel-level dense prediction andtemporal reasoning.<n>We propose bfReferDINO RVOS that inherits region-level vision-text alignment from foundational visual grounding models.
  arXiv  Detail & Related papers  (2025-01-24T16:24:15Z)
• VideoRepair: Improving Text-to-Video Generation via Misalignment Evaluation and Localized Refinement [63.4357918830628]
  VideoRepair is a model-agnostic, training-free video refinement framework.<n>It identifies fine-grained text-video misalignments and generates explicit spatial and textual feedback.<n>VideoRepair substantially outperforms recent baselines across various text-video alignment metrics.
  arXiv  Detail & Related papers  (2024-11-22T18:31:47Z)
• LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentation [14.152207010509763]
  We propose a novel LACOSTE model that exploits Location-Agnostic COntexts in Stereo and TEmporal images for improved surgical instrument segmentation. We extensively validate our approach on three public surgical video datasets.
  arXiv  Detail & Related papers  (2024-09-14T08:17:56Z)
• MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval [43.2299969152561]
  Methods for text-to-video retrieval on four datasets with both zero-shot and fine-tune evaluation protocols. Our method outperforms state-of-the-art methods for text-to-video retrieval on four datasets with both zero-shot and fine-tune evaluation protocols.
  arXiv  Detail & Related papers  (2022-04-26T16:06:31Z)
• Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation [140.4291169276062]
  Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference. Previous methods typically tackle RVOS through directly grounding linguistic reference over the image lattice. In this work, we put forward a two-stage, top-down RVOS solution. First, an exhaustive set of object tracklets is constructed by propagating object masks detected from several sampled frames to the entire video. Second, a Transformer-based tracklet-language grounding module is proposed, which models instance-level visual relations and cross-modal interactions simultaneously and efficiently.
  arXiv  Detail & Related papers  (2021-06-02T10:26:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.