HTT-Net: Hierarchical Text-guided Transition Modeling for Surgical Video Phase Recognition

• URL: http://arxiv.org/abs/2607.16787v1
• Date: Sat, 18 Jul 2026 11:49:40 GMT
• Title: HTT-Net: Hierarchical Text-guided Transition Modeling for Surgical Video Phase Recognition
• Abstract summary: We propose HTT-Net, a Hierarchical Text-guided Transition modeling Network for surgical video phase recognition.<n>Key idea is to introduce structured surgical semantic knowledge into phase-aware segment construction and semantic refinement.<n>Experiments on Cholec80 and LCRS-100 demonstrate the effectiveness of HTT-Net for robust surgical video phase recognition.
• Score: 22.19338002382572
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Surgical video phase recognition is a fundamental task in computer-assisted intervention, supporting workflow understanding, intraoperative guidance, and surgical quality assessment. Although recent visual-temporal models have achieved promising progress, accurate and temporally coherent phase recognition remains challenging due to local visual ambiguity, transient prediction noise, and insufficient use of procedural semantics. To address these challenges, we propose HTT-Net, a Hierarchical Text-guided Transition modeling Network for surgical video phase recognition. The key idea is to introduce structured surgical semantic knowledge into phase-aware segment construction and semantic refinement. Specifically, we construct a hierarchical surgical semantic memory with intra-phase descriptions, inter-phase transition descriptions, and fine-grained semantic units. Based on this memory, the proposed Transition-Aware Segment Construction (TAS-Con) organizes frame-level evidence into coherent segment representations and handles boundary clips with inter-phase transition descriptions. Furthermore, we introduce Transition-Aware Segment Calibration (TAS-Calib), which calibrates phase-aware segment representations through hierarchical surgical semantics and improves discrimination under visual ambiguity without dense frame-level vision-language fusion. Experiments on Cholec80 and LCRS-100 demonstrate the effectiveness of HTT-Net for robust surgical video phase recognition.

Related papers

• SurgNarrator: A Generative Retrieval Framework for Surgical Video Understanding [35.05885138023876]
  We propose SurgNarrator, a new generative retrieval framework tailored for surgical video understanding.<n>We construct a well-curated surgery-centric vocabulary from surgical captions to define a clinically meaningful retrieval space.<n>During inference, a hierarchical, procedure-aware retrieval strategy narrows the search space to the relevant procedure type, delivering fast and effective responses.
  arXiv  Detail & Related papers  (2026-08-05T10:39:19Z)
• Towards Voxel Spacing Consistency for Medical Image Segmentation [56.992050146446495]
  We study the correlation between image resampling and segmentation, and propose Consispace, a semantic-aware resampling framework.<n> Consispace achieves consistent voxel spacing in the axial direction while preserving anatomical and semantic consistency.
  arXiv  Detail & Related papers  (2026-06-30T15:42:23Z)
• HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space [50.603248889968306]
  We propose a hyperbolic surgical video-language pre-training framework that explicitly preserves the hierarchical structure.<n>Experiments on multiple surgical benchmarks show consistent gains in zero- and few-shot phase recognition.
  arXiv  Detail & Related papers  (2026-06-30T07:21:06Z)
• Towards Unified Surgical Scene Understanding:Bridging Reasoning and Grounding via MLLMs [32.18549469001096]
  SurgMLLM is a unified surgical scene understanding framework.<n>It bridges high-level reasoning and low-level visual grounding within a single model.<n>Extensive experiments show that SurgMLLM significantly advances surgical scene understanding.
  arXiv  Detail & Related papers  (2026-05-13T13:42:23Z)
• Where It Moves, It Matters: Referring Surgical Instrument Segmentation via Motion [54.359489807885616]
  SurgRef is a motion-guided framework that grounds free-form language expressions in instrument motion, rather than what they look like.<n>To train and evaluate SurgRef, we present Ref-IMotion, a diverse, multi-institutional video dataset with densetemporal masks and rich motion expressions.
  arXiv  Detail & Related papers  (2026-01-18T02:14:08Z)
• TGC-Net: A Structure-Aware and Semantically-Aligned Framework for Text-Guided Medical Image Segmentation [56.09179939570486]
  We propose TGC-Net, a CLIP-based framework focusing on parameter-efficient, task-specific adaptations.<n>TGC-Net achieves state-of-the-art performance with substantially fewer trainable parameters, including notable Dice gains on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-12-24T12:06:26Z)
• OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining [60.75854609803651]
  OphCLIP is a hierarchical retrieval-augmented vision-language pretraining framework for ophthalmic surgical workflow understanding.<n>OphCLIP learns both fine-grained and long-term visual representations by aligning short video clips with detailed narrative descriptions and full videos with structured titles.<n>Our OphCLIP also designs a retrieval-augmented pretraining framework to leverage the underexplored large-scale silent surgical procedure videos.
  arXiv  Detail & Related papers  (2024-11-23T02:53:08Z)
• Friends Across Time: Multi-Scale Action Segmentation Transformer for Surgical Phase Recognition [2.10407185597278]
  We propose the Multi-Scale Action Transformer (MS-AST) for offline surgical phase recognition and the Multi-Scale Action Causal Transformer (MS-ASCT) for online surgical phase recognition. Our method can achieve 95.26% and 96.15% accuracy on the Cholec80 dataset for online and offline surgical phase recognition, respectively.
  arXiv  Detail & Related papers  (2024-01-22T01:34:03Z)
• Pixel-Wise Recognition for Holistic Surgical Scene Understanding [33.40319680006502]
  This paper presents the Holistic and Multi-Granular Surgical Scene Understanding of Prostatectomies dataset.<n>Our benchmark models surgical scene understanding as a hierarchy of complementary tasks with varying levels of granularity.<n>To exploit our proposed benchmark, we introduce the Transformers for Actions, Phases, Steps, and Instrument (TAPIS) model.
  arXiv  Detail & Related papers  (2024-01-20T09:09:52Z)
• Exploiting Segment-level Semantics for Online Phase Recognition from Surgical Videos [13.858624044986811]
  We present a segment-attentive hierarchical consistency network (SAHC) for surgical phase recognition from videos. By regularizing the predictions of frames and their corresponding segments via a consistency loss, the network can generate semantic-consistent segments. Experimental results show that our method outperforms previous state-of-the-arts by a large margin.
  arXiv  Detail & Related papers  (2021-11-22T08:18:05Z)
• Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformer [57.18185972461453]
  We introduce for the first time in surgical workflow analysis Transformer to reconsider the ignored complementary effects of spatial and temporal features for accurate phase recognition. Our framework is lightweight and processes the hybrid embeddings in parallel to achieve a high inference speed.
  arXiv  Detail & Related papers  (2021-03-17T15:12:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.