HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space

• URL: http://arxiv.org/abs/2606.31245v1
• Date: Tue, 30 Jun 2026 07:21:06 GMT
• Title: HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space
• Abstract summary: We propose a hyperbolic surgical video-language pre-training framework that explicitly preserves the hierarchical structure.<n>Experiments on multiple surgical benchmarks show consistent gains in zero- and few-shot phase recognition.
• Score: 50.603248889968306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Surgical vision-language foundation models typically adopt educational materials, such as surgical lecture videos, to transfer surgical knowledge encoded in language into visual representations. These knowledge are multi-dimensional and hierarchical: fine-grained action cues appear in narration, mid-level key steps are summarized in subsection headings, and global procedural context, such as patient history and surgical strategy, is described in abstract texts. Prior work largely collapses these heterogeneous signals into a single flat embedding space, implicitly assuming independence across hierarchy levels. However, this is suboptimal because it ignores cross-level semantic containment, e.g., actions belong to steps, steps compose phases, weakens long-range dependency modeling. To this end, we propose a hyperbolic surgical video-language pre-training framework that explicitly preserves the hierarchical structure by mitigating structural false negatives induced by procedural context and enforcing semantic consistency between parent phases and their constituent child steps. Extensive experiments on multiple surgical benchmarks show consistent gains in zero- and few-shot phase recognition across procedures and institutions.

Related papers

• HTT-Net: Hierarchical Text-guided Transition Modeling for Surgical Video Phase Recognition [22.19338002382572]
  We propose HTT-Net, a Hierarchical Text-guided Transition modeling Network for surgical video phase recognition.<n>Key idea is to introduce structured surgical semantic knowledge into phase-aware segment construction and semantic refinement.<n>Experiments on Cholec80 and LCRS-100 demonstrate the effectiveness of HTT-Net for robust surgical video phase recognition.
  arXiv  Detail & Related papers  (2026-07-18T11:49:40Z)
• Multi-Level Narrative Evaluation Outperforms Lexical Features for Mental Health [23.379143894811566]
  Computational approaches to therapeutic writing have evolved from lexical counting to neural methods, yet remain fragmented.<n>No existing framework maps these techniques onto the hierarchical processes through which narratives are constructed.<n>This framework identifies macro-structural organization as the primary locus of clinical signal.
  arXiv  Detail & Related papers  (2026-04-30T13:31:56Z)
• Beyond Global Alignment: Fine-Grained Motion-Language Retrieval via Pyramidal Shapley-Taylor Learning [56.6025512458557]
  Motion-language retrieval aims to bridge the semantic gap between natural language and human motion.<n>Existing approaches predominantly focus on aligning entire motion sequences with global textual representations.<n>We propose a novel Pyramidal Shapley-Taylor (PST) learning framework for fine-grained motion-language retrieval.
  arXiv  Detail & Related papers  (2026-01-29T16:00:12Z)
• Where It Moves, It Matters: Referring Surgical Instrument Segmentation via Motion [54.359489807885616]
  SurgRef is a motion-guided framework that grounds free-form language expressions in instrument motion, rather than what they look like.<n>To train and evaluate SurgRef, we present Ref-IMotion, a diverse, multi-institutional video dataset with densetemporal masks and rich motion expressions.
  arXiv  Detail & Related papers  (2026-01-18T02:14:08Z)
• SurgLaVi: Large-Scale Hierarchical Dataset for Surgical Vision-Language Representation Learning [41.95743276961411]
  We present SurgLaVi, the largest and most diverse surgical vision-language dataset to date, comprising nearly 240k clip-caption pairs from more than 200 procedures.<n>At the core of SurgLaVi lies a fully automated pipeline that generates fine-grained transcriptions of surgical videos and segments them into coherent procedural units.<n>To ensure accessibility, we release SurgLaVi-beta, an open-source derivative of 113k clip-caption pairs constructed entirely from public data.
  arXiv  Detail & Related papers  (2025-09-09T21:21:10Z)
• OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining [60.75854609803651]
  OphCLIP is a hierarchical retrieval-augmented vision-language pretraining framework for ophthalmic surgical workflow understanding.<n>OphCLIP learns both fine-grained and long-term visual representations by aligning short video clips with detailed narrative descriptions and full videos with structured titles.<n>Our OphCLIP also designs a retrieval-augmented pretraining framework to leverage the underexplored large-scale silent surgical procedure videos.
  arXiv  Detail & Related papers  (2024-11-23T02:53:08Z)
• Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation [51.222684687924215]
  Surgical video-language pretraining faces unique challenges due to the knowledge domain gap and the scarcity of multi-modal data.<n>We propose a hierarchical knowledge augmentation approach and a novel Procedure-Encoded Surgical Knowledge-Augmented Video-Language Pretraining framework to tackle these issues.
  arXiv  Detail & Related papers  (2024-09-30T22:21:05Z)
• HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognition [51.222684687924215]
  HecVL is a novel hierarchical video-language pretraining approach for building a generalist surgical model.<n>By disentangling embedding spaces of different hierarchical levels, the learned multi-modal representations encode short-term and long-term surgical concepts in the same model.<n>We show that the same HecVL model for surgical phase recognition can be transferred across different surgical procedures and medical centers.
  arXiv  Detail & Related papers  (2024-05-16T13:14:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.