OphBiWSSD: Scaling Temporal Action Localization in Ophthalmic Surgeries with Bidirectional Weight-tied State Space Duality

• URL: http://arxiv.org/abs/2609.12409v1
• Date: Fri, 11 Sep 2026 04:01:25 GMT
• Title: OphBiWSSD: Scaling Temporal Action Localization in Ophthalmic Surgeries with Bidirectional Weight-tied State Space Duality
• Abstract summary: We present OphBiWSSD, a framework that reformulates surgical temporal action localization.<n>Our approach facilitates the synthesis of non-causal temporal cues with linear complexity.<n>Experiments on the OphNet benchmark demonstrate that OphBiWSSD achieves state-of-the-art temporal localization performance.
• Score: 4.82182197470124
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-frequency surgical maneuvers in ophthalmology necessitate high-fidelity temporal modeling, yet characterizing long-range procedural dependencies remains computationally prohibitive for attention-based architectures. Existing models often require aggressive temporal downsampling, which compromises the detection of fine-grained action boundaries and instrument-tissue interactions. To address these scalability constraints, we present OphBiWSSD, a framework that reformulates surgical temporal action localization leveraging Bidirectional State Space Duality. By employing a weight-tied selective scan mechanism that incorporates both preceding and succeeding surgical contexts, our approach facilitates the global synthesis of non-causal temporal cues with linear complexity. This streamlined architecture is well-suited to capture the bidirectional dependencies present in ophthalmic workflows, effectively bridging the gap between local boundary precision and long-range procedural context without incurring the quadratic memory overhead of traditional Transformers. Extensive experiments on the OphNet benchmark demonstrate that OphBiWSSD achieves state-of-the-art temporal localization performance, with mean Average Precisions of 44.42% on phases and 43.08% on operations, surpassing the baselines by 6.80% and 6.66%, respectively. Empirical validation indicates that our approach ensures precise temporal localization and offers a computationally viable pathway for deploying surgical intelligence systems in clinical environments. The code is publicly available at https://github.com/yo3nglau/OphBiWSSD.

Related papers

• MedMamba: Recasting Mamba for Medical Time Series Classification [6.852021597208217]
  Medical time series, such as electrocardiograms (ECG) and electroencephalograms (EEG), exhibit complex temporal dynamics.<n>We propose MedMamba, a principle-driven multi-scale bidirectional state space architecture tailored for medical time series classification.<n>Our design is guided by three key inductive biases of physiological signals: spatial centralization, multi-timescale temporal composition, and non-causal contextual dependency.
  arXiv  Detail & Related papers  (2026-04-17T01:20:02Z)
• Holistic Surgical Phase Recognition with Hierarchical Input Dependent State Space Models [56.2236083600999]
  We propose a novel hierarchical input-dependent state space model for surgical video analysis.<n>Our framework incorporates a temporally consistent visual feature extractor, which appends a state space model head to a visual feature extractor to propagate temporal information.<n> Experiments have shown that our method outperforms the current state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2025-06-26T14:43:57Z)
• Temporal Propagation of Asymmetric Feature Pyramid for Surgical Scene Segmentation [7.150163844454341]
  Surgical scene segmentation is crucial for robot-assisted laparoscopic surgery understanding.<n>Current approaches face two challenges: (i) static image limitations and fine-grained structural details.<n>We present temporal asymmetric feature propagation network, a bidirectional attention architecture enabling cross-frame feature propagation.<n>Our framework uniquely enables both temporal guidance and contextual reasoning for surgical scene understanding.
  arXiv  Detail & Related papers  (2025-04-18T03:41:23Z)
• SPRMamba: Surgical Phase Recognition for Endoscopic Submucosal Dissection with Mamba [6.531066045206769]
  We present SPRMamba, a novel framework for real-time surgical phase recognition.<n>It integrates a Mamba architecture with a Scaled Residual TranMamba block to synergize temporal modeling and localized detail extraction.<n>It achieves state-of-the-art performance (87.64% accuracy on ESD385, +1.0% over prior methods) demonstrating robust generalizability across surgical procedures.
  arXiv  Detail & Related papers  (2024-09-18T16:26:56Z)
• SurgPLAN: Surgical Phase Localization Network for Phase Recognition [14.857715124466594]
  We propose a Surgical Phase LocAlization Network, named SurgPLAN, to facilitate a more accurate and stable surgical phase recognition. We first devise a Pyramid SlowFast (PSF) architecture to serve as the visual backbone to capture multi-scale spatial and temporal features by two branches with different frame sampling rates.
  arXiv  Detail & Related papers  (2023-11-16T15:39:01Z)
• Phase-Specific Augmented Reality Guidance for Microscopic Cataract Surgery Using Long-Short Spatiotemporal Aggregation Transformer [14.568834378003707]
  Phaemulsification cataract surgery (PCS) is a routine procedure using a surgical microscope. PCS guidance systems extract valuable information from surgical microscopic videos to enhance proficiency. Existing PCS guidance systems suffer from non-phasespecific guidance, leading to redundant visual information. We propose a novel phase-specific augmented reality (AR) guidance system, which offers tailored AR information corresponding to the recognized surgical phase.
  arXiv  Detail & Related papers  (2023-09-11T02:56:56Z)
• GLSFormer : Gated - Long, Short Sequence Transformer for Step Recognition in Surgical Videos [57.93194315839009]
  We propose a vision transformer-based approach to learn temporal features directly from sequence-level patches. We extensively evaluate our approach on two cataract surgery video datasets, Cataract-101 and D99, and demonstrate superior performance compared to various state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-20T17:57:04Z)
• LoViT: Long Video Transformer for Surgical Phase Recognition [59.06812739441785]
  We present a two-stage method, called Long Video Transformer (LoViT) for fusing short- and long-term temporal information. Our approach outperforms state-of-the-art methods on the Cholec80 and AutoLaparo datasets consistently.
  arXiv  Detail & Related papers  (2023-05-15T20:06:14Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)
• Efficient Global-Local Memory for Real-time Instrument Segmentation of Robotic Surgical Video [53.14186293442669]
  We identify two important clues for surgical instrument perception, including local temporal dependency from adjacent frames and global semantic correlation in long-range duration. We propose a novel dual-memory network (DMNet) to relate both global and local-temporal knowledge. Our method largely outperforms the state-of-the-art works on segmentation accuracy while maintaining a real-time speed.
  arXiv  Detail & Related papers  (2021-09-28T10:10:14Z)
• Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformer [57.18185972461453]
  We introduce for the first time in surgical workflow analysis Transformer to reconsider the ignored complementary effects of spatial and temporal features for accurate phase recognition. Our framework is lightweight and processes the hybrid embeddings in parallel to achieve a high inference speed.
  arXiv  Detail & Related papers  (2021-03-17T15:12:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.