Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction

• URL: http://arxiv.org/abs/2607.14968v1
• Date: Thu, 16 Jul 2026 13:24:56 GMT
• Title: Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction
• Abstract summary: We propose Stitch-Inferencer, a real-time, model-agnostic inference framework that replaces implicit memory with an explicit image-space panoramic canvas.<n>By stitching valid observations across frames, Stitch-Inferencer preserves previously observed pixels in an online, instrument-free view.<n>Experiments on anatomy segmentation and point/box tracking demonstrate consistent improvements across diverse baselines.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Surgical video understanding is fundamental to navigation systems. Endoscopic perception is often hindered by a limited field-of-view and frequent instrument occlusions, making spatio-temporal context essential for robust inference. These challenges have motivated video models that aggregate information across frames. However, existing video models typically store past observations implicitly in learned feature representations, often requiring task-specific video training, substantial annotated data, and increased computational cost. We propose Stitch-Inferencer, a real-time, model-agnostic inference framework that replaces implicit feature memory with an explicit image-space panoramic canvas. By stitching valid observations across frames, Stitch-Inferencer preserves previously observed pixels in an online, instrument-free view, expanding the effective field-of-view and providing direct access to regions that are temporarily occluded or absent from the current frame. Downstream segmentation or tracking models are applied to a compact region of interest on the panorama, and their predictions are reprojected to the current frame, enabling existing models to exploit long-range context without retraining. Experiments on anatomy segmentation and point/box tracking demonstrate consistent improvements across diverse baselines while preserving real-time throughput. The stitching module alone runs at over 60 FPS, providing a practical inference-time solution to enhance endoscopic perception in computationally constrained intraoperative environments. Source code will be made publicly available.

Related papers

• Seen-to-Scene: Keep the Seen, Generate the Unseen for Video Outpainting [29.60843145809862]
  Video outpainting aims to expand the visible content of a video beyond the original frame boundaries.<n>We propose Seen-to-Scene, a novel framework that unifies propagationbased and generation-based paradigms for video outpainting.
  arXiv  Detail & Related papers  (2026-04-16T05:55:11Z)
• OmniStream: Mastering Perception, Reconstruction and Action in Continuous Streams [57.22276728212066]
  Modern visual agents require representations that are general, causal, and physically structured to operate in real-time streaming environments.<n>This paper introduces OmniStream, a unified streaming visual backbone that effectively perceives, reconstructs, and acts from diverse visual inputs.
  arXiv  Detail & Related papers  (2026-03-12T17:59:55Z)
• Learning Spatio-Temporal Feature Representations for Video-Based Gaze Estimation [50.05866669110754]
  Video-based gaze estimation methods aim to capture the inherently temporal dynamics of human eye gaze from multiple image frames.<n>We propose the Spatio-Temporal Gaze Network (ST-Gaze), a model that combines a CNN backbone with dedicated channel attention and self-attention modules.<n>We show that ST-Gaze achieves state-of-the-art performance both with and without person-specific adaptation.
  arXiv  Detail & Related papers  (2025-12-19T15:15:58Z)
• Online Long-term Point Tracking in the Foundation Model Era [0.0]
  Point tracking aims to identify the same physical point across video frames and serves as a geometry-aware representation of motion.<n>This thesis addresses the problem of long-term point tracking in an online setting, where frames are processed sequentially without access to future information.<n>We introduce Track-On, a transformer-based model that treats each tracked point as a query and processes video frames one at a time.
  arXiv  Detail & Related papers  (2025-07-12T09:24:28Z)
• Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better [61.381599921020175]
  Temporal consistency is critical in video prediction to ensure that outputs are coherent and free of artifacts.<n>Traditional methods, such as temporal attention and 3D convolution, may struggle with significant object motion.<n>We propose the Tracktention Layer, a novel architectural component that explicitly integrates motion information using point tracks.
  arXiv  Detail & Related papers  (2025-03-25T17:58:48Z)
• STOP: Integrated Spatial-Temporal Dynamic Prompting for Video Understanding [48.12128042470839]
  We propose an integrated Spatial-TempOral dynamic Prompting (STOP) model.<n>It consists of two complementary modules, the intra-frame spatial prompting and inter-frame temporal prompting.<n>STOP consistently achieves superior performance against state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-20T09:16:20Z)
• Distortion-Aware Network Pruning and Feature Reuse for Real-time Video Segmentation [49.17930380106643]
  We propose a novel framework to speed up any architecture with skip-connections for real-time vision tasks. Specifically, at the arrival of each frame, we transform the features from the previous frame to reuse them at specific spatial bins. We then perform partial computation of the backbone network on the regions of the current frame that captures temporal differences between the current and previous frame.
  arXiv  Detail & Related papers  (2022-06-20T07:20:02Z)
• FoV-Net: Field-of-View Extrapolation Using Self-Attention and Uncertainty [95.11806655550315]
  We utilize information from a video sequence with a narrow field-of-view to infer the scene at a wider field-of-view. We propose a temporally consistent field-of-view extrapolation framework, namely FoV-Net. Experiments show that FoV-Net does not only extrapolate the temporally consistent wide field-of-view scene better than existing alternatives.
  arXiv  Detail & Related papers  (2022-04-04T06:24:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.