S3-Tracker: Self-Supervised Surgical Tissue Tracking With Contrastive Random Walks

• URL: http://arxiv.org/abs/2609.14313v1
• Date: Sun, 13 Sep 2026 06:17:40 GMT
• Title: S3-Tracker: Self-Supervised Surgical Tissue Tracking With Contrastive Random Walks
• Abstract summary: We propose a self-supervised Track-Any-Point approach that learns from unlabeled surgical videos by establishing global pixel correspondences and inferring point trajectories through contrastive random walks.<n> Trained without annotations, our method achieves performance comparable to existing semi-supervised approaches while implicitly handling tissue deformation.
• Score: 7.730246056950143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robust point tracking in endoscopic videos is essential for computer-assisted intervention and autonomous robotic surgery, enabling continuous registration between intraoperative video and preoperative imaging despite soft tissue deformation. However, supervised tracking methods depend on large annotated datasets, while surgical conditions make reliable trajectory annotation challenging. We propose a self-supervised Track-Any-Point approach that learns from unlabeled surgical videos by establishing global pixel correspondences and inferring point trajectories through contrastive random walks. Trained without annotations, our method achieves performance comparable to existing semi-supervised approaches while implicitly handling tissue deformation. These findings demonstrate the feasibility of self-supervised point tracking in surgical environments and its potential to reduce reliance on annotated data.

Related papers

• Training-free Temporal Object Tracking in Surgical Videos [20.689670966095097]
  We present a novel approach for online object tracking in laparoscopic cholecystectomy (LC) surgical videos.<n>Our method addresses the challenges of costly pixel-level annotations and label inconsistencies inherent in existing datasets.
  arXiv  Detail & Related papers  (2026-03-08T23:09:16Z)
• SurgAtt-Tracker: Online Surgical Attention Tracking via Temporal Proposal Reranking and Motion-Aware Refinement [45.37105164372227]
  SurgAtt-Tracker is a holistic framework that robustly tracks surgical attention.<n>Experiments on multiple surgical datasets demonstrate that SurgAtt-Tracker achieves consistently state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-02-24T07:30:51Z)
• When Tracking Fails: Analyzing Failure Modes of SAM2 for Point-Based Tracking in Surgical Videos [0.43981305860983705]
  We systematically analyze the failure modes of point-based tracking in laparoscopic cholecystectomy videos.<n>Our results show that point-based tracking is competitive for surgical tools but consistently underperforms for anatomical targets.
  arXiv  Detail & Related papers  (2025-10-02T15:06:49Z)
• Monocular Marker-free Patient-to-Image Intraoperative Registration for Cochlear Implant Surgery [4.250558597144547]
  Our framework seamlessly integrates with monocular surgical microscopes, making it highly practical for clinical use without additional hardware dependencies and requirements.<n>Our results suggest that our approach achieves clinically relevant accuracy in predicting 6D camera poses for registering 3D preoperative CT scans to 2D surgical scenes with an angular error within 10 degrees in most cases.
  arXiv  Detail & Related papers  (2025-05-23T21:15:00Z)
• Tracking Everything in Robotic-Assisted Surgery [39.62251870446397]
  We present an annotated surgical tracking dataset for benchmarking tracking methods for surgical scenarios.<n>We evaluate state-of-the-art (SOTA) TAP-based algorithms on this dataset and reveal their limitations in challenging surgical scenarios.<n>We propose a new tracking method, namely SurgMotion, to solve the challenges and further improve the tracking performance.
  arXiv  Detail & Related papers  (2024-09-29T23:06:57Z)
• Real-time guidewire tracking and segmentation in intraoperative x-ray [52.51797358201872]
  We propose a two-stage deep learning framework for real-time guidewire segmentation and tracking. In the first stage, a Yolov5 detector is trained, using the original X-ray images as well as synthetic ones, to output the bounding boxes of possible target guidewires. In the second stage, a novel and efficient network is proposed to segment the guidewire in each detected bounding box.
  arXiv  Detail & Related papers  (2024-04-12T20:39:19Z)
• Pseudo-label Guided Cross-video Pixel Contrast for Robotic Surgical Scene Segmentation with Limited Annotations [72.15956198507281]
  We propose PGV-CL, a novel pseudo-label guided cross-video contrast learning method to boost scene segmentation. We extensively evaluate our method on a public robotic surgery dataset EndoVis18 and a public cataract dataset CaDIS.
  arXiv  Detail & Related papers  (2022-07-20T05:42:19Z)
• Crop-Transform-Paste: Self-Supervised Learning for Visual Tracking [137.26381337333552]
  In this work, we develop the Crop-Transform-Paste operation, which is able to synthesize sufficient training data. Since the object state is known in all synthesized data, existing deep trackers can be trained in routine ways without human annotation.
  arXiv  Detail & Related papers  (2021-06-21T07:40:34Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)
• Towards Unsupervised Learning for Instrument Segmentation in Robotic Surgery with Cycle-Consistent Adversarial Networks [54.00217496410142]
  We propose an unpaired image-to-image translation where the goal is to learn the mapping between an input endoscopic image and a corresponding annotation. Our approach allows to train image segmentation models without the need to acquire expensive annotations. We test our proposed method on Endovis 2017 challenge dataset and show that it is competitive with supervised segmentation methods.
  arXiv  Detail & Related papers  (2020-07-09T01:39:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.