Tracking Any Point Methods for Markerless 3D Tissue Tracking in Endoscopic Stereo Images

• URL: http://arxiv.org/abs/2508.07851v1
• Date: Mon, 11 Aug 2025 11:10:16 GMT
• Title: Tracking Any Point Methods for Markerless 3D Tissue Tracking in Endoscopic Stereo Images
• Authors: Konrad Reuter, Suresh Guttikonda, Sarah Latus, Lennart Maack, Christian Betz, Tobias Maurer, Alexander Schlaefer,
• Abstract summary: We propose a novel method for markerless 3D tissue tracking by leveraging 2D Tracking Any Point (TAP) networks.<n>Our method combines two CoTracker models, one for temporal tracking and one for stereo matching, to estimate 3D motion from stereo endoscopic images.<n>Experiments conducted on a synthetic 3D-printed phantom and a chicken tissue phantom yielded more reliable results.
• Score: 36.91957950791838
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Minimally invasive surgery presents challenges such as dynamic tissue motion and a limited field of view. Accurate tissue tracking has the potential to support surgical guidance, improve safety by helping avoid damage to sensitive structures, and enable context-aware robotic assistance during complex procedures. In this work, we propose a novel method for markerless 3D tissue tracking by leveraging 2D Tracking Any Point (TAP) networks. Our method combines two CoTracker models, one for temporal tracking and one for stereo matching, to estimate 3D motion from stereo endoscopic images. We evaluate the system using a clinical laparoscopic setup and a robotic arm simulating tissue motion, with experiments conducted on a synthetic 3D-printed phantom and a chicken tissue phantom. Tracking on the chicken tissue phantom yielded more reliable results, with Euclidean distance errors as low as 1.1 mm at a velocity of 10 mm/s. These findings highlight the potential of TAP-based models for accurate, markerless 3D tracking in challenging surgical scenarios.

Related papers

• DUSTrack: Semi-automated point tracking in ultrasound videos [0.559239450391449]
  This manuscript introduces DUSTrack, a semi-automated framework for tracking arbitrary points in B-mode ultrasound videos.<n>We combine deep learning and optical flow to deliver high-quality and robust tracking across diverse anatomical structures and motion patterns.<n>As an open-source solution, DUSTrack offers a powerful, flexible framework for point tracking to quantify tissue motion from ultrasound videos.
  arXiv  Detail & Related papers  (2025-07-18T21:22:39Z)
• MedTet: An Online Motion Model for 4D Heart Reconstruction [59.74234226055964]
  We present a novel approach to reconstruction of 3D cardiac motion from sparse intraoperative data.<n>Existing methods can accurately reconstruct 3D organ geometries from full 3D volumetric imaging.<n>We propose a versatile framework for reconstructing 3D motion from such partial data.
  arXiv  Detail & Related papers  (2024-12-03T17:18:33Z)
• SLAM assisted 3D tracking system for laparoscopic surgery [22.36252790404779]
  This work proposes a real-time monocular 3D tracking algorithm for post-registration tasks. Experiments from in-vivo and ex-vivo tests demonstrate that the proposed 3D tracking system provides robust 3D tracking.
  arXiv  Detail & Related papers  (2024-09-18T04:00:54Z)
• 3D Freehand Ultrasound using Visual Inertial and Deep Inertial Odometry for Measuring Patellar Tracking [4.252549987351643]
  Patellofemoral joint (PFJ) issues affect one in four people, with 20% experiencing chronic knee pain despite treatment. Traditional imaging methods like CT and MRI face challenges, including cost and metal artefacts. A new system to monitor joint motion could significantly improve understanding of PFJ dynamics.
  arXiv  Detail & Related papers  (2024-04-24T12:52:43Z)
• Motion-Guided Dual-Camera Tracker for Endoscope Tracking and Motion Analysis in a Mechanical Gastric Simulator [5.073179848641095]
  Motion-guided dual-camera vision tracker is proposed to provide robust and accurate tracking of the endoscope tip's 3D position.<n>The proposed tracker achieves superior performance against state-of-the-art vision trackers, achieving 42% and 72% improvements against the second-best method in average error and maximum error.
  arXiv  Detail & Related papers  (2024-03-08T08:31:46Z)
• Dense 3D Reconstruction Through Lidar: A Comparative Study on Ex-vivo Porcine Tissue [16.786601606755013]
  Researchers are actively investigating depth sensing and 3D reconstruction for vision-based surgical assistance. It remains difficult to achieve real-time, accurate, and robust 3D representations of the abdominal cavity for minimally invasive surgery. This work uses quantitative testing on fresh ex-vivo porcine tissue to thoroughly characterize the quality with which a 3D laser-based time-of-flight sensor can perform anatomical surface reconstruction.
  arXiv  Detail & Related papers  (2024-01-19T14:14:26Z)
• AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers [75.20925220246689]
  endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
  arXiv  Detail & Related papers  (2023-09-25T19:34:12Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose Estimation of Surgical Instruments [64.59698930334012]
  We present a multi-camera capture setup consisting of static and head-mounted cameras.<n>Second, we publish a multi-view RGB-D video dataset of ex-vivo spine surgeries, captured in a surgical wet lab and a real operating theatre.<n>Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments.
  arXiv  Detail & Related papers  (2023-05-05T13:42:19Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern [49.240017254888336]
  Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties. A current drawback is the limited field-of-view provided by the conventionally applied 2D probes. We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
  arXiv  Detail & Related papers  (2020-11-10T09:27:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.