SLAM assisted 3D tracking system for laparoscopic surgery

• URL: http://arxiv.org/abs/2409.11688v1
• Date: Wed, 18 Sep 2024 04:00:54 GMT
• Title: SLAM assisted 3D tracking system for laparoscopic surgery
• Authors: Jingwei Song, Ray Zhang, Wenwei Zhang, Hao Zhou, Maani Ghaffari,
• Abstract summary: This work proposes a real-time monocular 3D tracking algorithm for post-registration tasks. Experiments from in-vivo and ex-vivo tests demonstrate that the proposed 3D tracking system provides robust 3D tracking.
• Score: 22.36252790404779
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A major limitation of minimally invasive surgery is the difficulty in accurately locating the internal anatomical structures of the target organ due to the lack of tactile feedback and transparency. Augmented reality (AR) offers a promising solution to overcome this challenge. Numerous studies have shown that combining learning-based and geometric methods can achieve accurate preoperative and intraoperative data registration. This work proposes a real-time monocular 3D tracking algorithm for post-registration tasks. The ORB-SLAM2 framework is adopted and modified for prior-based 3D tracking. The primitive 3D shape is used for fast initialization of the monocular SLAM. A pseudo-segmentation strategy is employed to separate the target organ from the background for tracking purposes, and the geometric prior of the 3D shape is incorporated as an additional constraint in the pose graph. Experiments from in-vivo and ex-vivo tests demonstrate that the proposed 3D tracking system provides robust 3D tracking and effectively handles typical challenges such as fast motion, out-of-field-of-view scenarios, partial visibility, and "organ-background" relative motion.

Related papers

• Advanced XR-Based 6-DOF Catheter Tracking System for Immersive Cardiac Intervention Training [37.69303106863453]
  This paper presents a novel system for real-time 3D tracking and visualization of intracardiac echocardiography (ICE) catheters. A custom 3D-printed setup captures biplane video of the catheter, while a specialized computer vision algorithm reconstructs its 3D trajectory. The system's data is integrated into an interactive Unity-based environment, rendered through the Meta Quest 3 XR headset.
  arXiv  Detail & Related papers  (2024-11-04T21:05:40Z)
• SurgTrack: CAD-Free 3D Tracking of Real-world Surgical Instruments [21.536823332387993]
  Vision-based surgical navigation has received increasing attention due to its non-invasive, cost-effective, and flexible advantages. In particular, a critical element of the vision-based navigation system is tracking surgical instruments. We propose the SurgTrack, a two-stage 3D instrument tracking method for CAD-free and robust real-world applications.
  arXiv  Detail & Related papers  (2024-09-04T10:29:59Z)
• Head Pose Estimation and 3D Neural Surface Reconstruction via Monocular Camera in situ for Navigation and Safe Insertion into Natural Openings [4.592222359553848]
  We have chosen 3D Slicer as our base platform and monocular cameras are used as sensors. We used the neural radiance fields (NeRF) algorithm to complete the 3D model reconstruction of the human head. The individual's head pose, obtained through single-camera vision, is transmitted in real-time to the scene created within 3D Slicer.
  arXiv  Detail & Related papers  (2024-06-18T20:42:09Z)
• Plug-in for visualizing 3D tool tracking from videos of Minimally Invasive Surgeries [0.6629765271909505]
  This paper tackles instrument tracking and 3D visualization challenges in minimally invasive surgery (MIS) Conventional and robot-assisted MIS encounter issues with limited 2D camera projections and minimal hardware integration. The proposed method involves 2D tracking based on segmentation maps, facilitating creation of labeled dataset without extensive ground-truth knowledge.
  arXiv  Detail & Related papers  (2024-01-12T11:04:39Z)
• MinD-3D: Reconstruct High-quality 3D objects in Human Brain [50.534007259536715]
  Recon3DMind is an innovative task aimed at reconstructing 3D visuals from Functional Magnetic Resonance Imaging (fMRI) signals. We present the fMRI-Shape dataset, which includes data from 14 participants and features 360-degree videos of 3D objects. We propose MinD-3D, a novel and effective three-stage framework specifically designed to decode the brain's 3D visual information from fMRI signals.
  arXiv  Detail & Related papers  (2023-12-12T18:21:36Z)
• Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering [5.617649111108429]
  We present DiffPose, a self-supervised approach that leverages patient-specific simulation and differentiable physics-based rendering to achieve accurate 2D/3D registration without relying on manually labeled data. DiffPose achieves sub-millimeter accuracy across surgical datasets at intraoperative speeds, improving upon existing unsupervised methods by an order of magnitude and even outperforming supervised baselines.
  arXiv  Detail & Related papers  (2023-12-11T13:05:54Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds [50.54083964183614]
  It is non-trivial to perform accurate target-specific detection since the point cloud of objects in raw LiDAR scans is usually sparse and incomplete. We propose DMT, a Detector-free Motion prediction based 3D Tracking network that totally removes the usage of complicated 3D detectors.
  arXiv  Detail & Related papers  (2022-03-08T17:49:07Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
  Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
  arXiv  Detail & Related papers  (2020-08-31T17:10:48Z)
• Kinematic 3D Object Detection in Monocular Video [123.7119180923524]
  We propose a novel method for monocular video-based 3D object detection which carefully leverages kinematic motion to improve precision of 3D localization. We achieve state-of-the-art performance on monocular 3D object detection and the Bird's Eye View tasks within the KITTI self-driving dataset.
  arXiv  Detail & Related papers  (2020-07-19T01:15:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.