EVD Surgical Guidance with Retro-Reflective Tool Tracking and Spatial Reconstruction using Head-Mounted Augmented Reality Device

• URL: http://arxiv.org/abs/2306.15490v2
• Date: Mon, 3 Jul 2023 15:42:19 GMT
• Title: EVD Surgical Guidance with Retro-Reflective Tool Tracking and Spatial Reconstruction using Head-Mounted Augmented Reality Device
• Abstract summary: Augmented Reality (AR) has been used to facilitate surgical guidance during External Ventricular Drain (EVD) surgery. Key challenge is accurately estimating the spatial relationship between pre-operative images and actual patient anatomy in AR environment. This research proposes a novel framework utilizing Time of Flight (ToF) depth sensors integrated in commercially available AR Head Mounted Devices (HMDs) for precise EVD surgical guidance.
• Score: 15.006386435500234
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Augmented Reality (AR) has been used to facilitate surgical guidance during External Ventricular Drain (EVD) surgery, reducing the risks of misplacement in manual operations. During this procedure, the key challenge is accurately estimating the spatial relationship between pre-operative images and actual patient anatomy in AR environment. This research proposes a novel framework utilizing Time of Flight (ToF) depth sensors integrated in commercially available AR Head Mounted Devices (HMD) for precise EVD surgical guidance. As previous studies have proven depth errors for ToF sensors, we first assessed their properties on AR-HMDs. Subsequently, a depth error model and patient-specific parameter identification method are introduced for accurate surface information. A tracking pipeline combining retro-reflective markers and point clouds is then proposed for accurate head tracking. The head surface is reconstructed using depth data for spatial registration, avoiding fixing tracking targets rigidly on the patient's skull. Firstly, $7.580\pm 1.488 mm$ depth value error was revealed on human skin, indicating the significance of depth correction. Our results showed that the error was reduced by over $85\%$ using proposed depth correction method on head phantoms in different materials. Meanwhile, the head surface reconstructed with corrected depth data achieved sub-millimetre accuracy. An experiment on sheep head revealed $0.79 mm$ reconstruction error. Furthermore, a user study was conducted for the performance in simulated EVD surgery, where five surgeons performed nine k-wire injections on a head phantom with virtual guidance. Results of this study revealed $2.09 \pm 0.16 mm$ translational accuracy and $2.97\pm 0.91$ degree orientational accuracy.

Related papers

• Toward surface-based registration of a virtual preoperative cutting guide onto the mandible for reconstruction surgery [8.636647142945273]
  Patient-specific cutting guides transfer a computed tomography (CT)-based plan to the operating room.<n> printed guides add cost and lead time, cannot adapt after fabrication, and may interrupt surgery if sterility is lost.<n>We investigate a markerless augmented reality (AR) alternative that registers a virtual cutting guide to the exposed mandible from surface geometry.
  arXiv  Detail & Related papers  (2026-08-06T21:15:59Z)
• Training-free Detection and 6D Pose Estimation of Unseen Surgical Instruments [45.387920812487465]
  This work introduces a training-free pipeline for accurate multi-view 6D pose estimation of unseen surgical instruments.<n>The proposed method was rigorously evaluated on real-world surgical data from the MVPSP dataset.
  arXiv  Detail & Related papers  (2026-03-26T09:28:19Z)
• EndoSfM3D: Learning to 3D Reconstruct Any Endoscopic Surgery Scene using Self-supervised Foundation Model [2.8913847481700667]
  3D reconstruction of endoscopic surgery scenes plays a vital role in enhancing scene perception, enabling AR visualization, and supporting context-aware decision-making in image-guided surgery.<n>In intrinsic calibration is hindered by sterility constraints and the use of specialized endoscopes with continuous zoom and telescope rotation.<n>In this paper, we integrate intrinsic parameter estimation into a self-supervised monocular depth estimation framework by adapting the Depth Anything V2 (DA2) model for joint depth, pose, and intrinsics prediction.<n>Our method is validated on the SCARED and C3VD public datasets, demonstrating superior performance compared to recent state-of-the
  arXiv  Detail & Related papers  (2025-10-25T16:39:04Z)
• AR Surgical Navigation with Surface Tracing: Comparing In-Situ Visualization with Tool-Tracking Guidance for Neurosurgical Applications [0.0]
  This study presents a novel methodology for utilizing AR guidance to register anatomical targets and provide real-time instrument navigation.<n>The system registers target positions to the patient through a novel surface tracing method and uses real-time infrared tool tracking to aid in catheter placement.
  arXiv  Detail & Related papers  (2025-08-14T11:46:30Z)
• Seamless Augmented Reality Integration in Arthroscopy: A Pipeline for Articular Reconstruction and Guidance [4.8046407905667206]
  arthroscopy is a minimally invasive surgical procedure used to diagnose and treat joint problems. The arthroscope's restricted field of view and lack of depth perception pose challenges in navigating complex articular structures. We present a robust pipeline that incorporates simultaneous localization and mapping, depth estimation, and 3D Gaussian splatting. Our solution offers AR assistance for articular notch measurement and annotation anchoring in a human-in-the-loop manner.
  arXiv  Detail & Related papers  (2024-10-01T04:15:49Z)
• ToDER: Towards Colonoscopy Depth Estimation and Reconstruction with Geometry Constraint Adaptation [67.22294293695255]
  We propose a novel reconstruction pipeline with a bi-directional adaptation architecture named ToDER to get precise depth estimations. Experimental results demonstrate that our approach can precisely predict depth maps in both realistic and synthetic colonoscopy videos.
  arXiv  Detail & Related papers  (2024-07-23T14:24:26Z)
• Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy [0.2995885872626565]
  We develop a novel multi-task learning (MTL) approach with a shared encoder and two decoders, namely a surface normal decoder and a depth estimator. We demonstrate an improvement of 14.17% on relative error and 10.4% on $delta_1$ accuracy over the most accurate baseline state-of-the-art BTS approach.
  arXiv  Detail & Related papers  (2023-11-30T16:13:17Z)
• W-HMR: Monocular Human Mesh Recovery in World Space with Weak-Supervised Calibration [57.37135310143126]
  Previous methods for 3D motion recovery from monocular images often fall short due to reliance on camera coordinates. We introduce W-HMR, a weak-supervised calibration method that predicts "reasonable" focal lengths based on body distortion information. We also present the OrientCorrect module, which corrects body orientation for plausible reconstructions in world space.
  arXiv  Detail & Related papers  (2023-11-29T09:02:07Z)
• Automatic registration with continuous pose updates for marker-less surgical navigation in spine surgery [52.63271687382495]
  We present an approach that automatically solves the registration problem for lumbar spinal fusion surgery in a radiation-free manner. A deep neural network was trained to segment the lumbar spine and simultaneously predict its orientation, yielding an initial pose for preoperative models. An intuitive surgical guidance is provided thanks to the integration into an augmented reality based navigation system.
  arXiv  Detail & Related papers  (2023-08-05T16:26:41Z)
• A geometry-aware deep network for depth estimation in monocular endoscopy [17.425158094539462]
  The proposed method is extensively validated across different datasets and clinical images. The generalizability of the proposed method achieves mean RMSE values of 12.604 (T1-L1), 9.930 (T2-L2), and 13.893 (colon) on the ColonDepth dataset.
  arXiv  Detail & Related papers  (2023-04-20T11:59:32Z)
• 3D endoscopic depth estimation using 3D surface-aware constraints [16.161276518580262]
  We show that depth estimation can be reformed from a 3D surface perspective. We propose a loss function for depth estimation that integrates the surface-aware constraints. Camera parameters are incorporated into the training pipeline to increase the control and transparency of the depth estimation.
  arXiv  Detail & Related papers  (2022-03-04T04:47:20Z)
• Cephalometric Landmark Regression with Convolutional Neural Networks on 3D Computed Tomography Data [68.8204255655161]
  Cephalometric analysis performed on lateral radiographs doesn't fully exploit the structure of 3D objects due to projection onto the lateral plane. We present a series of experiments with state of the art 3D convolutional neural network (CNN) based methods for keypoint regression. For the first time, we extensively evaluate the described methods and demonstrate their effectiveness in the estimation of Frankfort Horizontal and cephalometric points locations.
  arXiv  Detail & Related papers  (2020-07-20T12:45:38Z)
• Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery [58.63306322525082]
  Most applications rely on accurate real-time segmentation of high-resolution surgical images. We design a light-weight and highly-efficient deep residual architecture which is tuned to perform real-time inference of high-resolution images.
  arXiv  Detail & Related papers  (2020-07-08T21:38:29Z)
• Appearance Learning for Image-based Motion Estimation in Tomography [60.980769164955454]
  In tomographic imaging, anatomical structures are reconstructed by applying a pseudo-inverse forward model to acquired signals. Patient motion corrupts the geometry alignment in the reconstruction process resulting in motion artifacts. We propose an appearance learning approach recognizing the structures of rigid motion independently from the scanned object.
  arXiv  Detail & Related papers  (2020-06-18T09:49:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.