Occlusion-robust Visual Markerless Bone Tracking for Computer-Assisted
Orthopaedic Surgery
- URL: http://arxiv.org/abs/2108.10608v1
- Date: Tue, 24 Aug 2021 09:49:08 GMT
- Title: Occlusion-robust Visual Markerless Bone Tracking for Computer-Assisted
Orthopaedic Surgery
- Authors: Xue Hu, Anh Nguyen, Ferdinando Rodriguez y Baena
- Abstract summary: We propose a RGB-D sensing-based markerless tracking method that is robust against occlusion.
By using a high-quality commercial RGB-D camera, our proposed visual tracking method achieves an accuracy of 1-2 degress and 2-4 mm on a model knee.
- Score: 41.681134859412246
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Conventional computer-assisted orthopaedic navigation systems rely on the
tracking of dedicated optical markers for patient poses, which makes the
surgical workflow more invasive, tedious, and expensive. Visual tracking has
recently been proposed to measure the target anatomy in a markerless and
effortless way, but the existing methods fail under real-world occlusion caused
by intraoperative interventions. Furthermore, such methods are
hardware-specific and not accurate enough for surgical applications. In this
paper, we propose a RGB-D sensing-based markerless tracking method that is
robust against occlusion. We design a new segmentation network that features
dynamic region-of-interest prediction and robust 3D point cloud segmentation.
As it is expensive to collect large-scale training data with occlusion
instances, we also propose a new method to create synthetic RGB-D images for
network training. Experimental results show that our proposed markerless
tracking method outperforms recent state-of-the-art approaches by a large
margin, especially when an occlusion exists. Furthermore, our method
generalises well to new cameras and new target models, including a cadaver,
without the need for network retraining. In practice, by using a high-quality
commercial RGB-D camera, our proposed visual tracking method achieves an
accuracy of 1-2 degress and 2-4 mm on a model knee, which meets the standard
for clinical applications.
Related papers
- DNS SLAM: Dense Neural Semantic-Informed SLAM [92.39687553022605]
DNS SLAM is a novel neural RGB-D semantic SLAM approach featuring a hybrid representation.
Our method integrates multi-view geometry constraints with image-based feature extraction to improve appearance details.
Our experimental results achieve state-of-the-art performance on both synthetic data and real-world data tracking.
arXiv Detail & Related papers (2023-11-30T21:34:44Z) - A Distance-Geometric Method for Recovering Robot Joint Angles From an
RGB Image [7.971699294672282]
We present a novel method for retrieving the joint angles of a robot manipulator using only a single RGB image of its current configuration.
Our approach, based on a distance-geometric representation of the configuration space, exploits the knowledge of a robot's kinematic model.
arXiv Detail & Related papers (2023-01-05T12:57:45Z) - Robust Landmark-based Stent Tracking in X-ray Fluoroscopy [10.917460255497227]
We propose an end-to-end deep learning framework for single stent tracking.
It consists of three hierarchical modules: U-Net based landmark detection, ResNet based stent proposal and feature extraction.
Experiments show that our method performs significantly better in detection compared with the state-of-the-art point-based tracking models.
arXiv Detail & Related papers (2022-07-20T14:20:03Z) - A Temporal Learning Approach to Inpainting Endoscopic Specularities and
Its effect on Image Correspondence [13.25903945009516]
We propose using a temporal generative adversarial network (GAN) to inpaint the hidden anatomy under specularities.
This is achieved using in-vivo data of gastric endoscopy (Hyper-Kvasir) in a fully unsupervised manner.
We also assess the effect of our method in computer vision tasks that underpin 3D reconstruction and camera motion estimation.
arXiv Detail & Related papers (2022-03-31T13:14:00Z) - HMD-EgoPose: Head-Mounted Display-Based Egocentric Marker-Less Tool and
Hand Pose Estimation for Augmented Surgical Guidance [0.0]
We present HMD-EgoPose, a single-shot learning-based approach to hand and object pose estimation.
We demonstrate state-of-the-art performance on a benchmark dataset for marker-less hand and surgical instrument pose tracking.
arXiv Detail & Related papers (2022-02-24T04:07:34Z) - Real-time landmark detection for precise endoscopic submucosal
dissection via shape-aware relation network [51.44506007844284]
We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery.
We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks.
We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
arXiv Detail & Related papers (2021-11-08T07:57:30Z) - SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based
Augmented Reality for Surgical Guidance [18.060445966264727]
SurgeonAssist-Net is a framework making action-and-workflow-driven virtual assistance accessible to commercially available optical see-through head-mounted displays (OST-HMDs)
Our implementation competes with state-of-the-art approaches in prediction accuracy for automated task recognition.
It is capable of near real-time performance on the Microsoft HoloLens 2 OST-HMD.
arXiv Detail & Related papers (2021-07-13T21:12:34Z) - Weakly-supervised Learning For Catheter Segmentation in 3D Frustum
Ultrasound [74.22397862400177]
We propose a novel Frustum ultrasound based catheter segmentation method.
The proposed method achieved the state-of-the-art performance with an efficiency of 0.25 second per volume.
arXiv Detail & Related papers (2020-10-19T13:56:22Z) - Anchor-free Small-scale Multispectral Pedestrian Detection [88.7497134369344]
We propose a method for effective and efficient multispectral fusion of the two modalities in an adapted single-stage anchor-free base architecture.
We aim at learning pedestrian representations based on object center and scale rather than direct bounding box predictions.
Results show our method's effectiveness in detecting small-scaled pedestrians.
arXiv Detail & Related papers (2020-08-19T13:13:01Z) - Deep Q-Network-Driven Catheter Segmentation in 3D US by Hybrid
Constrained Semi-Supervised Learning and Dual-UNet [74.22397862400177]
We propose a novel catheter segmentation approach, which requests fewer annotations than the supervised learning method.
Our scheme considers a deep Q learning as the pre-localization step, which avoids voxel-level annotation.
With the detected catheter, patch-based Dual-UNet is applied to segment the catheter in 3D volumetric data.
arXiv Detail & Related papers (2020-06-25T21:10:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.