MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization

• URL: http://arxiv.org/abs/2103.08105v1
• Date: Mon, 15 Mar 2021 02:53:41 GMT
• Title: MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization
• Abstract summary: We propose a new framework to more precisely estimate the pose of a rigid surgical instrument. Our strategy is based on a novel pose estimation model called MBAPose and the use of synthetic training data.
• Score: 9.925707806264613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Surgical robots are controlled using a priori models based on robots' geometric parameters, which are calibrated before the surgical procedure. One of the challenges in using robots in real surgical settings is that parameters change over time, consequently deteriorating control accuracy. In this context, our group has been investigating online calibration strategies without added sensors. In one step toward that goal, we have developed an algorithm to estimate the pose of the instruments' shafts in endoscopic images. In this study, we build upon that earlier work and propose a new framework to more precisely estimate the pose of a rigid surgical instrument. Our strategy is based on a novel pose estimation model called MBAPose and the use of synthetic training data. Our experiments demonstrated an improvement of 21 % for translation error and 26 % for orientation error on synthetic test data with respect to our previous work. Results with real test data provide a baseline for further research.

Related papers

• SurfSurg6D: Geometry Consistent Dense Correspondence for Textureless Surgical Instrument Pose Estimation [27.083882212802617]
  Surgical instrument pose estimation provides crucial information for promising applications.<n>We first construct a new dataset SynSurg6D, to alleviate the data shortage in this task.<n>We then propose SurfSurg6D, a dense-correspondence framework tailored for surgical instrument pose estimation.
  arXiv  Detail & Related papers  (2026-05-25T08:48:39Z)
• Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics [131.25998619318702]
  We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date.<n> GR00T-H is the first open foundation vision-language-action model for medical robotics.<n>We train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation.
  arXiv  Detail & Related papers  (2026-04-22T19:05:17Z)
• SurgCalib: Gaussian Splatting-Based Hand-Eye Calibration for Robot-Assisted Minimally Invasive Surgery [14.264710991558124]
  We present a Gaussian Splatting-based framework for hand-eye calibration of the da Vinci surgical robot.<n>In a vision-guided robotic system, accurate estimation of the rigid transformation between the robot base and the camera frame is essential for reliable closed-loop control.
  arXiv  Detail & Related papers  (2026-03-09T22:17:02Z)
• On-the-fly hand-eye calibration for the da Vinci surgical robot [35.18343759591773]
  We propose a calibration framework to produce accurate tool localization results through computing the hand-eye transformation matrix on-the-fly.<n>Results show a significant reduction in tool localization errors under the proposed calibration framework.
  arXiv  Detail & Related papers  (2026-01-21T10:58:28Z)
• Efficient Surgical Robotic Instrument Pose Reconstruction in Real World Conditions Using Unified Feature Detection [21.460727996614704]
  MIS robots have long kinematic chains and partial visibility of their degrees of freedom in the camera.<n>We propose a novel framework that unifies the detection of geometric primitives through a shared encoding.<n>This architecture detects both keypoints and edges in a single inference and is trained on large-scale synthetic data with projective labeling.
  arXiv  Detail & Related papers  (2025-10-03T22:03:28Z)
• Semantic Segmentation for Preoperative Planning in Transcatheter Aortic Valve Replacement [61.573750959726475]
  We consider medical guidelines for preoperative planning of the transcatheter aortic valve replacement (TAVR) and identify tasks that may be supported via semantic segmentation models.<n>We first derive fine-grained TAVR-relevant pseudo-labels from coarse-grained anatomical information, in order to train segmentation models and quantify how well they are able to find these structures in the scans.
  arXiv  Detail & Related papers  (2025-07-22T13:24:45Z)
• Reconstructing Humans with a Biomechanically Accurate Skeleton [55.06027148976482]
  We introduce a method for reconstructing 3D humans from a single image using a biomechanically accurate skeleton model. Compared to state-of-the-art methods for 3D human mesh recovery, our model achieves competitive performance on standard benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T17:56:24Z)
• SurgRIPE challenge: Benchmark of Surgical Robot Instrument Pose Estimation [32.9422323323913]
  Vision-based methods for surgical instrument pose estimation provide a practical approach to tool tracking, but they often require markers to be attached to the instruments. Recently, more research has focused on the development of marker-less methods based on deep learning. We introduce the Surgical Robot Instrument Pose Estimation (SurgRIPE) challenge, hosted at the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) in 2023. The SurgRIPE challenge has successfully established a new benchmark for the field, encouraging further research and development in surgical robot instrument pose estimation.
  arXiv  Detail & Related papers  (2025-01-06T13:02:44Z)
• Automated Surgical Skill Assessment in Endoscopic Pituitary Surgery using Real-time Instrument Tracking on a High-fidelity Bench-top Phantom [9.41936397281689]
  Improved surgical skill is generally associated with improved patient outcomes, but assessment is subjective and labour-intensive. A new public dataset is introduced, focusing on simulated surgery, using the nasal phase of endoscopic pituitary surgery as an exemplar. A Multilayer Perceptron achieved 87% accuracy in predicting surgical skill level (novice or expert), with the "ratio of total procedure time to instrument visible time" correlated with higher surgical skill.
  arXiv  Detail & Related papers  (2024-09-25T15:27:44Z)
• Enhanced Knee Kinematics: Leveraging Deep Learning and Morphing Algorithms for 3D Implant Modeling [2.752817022620644]
  This study proposes a novel approach using machine learning algorithms and morphing techniques for precise 3D reconstruction of implanted knee models. A convolutional neural network is trained to automatically segment the femur contour of the implanted components. A morphing algorithm generates a personalized 3D model of the implanted knee joint.
  arXiv  Detail & Related papers  (2024-08-02T20:11:04Z)
• Realistic Data Generation for 6D Pose Estimation of Surgical Instruments [4.226502078427161]
  6D pose estimation of surgical instruments is critical to enable the automatic execution of surgical maneuvers. In household and industrial settings, synthetic data, generated with 3D computer graphics software, has been shown as an alternative to minimize annotation costs. We propose an improved simulation environment for surgical robotics that enables the automatic generation of large and diverse datasets.
  arXiv  Detail & Related papers  (2024-06-11T14:59:29Z)
• EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration [49.90228618894857]
  We introduce a new approach to hand-eye calibration called EasyHeC, which is markerless, white-box, and delivers superior accuracy and robustness. We propose to use two key technologies: differentiable rendering-based camera pose optimization and consistency-based joint space exploration. Our evaluation demonstrates superior performance in synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-05-02T03:49:54Z)
• A parameter refinement method for Ptychography based on Deep Learning concepts [55.41644538483948]
  coarse parametrisation in propagation distance, position errors and partial coherence frequently menaces the experiment viability. A modern Deep Learning framework is used to correct autonomously the setup incoherences, thus improving the quality of a ptychography reconstruction. We tested our system on both synthetic datasets and also on real data acquired at the TwinMic beamline of the Elettra synchrotron facility.
  arXiv  Detail & Related papers  (2021-05-18T10:15:17Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery [58.63306322525082]
  Most applications rely on accurate real-time segmentation of high-resolution surgical images. We design a light-weight and highly-efficient deep residual architecture which is tuned to perform real-time inference of high-resolution images.
  arXiv  Detail & Related papers  (2020-07-08T21:38:29Z)
• Appearance Learning for Image-based Motion Estimation in Tomography [60.980769164955454]
  In tomographic imaging, anatomical structures are reconstructed by applying a pseudo-inverse forward model to acquired signals. Patient motion corrupts the geometry alignment in the reconstruction process resulting in motion artifacts. We propose an appearance learning approach recognizing the structures of rigid motion independently from the scanned object.
  arXiv  Detail & Related papers  (2020-06-18T09:49:11Z)
• Leveraging Vision and Kinematics Data to Improve Realism of Biomechanic Soft-tissue Simulation for Robotic Surgery [13.657060682152409]
  We investigate how live data acquired during any robotic endoscopic surgical procedure may be used to correct for inaccurate FEM simulation results. We use an open-source da Vinci Surgical System to probe a soft-tissue phantom and replay the interaction in simulation. We train the network to correct for the difference between the predicted mesh position and the measured point cloud.
  arXiv  Detail & Related papers  (2020-03-14T00:16:08Z)
• Single-Shot Pose Estimation of Surgical Robot Instruments' Shafts from Monocular Endoscopic Images [8.966454850713522]
  We propose a new method to estimate the pose of the surgical instruments' shafts using a monocular endoscope. Our method is based on the use of an automatically annotated training dataset and an improved pose-estimation deep-learning architecture.
  arXiv  Detail & Related papers  (2020-03-03T00:38:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.