Robust Surgical Tool Tracking with Pixel-based Probabilities for Projected Geometric Primitives

• URL: http://arxiv.org/abs/2403.04971v1
• Date: Fri, 8 Mar 2024 00:57:03 GMT
• Title: Robust Surgical Tool Tracking with Pixel-based Probabilities for Projected Geometric Primitives
• Authors: Christopher D'Ambrosia, Florian Richter, Zih-Yun Chiu, Nikhil Shinde, Fei Liu, Henrik I. Christensen, Michael C. Yip
• Abstract summary: Controlling robotic manipulators via visual feedback requires a known coordinate frame transformation between the robot and the camera. Uncertainties in mechanical systems as well as camera calibration create errors in this coordinate frame transformation. We estimate the camera-to-base transform and joint angle measurement errors for surgical robotic tools using an image based insertion-shaft detection algorithm and probabilistic models.
• Score: 28.857732667640068
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Controlling robotic manipulators via visual feedback requires a known coordinate frame transformation between the robot and the camera. Uncertainties in mechanical systems as well as camera calibration create errors in this coordinate frame transformation. These errors result in poor localization of robotic manipulators and create a significant challenge for applications that rely on precise interactions between manipulators and the environment. In this work, we estimate the camera-to-base transform and joint angle measurement errors for surgical robotic tools using an image based insertion-shaft detection algorithm and probabilistic models. We apply our proposed approach in both a structured environment as well as an unstructured environment and measure to demonstrate the efficacy of our methods.

Related papers

• 3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects [13.58353565350936]
  We contribute methodology to jointly estimate the geometry and pose of objects grasped by a robot. Our method transforms the estimated geometry into the robot's coordinate frame. We empirically evaluate our approach on a robot manipulator holding a diverse set of real-world objects.
  arXiv  Detail & Related papers  (2024-07-14T21:02:55Z)
• VICAN: Very Efficient Calibration Algorithm for Large Camera Networks [49.17165360280794]
  We introduce a novel methodology that extends Pose Graph Optimization techniques. We consider the bipartite graph encompassing cameras, object poses evolving dynamically, and camera-object relative transformations at each time step. Our framework retains compatibility with traditional PGO solvers, but its efficacy benefits from a custom-tailored optimization scheme.
  arXiv  Detail & Related papers  (2024-03-25T17:47:03Z)
• Robot Hand-Eye Calibration using Structure-from-Motion [9.64487611393378]
  We propose a new flexible method for hand-eye calibration. We show that the solution can be obtained in linear form. We conduct a large number of experiments which validate the quality of the method by comparing it with existing ones.
  arXiv  Detail & Related papers  (2023-11-20T14:41:44Z)
• Online estimation of the hand-eye transformation from surgical scenes [11.797350284719803]
  We present a neural network-based solution that estimates the transformation from a sequence of images and kinematic data. The proposed algorithm shows that the calibration procedure can be simplified by using deep learning techniques.
  arXiv  Detail & Related papers  (2023-06-04T04:55:02Z)
• EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration [49.90228618894857]
  We introduce a new approach to hand-eye calibration called EasyHeC, which is markerless, white-box, and delivers superior accuracy and robustness. We propose to use two key technologies: differentiable rendering-based camera pose optimization and consistency-based joint space exploration. Our evaluation demonstrates superior performance in synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-05-02T03:49:54Z)
• Online Learning of Wheel Odometry Correction for Mobile Robots with Attention-based Neural Network [63.8376359764052]
  Modern robotic platforms need a reliable localization system to operate daily beside humans. Simple pose estimation algorithms based on filtered wheel and inertial odometry often fail in the presence of abrupt kinematic changes and wheel slips. We propose an innovative online learning approach for wheel odometry correction, paving the way for a robust multi-source localization system.
  arXiv  Detail & Related papers  (2023-03-21T10:30:31Z)
• A Distance-Geometric Method for Recovering Robot Joint Angles From an RGB Image [7.971699294672282]
  We present a novel method for retrieving the joint angles of a robot manipulator using only a single RGB image of its current configuration. Our approach, based on a distance-geometric representation of the configuration space, exploits the knowledge of a robot's kinematic model.
  arXiv  Detail & Related papers  (2023-01-05T12:57:45Z)
• Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment [60.31418655784291]
  We propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i.e., both kinematic and visual data, from simulator to real robot. It remedies the domain gap with enhanced transferable features by using temporal cues in videos, and inherent correlations in multi-modal towards recognizing gesture. Results show that our approach recovers the performance with great improvement gains, up to 12.91% in ACC and 20.16% in F1score without using any annotations in real robot.
  arXiv  Detail & Related papers  (2021-03-06T09:10:03Z)
• A Stitching Algorithm for Automated Surface Inspection of Rotationally Symmetric Components [0.0]
  This paper presents a novel approach to stitching surface images of rotationally symmetric parts. It uses a feature-based stitching approach to create a distortion-free and true-to-life image from a video file. The developed process enables, for example, condition monitoring without having to view many individual images.
  arXiv  Detail & Related papers  (2020-12-01T07:03:45Z)
• Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimation [89.82169646672872]
  We propose an unsupervised vision-based system to estimate the joint configurations of the robot arm from a sequence of RGB or RGB-D images without knowing the model a priori. We combine a classical geometric formulation with deep learning and extend the use of epipolar multi-rigid-body constraints to solve this task.
  arXiv  Detail & Related papers  (2020-11-30T20:46:48Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.