Online estimation of the hand-eye transformation from surgical scenes
- URL: http://arxiv.org/abs/2306.02261v1
- Date: Sun, 4 Jun 2023 04:55:02 GMT
- Title: Online estimation of the hand-eye transformation from surgical scenes
- Authors: Krittin Pachtrachai, Francisco Vasconcelos, and Danail Stoyanov
- Abstract summary: We present a neural network-based solution that estimates the transformation from a sequence of images and kinematic data.
The proposed algorithm shows that the calibration procedure can be simplified by using deep learning techniques.
- Score: 11.797350284719803
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Hand-eye calibration algorithms are mature and provide accurate
transformation estimations for an effective camera-robot link but rely on a
sufficiently wide range of calibration data to avoid errors and degenerate
configurations. To solve the hand-eye problem in robotic-assisted minimally
invasive surgery and also simplify the calibration procedure by using neural
network method cooporating with the new objective function. We present a neural
network-based solution that estimates the transformation from a sequence of
images and kinematic data which significantly simplifies the calibration
procedure. The network utilises the long short-term memory architecture to
extract temporal information from the data and solve the hand-eye problem. The
objective function is derived from the linear combination of remote centre of
motion constraint, the re-projection error and its derivative to induce a small
change in the hand-eye transformation. The method is validated with the data
from da Vinci Si and the result shows that the estimated hand-eye matrix is
able to re-project the end-effector from the robot coordinate to the camera
coordinate within 10 to 20 pixels of accuracy in both testing dataset. The
calibration performance is also superior to the previous neural network-based
hand-eye method. The proposed algorithm shows that the calibration procedure
can be simplified by using deep learning techniques and the performance is
improved by the assumption of non-static hand-eye transformations.
Related papers
- Kalib: Markerless Hand-Eye Calibration with Keypoint Tracking [52.4190876409222]
Hand-eye calibration involves estimating the transformation between the camera and the robot.
Recent advancements in deep learning offer markerless techniques, but they present challenges.
We propose Kalib, an automatic and universal markerless hand-eye calibration pipeline.
arXiv Detail & Related papers (2024-08-20T06:03:40Z) - Robust Surgical Tool Tracking with Pixel-based Probabilities for
Projected Geometric Primitives [28.857732667640068]
Controlling robotic manipulators via visual feedback requires a known coordinate frame transformation between the robot and the camera.
Uncertainties in mechanical systems as well as camera calibration create errors in this coordinate frame transformation.
We estimate the camera-to-base transform and joint angle measurement errors for surgical robotic tools using an image based insertion-shaft detection algorithm and probabilistic models.
arXiv Detail & Related papers (2024-03-08T00:57:03Z) - EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable
Rendering and Space Exploration [49.90228618894857]
We introduce a new approach to hand-eye calibration called EasyHeC, which is markerless, white-box, and delivers superior accuracy and robustness.
We propose to use two key technologies: differentiable rendering-based camera pose optimization and consistency-based joint space exploration.
Our evaluation demonstrates superior performance in synthetic and real-world datasets.
arXiv Detail & Related papers (2023-05-02T03:49:54Z) - A Distance-Geometric Method for Recovering Robot Joint Angles From an
RGB Image [7.971699294672282]
We present a novel method for retrieving the joint angles of a robot manipulator using only a single RGB image of its current configuration.
Our approach, based on a distance-geometric representation of the configuration space, exploits the knowledge of a robot's kinematic model.
arXiv Detail & Related papers (2023-01-05T12:57:45Z) - Differentiable Uncalibrated Imaging [25.67247660827913]
We propose a differentiable imaging framework to address uncertainty in measurement coordinates such as sensor locations and projection angles.
We apply implicit neural networks, also known as neural fields, which are naturally differentiable with respect to the input coordinates.
Differentiability is key as it allows us to jointly fit a measurement representation, optimize over the uncertain measurement coordinates, and perform image reconstruction which in turn ensures consistent calibration.
arXiv Detail & Related papers (2022-11-18T22:48:09Z) - A parameter refinement method for Ptychography based on Deep Learning
concepts [55.41644538483948]
coarse parametrisation in propagation distance, position errors and partial coherence frequently menaces the experiment viability.
A modern Deep Learning framework is used to correct autonomously the setup incoherences, thus improving the quality of a ptychography reconstruction.
We tested our system on both synthetic datasets and also on real data acquired at the TwinMic beamline of the Elettra synchrotron facility.
arXiv Detail & Related papers (2021-05-18T10:15:17Z) - Searching for Efficient Architecture for Instrument Segmentation in
Robotic Surgery [58.63306322525082]
Most applications rely on accurate real-time segmentation of high-resolution surgical images.
We design a light-weight and highly-efficient deep residual architecture which is tuned to perform real-time inference of high-resolution images.
arXiv Detail & Related papers (2020-07-08T21:38:29Z) - Intra Order-preserving Functions for Calibration of Multi-Class Neural
Networks [54.23874144090228]
A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores.
Previous post-hoc calibration techniques work only with simple calibration functions.
We propose a new neural network architecture that represents a class of intra order-preserving functions.
arXiv Detail & Related papers (2020-03-15T12:57:21Z) - Calibrating Deep Neural Networks using Focal Loss [77.92765139898906]
Miscalibration is a mismatch between a model's confidence and its correctness.
We show that focal loss allows us to learn models that are already very well calibrated.
We show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.
arXiv Detail & Related papers (2020-02-21T17:35:50Z) - Vanishing Point Detection with Direct and Transposed Fast Hough
Transform inside the neural network [0.0]
In this paper, we suggest a new neural network architecture for vanishing point detection in images.
The key element is the use of the direct and transposed Fast Hough Transforms separated by convolutional layer blocks with standard activation functions.
arXiv Detail & Related papers (2020-02-04T09:10:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.