Overcoming Imperfect Kinematics in Surgical Robotics Through Sim-to-Real Visuomotor Learning

• URL: http://arxiv.org/abs/2606.21396v1
• Date: Fri, 19 Jun 2026 13:02:35 GMT
• Title: Overcoming Imperfect Kinematics in Surgical Robotics Through Sim-to-Real Visuomotor Learning
• Abstract summary: This work uses a learning-policy to actively compensate for hardware inaccuracies using closed-loop visual feedback that was trained from a teacher-student learning framework.<n>The policy can fuse unreliable internal readings with precise external visual data, allowing it to correct for kinematic errors in real time without needing a perfect physical model.<n>This research provides a foundational and reliable control methodology, paving the way for more advanced and robust surgical automation.
• Score: 3.24203536935265
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robot-Assisted Surgery is integral to modern minimally invasive procedures, with automation emerging as the next frontier to enhance precision and reduce surgeon fatigue. This evolution is largely impeded by the inherent kinematic inaccuracies of surgical robots, where unreliable internal sensors lead to significant control errors. While previous methods attempted to mitigate these issues through complex model-based calibration, they often suffer from high cost and limited effectiveness. This work utilises a learning-policy to actively compensate for hardware inaccuracies using closed-loop visual feedback that was trained from a teacher-student learning framework. The policy can fuse unreliable internal readings with precise external visual data, allowing it to correct for kinematic errors in real time without needing a perfect physical model. The learned policy was successfully deployed on the da Vinci Research Kit, where experiments validated the fundamental feasibility of using external vision to overcome internal sensor deficits. This research provides a foundational and reliable control methodology, paving the way for more advanced and robust surgical automation.

Related papers

• Learning Context-Aware Neural ODE Dynamics for Adaptive Robotic Control [55.01272784312405]
  We propose a context-aware dynamics model based on neural ordinary differential equations.<n>We validate the approach across diverse robotic platforms, including a quadrotor in simulation, as well as a Sphero BOLT robot and a Fanuc manipulator in real-world experiments.
  arXiv  Detail & Related papers  (2026-06-13T20:55:46Z)
• A Rapid Instrument Exchange System for Humanoid Robots in Minimally Invasive Surgery [9.137620295313711]
  This paper proposes an immersive teleoperated rapid instrument exchange system.<n>The system utilizes a low-latency mechanism based on single-axis compliant docking and environmental constraint release.
  arXiv  Detail & Related papers  (2026-04-03T04:05:00Z)
• Real-time Rendering-based Surgical Instrument Tracking via Evolutionary Optimization [21.436806983625885]
  We propose a versatile tracking pipeline that jointly estimates surgical instrument pose and joint configurations.<n>The proposed framework further generalizes to joint angle-free and bi-manual tracking settings, making it suitable for both vision feedback control and online surgery video calibration.
  arXiv  Detail & Related papers  (2026-03-12T00:30:30Z)
• Give me scissors: Collision-Free Dual-Arm Surgical Assistive Robot for Instrument Delivery [4.187693745535412]
  During surgery, scrub nurses are required to frequently deliver surgical instruments to surgeons.<n>Existing research on robotic scrub nurses relies on predefined paths for instrument delivery.<n>We present a collision-free dual-arm surgical assistive robot capable of performing instrument delivery.
  arXiv  Detail & Related papers  (2026-03-03T03:14:59Z)
• Hysteresis-Aware Neural Network Modeling and Whole-Body Reinforcement Learning Control of Soft Robots [14.02771001060961]
  We present a soft robotic system designed for surgical applications.<n>We propose a whole-body neural network model that accurately captures and predicts the soft robot's whole-body motion.<n>The proposed method showed strong performance in phantom-based surgical experiments.
  arXiv  Detail & Related papers  (2025-04-18T09:34:56Z)
• Embodied Neuromorphic Control Applied on a 7-DOF Robotic Manipulator [10.642836177302533]
  Inverse dynamics is a fundamental robotics problem, which maps from joint space to torque space of robotic systems.<n>We use Spiking Neural Network to leverage continuity of the motion data to improve control accuracy and eliminate tuning parameters.<n>This work advances embodied neuromorphic control by one step forward from proof of concept to applications in complex real-world tasks.
  arXiv  Detail & Related papers  (2025-04-17T07:13:37Z)
• Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot [54.80144694888735]
  We introduce Demonstration-guided EXploration (DEX), an efficient reinforcement learning algorithm. Our method estimates expert-like behaviors with higher values to facilitate productive interactions. Experiments on $10$ surgical manipulation tasks from SurRoL, a comprehensive surgical simulation platform, demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-02-20T05:38:54Z)
• Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing [88.99939660183881]
  We propose a framework for autonomous robotic navigation for subretinal injection. Our method consists of an instrument pose estimation method, an online registration between the robotic and the i OCT system, and trajectory planning tailored for navigation to an injection target. Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method.
  arXiv  Detail & Related papers  (2023-01-17T21:41:21Z)
• Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning [121.9708998627352]
  Recent work has shown that, in practical robot learning applications, the effects of adversarial training do not pose a fair trade-off. This work revisits the robustness-accuracy trade-off in robot learning by analyzing if recent advances in robust training methods and theory can make adversarial training suitable for real-world robot applications.
  arXiv  Detail & Related papers  (2022-04-15T08:12:15Z)
• Adversarial Training is Not Ready for Robot Learning [55.493354071227174]
  Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations. We show theoretically and experimentally that neural controllers obtained via adversarial training are subjected to three types of defects. Our results suggest that adversarial training is not yet ready for robot learning.
  arXiv  Detail & Related papers  (2021-03-15T07:51:31Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.