Hybrid Offline-Online Reinforcement Learning for Sensorless, High-Precision Force Regulation in Surgical Robotic Grasping

• URL: http://arxiv.org/abs/2602.23870v1
• Date: Fri, 27 Feb 2026 10:11:59 GMT
• Title: Hybrid Offline-Online Reinforcement Learning for Sensorless, High-Precision Force Regulation in Surgical Robotic Grasping
• Abstract summary: We present a sensorless control framework that combines physics-consistent modeling and hybrid reinforcement learning.<n>We develop a first-principles digital twin of the da Vinci Xi grasping mechanism that captures coupled electrical, transmission, and jaw dynamics.<n>In simulation, the controller maintains grasp force within 1% of the desired reference during multi-harmonic jaw motion.
• Score: 2.874057693956189
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Precise grasp force regulation in tendon-driven surgical instruments is fundamentally limited by nonlinear coupling between motor dynamics, transmission compliance, friction, and distal mechanics. Existing solutions typically rely on distal force sensing or analytical compensation, increasing hardware complexity or degrading performance under dynamic motion. We present a sensorless control framework that combines physics-consistent modeling and hybrid reinforcement learning to achieve high-precision distal force regulation in a proximally actuated surgical end-effector. We develop a first-principles digital twin of the da Vinci Xi grasping mechanism that captures coupled electrical, transmission, and jaw dynamics within a unified differential-algebraic formulation. To safely learn control policies in this stiff and highly nonlinear system, we introduce a three-stage pipeline:(i)a receding-horizon CMA-ES oracle that generates dynamically feasible expert trajectories,(ii)fully offline policy learning via Implicit Q-Learning to ensure stable initialization without unsafe exploration, and (iii)online refinement using TD3 for adaptation to on-policy dynamics. The resulting policy directly maps proximal measurements to motor voltages and requires no distal sensing. In simulation, the controller maintains grasp force within 1% of the desired reference during multi-harmonic jaw motion. Hardware experiments demonstrate average force errors below 4% across diverse trajectories, validating sim-to-real transfer. The learned policy contains approximately 71k param and executes at kH rates, enabling real-time deployment. These results demonstrate that high-fidelity modeling combined with structured offline-online RL can recover precise distal force behavior without additional sensing, offering a scalable and mechanically compatible solution for surgical robotic manipulation.

Related papers

• A Neural Network Based Teleoperation for Remote Controlled Vehicles [98.38733381483934]
  Direct teleoperation of vehicles faces critical technical bottlenecks.<n> communication latency and the operator's inability to physically perceive unmodeled environmental disturbances.<n>We propose a tailored unilateral teleoperation framework.
  arXiv  Detail & Related papers  (2026-08-11T01:51:08Z)
• Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation [17.63466252699545]
  Human-like dexterous hands offer human-level manipulation capabilities but remain difficult to train the control policies that can deploy on real hardware.<n>We present a sim-to-real reinforcement learning method that leverages dense tactile feedback combined with joint torque sensing.<n>This is the first demonstration of controllable grasping on a multi-finger dexterous hand trained entirely in simulation and transferred zero-shot on real hardware.
  arXiv  Detail & Related papers  (2026-07-06T11:15:36Z)
• T-Rex: Tactile-Reactive Dexterous Manipulation [139.71263755530654]
  We introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder.<n>We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation.
  arXiv  Detail & Related papers  (2026-06-15T17:59:55Z)
• Physics-Informed Reinforcement Learning of Spatial Density Velocity Potentials for Map-Free Racing [73.88859384645264]
  Out-Of-Distribution (OOD) generalization to various racetracks utilizes Machine Learning (ML) to encode the mathematical relation between sensor data and vehicle actuation for end-to-end control.<n>This paper presents a DRL method that parameterizes nonlinear vehicle dynamics from the spectral distribution of depth measurements with a non-geometric, physics-informed reward, to infer vehicle time-optimal and overtaking racing controls with an Artificial Neural Network (ANN)<n>The policy outperforms human demonstrations by 12% in OOD tracks on proportionally scaled hardware, by maximizing the friction circle with tire dynamics that resemble an empirical Pacejka tire model.
  arXiv  Detail & Related papers  (2026-04-10T17:12:07Z)
• Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks [53.12548325396181]
  Tendon drives paired with soft muscle actuation enable faster and safer robots.<n>We propose a sim-to-real pipeline that learns a neural network model of this complex actuation.<n>We successfully deploy precise goal-reaching and dynamic ball-in-a-cup policies trained entirely in simulation.
  arXiv  Detail & Related papers  (2026-04-10T16:52:54Z)
• Real-Time Projected Adaptive Control for Closed-Chain Co-Manipulative Continuum Robots [0.764671395172401]
  In co-manipulative continuum robots ( CCRs), multiple continuum arms cooperate by grasping a common flexible object.<n>In practical tasks, the flexible object's physical parameters are often unknown and vary between operations.<n>This paper presents a projected adaptive control framework for CCRs formulated at the dynamic level.
  arXiv  Detail & Related papers  (2026-04-05T22:03:23Z)
• PHANTOM Hand [12.871356369910323]
  Tendon-driven underactuated hands excel in adaptive grasping but often suffer from kinematic unpredictability and highly non-actuated force transmission.<n>We introduce the PHANTOM Hand (Hybrid Precision-Augmented Compliance): a modular, human-scale system featuring 6 actuators and 15 degrees of freedom (DoFs)<n>We propose a unified framework that bridges the gap between precise analytic shaping and robust compliant grasping.
  arXiv  Detail & Related papers  (2026-03-24T12:52:18Z)
• Master Micro Residual Correction with Adaptive Tactile Fusion and Force-Mixed Control for Contact-Rich Manipulation [11.467329921917546]
  M2-ResiPolicy is a novel Master-Micro residual control architecture that synergizes high-level action guidance with low-level correction.<n>M2-ResiPolicy significantly outperforms standard Diffusion Policy and state-of-the-art Reactive Diffusion Policy.
  arXiv  Detail & Related papers  (2026-03-16T11:47:52Z)
• Tendon Force Modeling for Sim2Real Transfer of Reinforcement Learning Policies for Tendon-Driven Robots [4.374417345150659]
  We propose a method to model the tendon forces produced by typical servo motors.<n>We then utilize our force estimation model in a GPU-accelerated tendon force-driven rigid body simulation to train RL-based controllers.<n>Our model is capable of predicting tendon forces within 3% of the maximum motor force and is robot-agnostic.
  arXiv  Detail & Related papers  (2026-03-04T18:16:58Z)
• Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation [12.509181374985936]
  Human-like dexterous hands with multiple fingers offer human-level manipulation capabilities.<n>But training control policies that can directly deploy on real hardware remains difficult due to contact-rich physics.<n>We present a practical framework that utilizes dense tactile feedback combined with joint torque sensing to regulate physical interactions.
  arXiv  Detail & Related papers  (2026-01-06T07:26:39Z)
• PHANTOM: Physics-Aware Adversarial Attacks against Federated Learning-Coordinated EV Charging Management System [2.5019498860784926]
  We propose PHANTOM, a physics-aware adversarial network that is trained and optimized through a multi-agent reinforcement learning model.<n>Results demonstrate how learned attack policies disrupt load balancing and induce voltage instabilities that propagate across T and D boundaries.
  arXiv  Detail & Related papers  (2025-12-26T20:54:16Z)
• Dynamic Manipulation of Deformable Objects in 3D: Simulation, Benchmark and Learning Strategy [88.8665000676562]
  Prior methods often simplify the problem to low-speed or 2D settings, limiting their applicability to real-world 3D tasks.<n>To mitigate data scarcity, we introduce a novel simulation framework and benchmark grounded in reduced-order dynamics.<n>We propose Dynamics Informed Diffusion Policy (DIDP), a framework that integrates imitation pretraining with physics-informed test-time adaptation.
  arXiv  Detail & Related papers  (2025-05-23T03:28:25Z)
• DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control [62.24301794794304]
  Deep Adaptive Trajectory Tracking (DATT) is a learning-based approach that can precisely track arbitrary, potentially infeasible trajectories in the presence of large disturbances in the real world. DATT significantly outperforms competitive adaptive nonlinear and model predictive controllers for both feasible smooth and infeasible trajectories in unsteady wind fields. It can efficiently run online with an inference time less than 3.2 ms, less than 1/4 of the adaptive nonlinear model predictive control baseline.
  arXiv  Detail & Related papers  (2023-10-13T12:22:31Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• Learning Compliance Adaptation in Contact-Rich Manipulation [81.40695846555955]
  We propose a novel approach for learning predictive models of force profiles required for contact-rich tasks. The approach combines an anomaly detection based on Bidirectional Gated Recurrent Units (Bi-GRU) and an adaptive force/impedance controller.
  arXiv  Detail & Related papers  (2020-05-01T05:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.