Enhancing Sim2Real Transfer for Torque-Controlled Robots through Real2Sim Dynamics Estimation and Reinforcement Learning

• URL: http://arxiv.org/abs/2608.22629v1
• Date: Sun, 23 Aug 2026 22:25:33 GMT
• Title: Enhancing Sim2Real Transfer for Torque-Controlled Robots through Real2Sim Dynamics Estimation and Reinforcement Learning
• Abstract summary: Transferring reinforcement learning policies from simulation to Real-World robots remains a major challenge.<n>We propose a Real2Sim2Real pipeline that improves Sim2Real transfer for torque-controlled robotic arms.<n>This work highlights the effectiveness of accurate physical modelling in enabling stable and generalizable torque-based reinforcement learning policies.
• Score: 2.071516130824992
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transferring reinforcement learning policies from simulation to Real-World robots remains a major challenge, particularly when dealing with low-level torque control, where even small modelling inaccuracies can lead to unstable or unsafe behaviours. In this work, we propose a Real2Sim2Real pipeline that improves Sim2Real transfer for torque-controlled robotic arms by combining trajectory matching, parameter optimization via genetic algorithms, and domain randomization. Using the 7-DOF Franka Emika Panda robot, we first identify friction, inertia, and gravity compensation parameters by minimizing the error between real and simulated joint trajectories. These calibrated dynamics are then used to train a TQC-based reinforcement learning agent in simulation. The trained policy is evaluated in both Gazebo and MuJoCo environments, and finally deployed on the real robot. Our results demonstrate a significant improvement in tracking accuracy and policy robustness after parameter tuning, with smooth policy transfer from simulation to the Real-World across multiple target-reaching tasks. This work highlights the effectiveness of accurate physical modelling in enabling stable and generalizable torque-based reinforcement learning policies.

Related papers

• Actuator Reality Shaping for Zero-Shot Sim-to-Real Robot Learning [0.5277024349608834]
  Sim-to-real transfer in robot learning is often limited by discrepancies between the ideal actuator dynamics assumed during policy training and the nonlinear, hardware-dependent behavior of physical motors.<n>We introduce an alternative paradigm: actuator reality shaping.<n>We validate the approach on a single-joint high-gear-ratio servo under external loads and a 7-DOF robotic arm reaching task.
  arXiv  Detail & Related papers  (2026-07-02T14:12:41Z)
• Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks [53.12548325396181]
  Tendon drives paired with soft muscle actuation enable faster and safer robots.<n>We propose a sim-to-real pipeline that learns a neural network model of this complex actuation.<n>We successfully deploy precise goal-reaching and dynamic ball-in-a-cup policies trained entirely in simulation.
  arXiv  Detail & Related papers  (2026-04-10T16:52:54Z)
• HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation [22.6954724792851]
  We propose a two-stage gradient-based system identification framework built on the differentiable simulator MuJoCo XLA.<n>The first stage calibrates the nominal robot model using real-world data to reduce intrinsic sim-to-real discrepancies.<n>The second stage further identifies the mass distribution of the unknown payload.
  arXiv  Detail & Related papers  (2026-03-16T10:38:05Z)
• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• DrEureka: Language Model Guided Sim-To-Real Transfer [64.14314476811806]
  Transferring policies learned in simulation to the real world is a promising strategy for acquiring robot skills at scale. In this paper, we investigate using Large Language Models (LLMs) to automate and accelerate sim-to-real design. Our approach is capable of solving novel robot tasks, such as quadruped balancing and walking atop a yoga ball.
  arXiv  Detail & Related papers  (2024-06-04T04:53:05Z)
• Towards Transferring Tactile-based Continuous Force Control Policies from Simulation to Robot [19.789369416528604]
  grasp force control aims to manipulate objects safely by limiting the amount of force exerted on the object. Prior works have either hand-modeled their force controllers, employed model-based approaches, or have not shown sim-to-real transfer. We propose a model-free deep reinforcement learning approach trained in simulation and then transferred to the robot without further fine-tuning.
  arXiv  Detail & Related papers  (2023-11-13T11:29:06Z)
• Robust Visual Sim-to-Real Transfer for Robotic Manipulation [79.66851068682779]
  Learning visuomotor policies in simulation is much safer and cheaper than in the real world. However, due to discrepancies between the simulated and real data, simulator-trained policies often fail when transferred to real robots. One common approach to bridge the visual sim-to-real domain gap is domain randomization (DR)
  arXiv  Detail & Related papers  (2023-07-28T05:47:24Z)
• Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robots [14.760426243769308]
  In this work, we demonstrate robust sim-to-real transfer of control policies on the BALLU robots via system identification. Rather than relying on standard supervised learning formulations, we utilize deep reinforcement learning to train an external force policy. We analyze the improved simulation fidelity by comparing the simulation trajectories against the real-world ones.
  arXiv  Detail & Related papers  (2023-03-16T18:49:05Z)
• Real2Sim2Real Transfer for Control of Cable-driven Robots via a Differentiable Physics Engine [9.268539775233346]
  Tensegrity robots exhibit high strength-to-weight ratios and significant deformations. They are hard to control, however, due to high dimensionality, complex dynamics, and a coupled architecture. This paper describes a Real2Sim2Real (R2S2R) strategy for modeling tensegrity robots.
  arXiv  Detail & Related papers  (2022-09-13T18:51:26Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.