Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching

• URL: http://arxiv.org/abs/2605.09789v1
• Date: Sun, 10 May 2026 22:20:20 GMT
• Title: Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching
• Abstract summary: Domain-Randomized Instance Set (DRIS) represents and propagates a set of randomized instances simultaneously.<n>We show that DRIS yields more robust policies and alleviates the need for real-world fine-tuning.<n>We demonstrate this on a challenging reactive catching task.
• Score: 5.503129542452224
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dexterous manipulation is physics-intensive and highly sensitive to modeling errors and perception noise, making sim-to-real transfer prohibitively challenging. Domain randomization (DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, we propose Domain-Randomized Instance Set (DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real-world fine-tuning, even with a modest number of instances (e.g., 10). We demonstrate this on a challenging reactive catching task. Unlike traditional catching setups that use end-effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), our system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero-shot sim-to-real transfer.

Related papers

• Enhancing Sim2Real Transfer for Torque-Controlled Robots through Real2Sim Dynamics Estimation and Reinforcement Learning [2.071516130824992]
  Transferring reinforcement learning policies from simulation to Real-World robots remains a major challenge.<n>We propose a Real2Sim2Real pipeline that improves Sim2Real transfer for torque-controlled robotic arms.<n>This work highlights the effectiveness of accurate physical modelling in enabling stable and generalizable torque-based reinforcement learning policies.
  arXiv  Detail & Related papers  (2026-08-23T22:25:33Z)
• Provably Safe Trajectory Generation for Manipulators Under Motion and Environmental Uncertainties [26.41382315967256]
  This article proposes a novel risk-bounded motion planning framework to address the unmet need for safe optimal motion planning.<n>Our approach integrates a rigid manipulator deep Koopman operator (RM-DeKO) model to predict the robot's state distribution under motion uncertainty.<n>We then introduce an efficient hierarchical verification method that combines parallel physics simulations with sum-of-squares (SOS) programming as a filter for fine-grained, formal certification of collision risk.
  arXiv  Detail & Related papers  (2026-03-10T01:48:48Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Failure Forecasting Boosts Robustness of Sim2Real Rhythmic Insertion Policies [32.99644520625179]
  This paper addresses the challenges of Rhythmic Insertion Tasks (RIT), where a robot must repeatedly perform high-precision insertions.<n>We propose a framework that integrates a reinforcement learning-based insertion policy with a failure forecasting module.<n>Our method achieves a high one-time success rate and robustly maintains performance over long-horizon repetitive tasks.
  arXiv  Detail & Related papers  (2025-07-09T03:38:44Z)
• Dynamic Manipulation of Deformable Objects in 3D: Simulation, Benchmark and Learning Strategy [88.8665000676562]
  Prior methods often simplify the problem to low-speed or 2D settings, limiting their applicability to real-world 3D tasks.<n>To mitigate data scarcity, we introduce a novel simulation framework and benchmark grounded in reduced-order dynamics.<n>We propose Dynamics Informed Diffusion Policy (DIDP), a framework that integrates imitation pretraining with physics-informed test-time adaptation.
  arXiv  Detail & Related papers  (2025-05-23T03:28:25Z)
• Neural Fidelity Calibration for Informative Sim-to-Real Adaptation [10.117298045153564]
  Deep reinforcement learning can seamlessly transfer agile locomotion and navigation skills from the simulator to real world.<n>However, bridging the sim-to-real gap with domain randomization or adversarial methods often demands expert physics knowledge to ensure policy robustness.<n>We propose Neural Fidelity (NFC), a novel framework that employs conditional score-based diffusion models to calibrate simulator physical coefficients and residual fidelity domains online during robot execution.
  arXiv  Detail & Related papers  (2025-04-11T15:12:12Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• Domain Randomization via Entropy Maximization [32.76511125425768]
  We propose a novel approach to address sim-to-real transfer, which automatically shapes dynamics distributions during training in simulation without requiring real-world data. We introduce DOmain RAndomization via Entropy MaximizatiON (DORAEMON), a constrained optimization problem that directly maximizes the entropy of the training distribution. We empirically validate the consistent benefits of DORAEMON in obtaining highly adaptive and generalizable policies.
  arXiv  Detail & Related papers  (2023-11-03T12:54:05Z)
• Robust Visual Sim-to-Real Transfer for Robotic Manipulation [79.66851068682779]
  Learning visuomotor policies in simulation is much safer and cheaper than in the real world. However, due to discrepancies between the simulated and real data, simulator-trained policies often fail when transferred to real robots. One common approach to bridge the visual sim-to-real domain gap is domain randomization (DR)
  arXiv  Detail & Related papers  (2023-07-28T05:47:24Z)
• Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimization [63.75188254377202]
  Deep reinforcement learning algorithms can perform poorly in real-world tasks due to discrepancy between source and target environments. We propose a novel model-free actor-critic algorithm to learn robust policies without modeling the disturbance in advance. Experiments in several robot control tasks demonstrate that SCPO learns robust policies against the disturbance in transition dynamics.
  arXiv  Detail & Related papers  (2021-12-20T13:13:05Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.