The Reality Gap in Robotics: Challenges, Solutions, and Best Practices

• URL: http://arxiv.org/abs/2510.20808v1
• Date: Thu, 23 Oct 2025 17:58:53 GMT
• Title: The Reality Gap in Robotics: Challenges, Solutions, and Best Practices
• Authors: Elie Aljalbout, Jiaxu Xing, Angel Romero, Iretiayo Akinola, Caelan Reed Garrett, Eric Heiden, Abhishek Gupta, Tucker Hermans, Yashraj Narang, Dieter Fox, Davide Scaramuzza, Fabio Ramos,
• Abstract summary: Sim-to-real transfer is one of the most pressing challenges in robotics.<n>Recent advances in sim-to-real transfer have demonstrated promising results across various platforms.<n>However, challenges persist, and a deeper understanding of the reality gap's root causes and solutions is necessary.
• Score: 42.74136594950574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning has facilitated significant advancements across various robotics domains, including navigation, locomotion, and manipulation. Many such achievements have been driven by the extensive use of simulation as a critical tool for training and testing robotic systems prior to their deployment in real-world environments. However, simulations consist of abstractions and approximations that inevitably introduce discrepancies between simulated and real environments, known as the reality gap. These discrepancies significantly hinder the successful transfer of systems from simulation to the real world. Closing this gap remains one of the most pressing challenges in robotics. Recent advances in sim-to-real transfer have demonstrated promising results across various platforms, including locomotion, navigation, and manipulation. By leveraging techniques such as domain randomization, real-to-sim transfer, state and action abstractions, and sim-real co-training, many works have overcome the reality gap. However, challenges persist, and a deeper understanding of the reality gap's root causes and solutions is necessary. In this survey, we present a comprehensive overview of the sim-to-real landscape, highlighting the causes, solutions, and evaluation metrics for the reality gap and sim-to-real transfer.

Related papers

• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio [138.07247714782412]
  MultiGen is a framework that integrates large-scale generative models into traditional physics simulators.<n>We demonstrate effective zero-shot transfer to real-world pouring with novel containers and liquids.
  arXiv  Detail & Related papers  (2025-07-03T17:59:58Z)
• From Abstraction to Reality: DARPA's Vision for Robust Sim-to-Real Autonomy [6.402441477393285]
  TIAMAT aims to address rapid and robust transfer of autonomy technologies across dynamic and complex environments.<n>Current methods for simulation-to-reality (sim-to-real) transfer often rely on high-fidelity simulations.<n>TIAMAT's approaches aim to achieve abstract-to-real transfer for effective and rapid real-world adaptation.
  arXiv  Detail & Related papers  (2025-03-14T02:06:10Z)
• An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer with Differentiable Simulation [13.15220962477623]
  This paper introduces a novel Real-Sim-Real loop framework to address the gap between simulation and real-world conditions.<n>A key contribution of our work is the design of an informative cost function that encourages the collection of diverse and representative real-world data.<n>Our approach is implemented on the versatile Mujoco MJX platform, and our framework is compatible with a wide range of robotic systems.
  arXiv  Detail & Related papers  (2025-03-13T07:27:05Z)
• How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence [16.24370001383615]
  How to adapt driving knowledge learned in simulation to reality becomes a critical issue. Virtual simulation world differs from the real world in many aspects such as lighting, textures, vehicle dynamics, and agents' behaviors. Three categories of approaches to address the reality gap issue: transferring knowledge from simulation to reality (sim2real), learning in digital twins (DTs), and learning by parallel intelligence (PI)
  arXiv  Detail & Related papers  (2023-05-02T09:00:32Z)
• Robot Learning from Randomized Simulations: A Review [59.992761565399185]
  Deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. State-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive. We focus on a technique named 'domain randomization' which is a method for learning from randomized simulations.
  arXiv  Detail & Related papers  (2021-11-01T13:55:41Z)
• Reactive Long Horizon Task Execution via Visual Skill and Precondition Models [59.76233967614774]
  We describe an approach for sim-to-real training that can accomplish unseen robotic tasks using models learned in simulation to ground components of a simple task planner. We show an increase in success rate from 91.6% to 98% in simulation and from 10% to 80% success rate in the real-world as compared with naive baselines.
  arXiv  Detail & Related papers  (2020-11-17T15:24:01Z)
• Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey [0.07366405857677225]
  We cover the background behind sim-to-real transfer in deep reinforcement learning. We overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation.
  arXiv  Detail & Related papers  (2020-09-24T21:05:46Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.