Sim and Real: Better Together

• URL: http://arxiv.org/abs/2110.00445v2
• Date: Tue, 5 Oct 2021 07:02:18 GMT
• Title: Sim and Real: Better Together
• Authors: Shirli Di Castro Shashua, Dotan Di Castro, Shie Mannor
• Abstract summary: We demonstrate how to learn simultaneously from both simulation and interaction with the real environment. We propose an algorithm for balancing the large number of samples from the high throughput but less accurate simulation. We analyze such multi-environment interaction theoretically, and provide convergence properties, through a novel theoretical replay buffer analysis.
• Score: 47.14469055555684
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Simulation is used extensively in autonomous systems, particularly in robotic manipulation. By far, the most common approach is to train a controller in simulation, and then use it as an initial starting point for the real system. We demonstrate how to learn simultaneously from both simulation and interaction with the real environment. We propose an algorithm for balancing the large number of samples from the high throughput but less accurate simulation and the low-throughput, high-fidelity and costly samples from the real environment. We achieve that by maintaining a replay buffer for each environment the agent interacts with. We analyze such multi-environment interaction theoretically, and provide convergence properties, through a novel theoretical replay buffer analysis. We demonstrate the efficacy of our method on a sim-to-real environment.

Related papers

• Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications [23.94013806312391]
  We propose a novel approach that dynamically adjusts simulation environment parameters online using in-context learning. We validate our approach across two tasks: object scooping and table air hockey. Our approach delivers efficient and smooth system identification, advancing the deployment of robots in dynamic real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-27T07:13:38Z)
• Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL [25.991354823569033]
  We show that in many regimes, while direct sim2real transfer may fail, we can utilize the simulator to learn a set of emphexploratory policies. In particular, in the setting of low-rank MDPs, we show that coupling these exploratory policies with simple, practical approaches. This is the first evidence that simulation transfer yields a provable gain in reinforcement learning in settings where direct sim2real transfer fails.
  arXiv  Detail & Related papers  (2024-10-26T19:12:27Z)
• Zero-shot Sim2Real Adaptation Across Environments [45.44896435487879]
  We propose a Reverse Action Transformation (RAT) policy which learns to imitate simulated policies in the real-world. RAT can then be deployed on top of a Universal Policy Network to achieve zero-shot adaptation to new environments.
  arXiv  Detail & Related papers  (2023-02-08T11:59:07Z)
• Provable Sim-to-real Transfer in Continuous Domain with Partial Observations [39.18274543757048]
  Sim-to-real transfer trains RL agents in the simulated environments and then deploys them in the real world. We show that a popular robust adversarial training algorithm is capable of learning a policy from the simulated environment that is competitive to the optimal policy in the real-world environment.
  arXiv  Detail & Related papers  (2022-10-27T16:37:52Z)
• Inferring Articulated Rigid Body Dynamics from RGBD Video [18.154013621342266]
  We introduce a pipeline that combines inverse rendering with differentiable simulation to create digital twins of real-world articulated mechanisms. Our approach accurately reconstructs the kinematic tree of an articulated mechanism being manipulated by a robot.
  arXiv  Detail & Related papers  (2022-03-20T08:19:02Z)
• Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation [98.51313127382937]
  We focus on the use of labels in the synthetic domain alone. Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator. We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
  arXiv  Detail & Related papers  (2021-11-15T18:37:43Z)
• DriveGAN: Towards a Controllable High-Quality Neural Simulation [147.6822288981004]
  We introduce a novel high-quality neural simulator referred to as DriveGAN. DriveGAN achieves controllability by disentangling different components without supervision. We train DriveGAN on multiple datasets, including 160 hours of real-world driving data.
  arXiv  Detail & Related papers  (2021-04-30T15:30:05Z)
• Auto-Tuned Sim-to-Real Transfer [143.44593793640814]
  Policies trained in simulation often fail when transferred to the real world. Current approaches to tackle this problem, such as domain randomization, require prior knowledge and engineering. We propose a method for automatically tuning simulator system parameters to match the real world.
  arXiv  Detail & Related papers  (2021-04-15T17:59:55Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.