Trade-off on Sim2Real Learning: Real-world Learning Faster than
Simulations
- URL: http://arxiv.org/abs/2007.10675v4
- Date: Mon, 10 Jan 2022 06:24:14 GMT
- Title: Trade-off on Sim2Real Learning: Real-world Learning Faster than
Simulations
- Authors: Jingyi Huang, Yizheng Zhang, Fabio Giardina, Andre Rosendo
- Abstract summary: We compare a Deep Bayesian Learning algorithm with a model-free DRL algorithm while analyzing our results collected from both simulations and real-world experiments.
While considering Sim and Real learning, our experiments show that the sample-efficient Deep Bayesian RL performance is better than DRL even when computation time is taken in consideration.
- Score: 1.949912057689623
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep Reinforcement Learning (DRL) experiments are commonly performed in
simulated environments due to the tremendous training sample demands from deep
neural networks. In contrast, model-based Bayesian Learning allows a robot to
learn good policies within a few trials in the real world. Although it takes
fewer iterations, Bayesian methods pay a relatively higher computational cost
per trial, and the advantage of such methods is strongly tied to dimensionality
and noise. In here, we compare a Deep Bayesian Learning algorithm with a
model-free DRL algorithm while analyzing our results collected from both
simulations and real-world experiments. While considering Sim and Real
learning, our experiments show that the sample-efficient Deep Bayesian RL
performance is better than DRL even when computation time (as opposed to number
of iterations) is taken in consideration. Additionally, the difference in
computation time between Deep Bayesian RL performed in simulation and in
experiments point to a viable path to traverse the reality gap. We also show
that a mix between Sim and Real does not outperform a purely Real approach,
pointing to the possibility that reality can provide the best prior knowledge
to a Bayesian Learning. Roboticists design and build robots every day, and our
results show that a higher learning efficiency in the real-world will shorten
the time between design and deployment by skipping simulations.
Related papers
- Sim-to-Real Deep Reinforcement Learning with Manipulators for
Pick-and-place [1.7478203318226313]
When transferring a Deep Reinforcement Learning model from simulation to the real world, the performance could be unsatisfactory.
This paper proposes a self-supervised vision-based DRL method that allows robots to pick and place objects effectively.
arXiv Detail & Related papers (2023-09-17T11:51:18Z) - Continual learning autoencoder training for a particle-in-cell
simulation via streaming [52.77024349608834]
upcoming exascale era will provide a new generation of physics simulations with high resolution.
These simulations will have a high resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible.
This work presents an approach that trains a neural network concurrently to a running simulation without data on a disk.
arXiv Detail & Related papers (2022-11-09T09:55:14Z) - Simulation-Based Parallel Training [55.41644538483948]
We present our ongoing work to design a training framework that alleviates those bottlenecks.
It generates data in parallel with the training process.
We present a strategy to mitigate this bias with a memory buffer.
arXiv Detail & Related papers (2022-11-08T09:31:25Z) - Provable Sim-to-real Transfer in Continuous Domain with Partial
Observations [39.18274543757048]
Sim-to-real transfer trains RL agents in the simulated environments and then deploys them in the real world.
We show that a popular robust adversarial training algorithm is capable of learning a policy from the simulated environment that is competitive to the optimal policy in the real-world environment.
arXiv Detail & Related papers (2022-10-27T16:37:52Z) - SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via
Differentiable Physics-Based Simulation and Rendering [49.78647219715034]
We propose a sensing-aware model-based reinforcement learning system called SAM-RL.
With the sensing-aware learning pipeline, SAM-RL allows a robot to select an informative viewpoint to monitor the task process.
We apply our framework to real world experiments for accomplishing three manipulation tasks: robotic assembly, tool manipulation, and deformable object manipulation.
arXiv Detail & Related papers (2022-10-27T05:30:43Z) - Distilled Domain Randomization [23.178141671320436]
We propose to combine reinforcement learning from randomized physics simulations with policy distillation.
Our algorithm, called Distilled Domain Randomization (DiDoR), distills so-called teacher policies, which are experts on domains.
This way, DiDoR learns controllers which transfer directly from simulation to reality, without requiring data from the target domain.
arXiv Detail & Related papers (2021-12-06T16:35:08Z) - Robot Learning from Randomized Simulations: A Review [59.992761565399185]
Deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data.
State-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive.
We focus on a technique named 'domain randomization' which is a method for learning from randomized simulations.
arXiv Detail & Related papers (2021-11-01T13:55:41Z) - On the Theory of Reinforcement Learning with Once-per-Episode Feedback [120.5537226120512]
We introduce a theory of reinforcement learning in which the learner receives feedback only once at the end of an episode.
This is arguably more representative of real-world applications than the traditional requirement that the learner receive feedback at every time step.
arXiv Detail & Related papers (2021-05-29T19:48:51Z) - Reactive Long Horizon Task Execution via Visual Skill and Precondition
Models [59.76233967614774]
We describe an approach for sim-to-real training that can accomplish unseen robotic tasks using models learned in simulation to ground components of a simple task planner.
We show an increase in success rate from 91.6% to 98% in simulation and from 10% to 80% success rate in the real-world as compared with naive baselines.
arXiv Detail & Related papers (2020-11-17T15:24:01Z) - Sim-to-Real Transfer with Incremental Environment Complexity for
Reinforcement Learning of Depth-Based Robot Navigation [1.290382979353427]
Soft-Actor Critic (SAC) training strategy using incremental environment complexity is proposed to drastically reduce the need for additional training in the real world.
The application addressed is depth-based mapless navigation, where a mobile robot should reach a given waypoint in a cluttered environment with no prior mapping information.
arXiv Detail & Related papers (2020-04-30T10:47:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.