A Framework for Studying Reinforcement Learning and Sim-to-Real in Robot Soccer

• URL: http://arxiv.org/abs/2008.12624v1
• Date: Tue, 18 Aug 2020 23:52:32 GMT
• Title: A Framework for Studying Reinforcement Learning and Sim-to-Real in Robot Soccer
• Authors: Hansenclever F. Bassani, Renie A. Delgado, Jos\'e Nilton de O. Lima Junior, Heitor R. Medeiros, Pedro H. M. Braga, Mateus G. Machado, Lucas H. C. Santos and Alain Tapp
• Abstract summary: This article introduces an open framework, called VSSS-RL, for studying Reinforcement Learning (RL) and sim-to-real in robot soccer. We propose a simulated environment in which continuous or discrete control policies can be trained to control the complete behavior of soccer agents. Our results show that the trained policies learned a broad repertoire of behaviors that are difficult to implement with handcrafted control policies.
• Score: 1.1785354380793065
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This article introduces an open framework, called VSSS-RL, for studying Reinforcement Learning (RL) and sim-to-real in robot soccer, focusing on the IEEE Very Small Size Soccer (VSSS) league. We propose a simulated environment in which continuous or discrete control policies can be trained to control the complete behavior of soccer agents and a sim-to-real method based on domain adaptation to adapt the obtained policies to real robots. Our results show that the trained policies learned a broad repertoire of behaviors that are difficult to implement with handcrafted control policies. With VSSS-RL, we were able to beat human-designed policies in the 2019 Latin American Robotics Competition (LARC), achieving 4th place out of 21 teams, being the first to apply Reinforcement Learning (RL) successfully in this competition. Both environment and hardware specifications are available open-source to allow reproducibility of our results and further studies.

Related papers

• Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning [34.055177769808914]
  We introduce a dynamic, interactive RL testbed based on robot air hockey. Our testbed allows a varied assessment of RL capabilities. The robot air hockey testbed also supports sim-to-real transfer with three domains.
  arXiv  Detail & Related papers  (2024-05-06T02:13:08Z)
• SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning [85.21378553454672]
  We develop a library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment. We find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation. These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent robustness recovery and correction behaviors.
  arXiv  Detail & Related papers  (2024-01-29T10:01:10Z)
• Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion [66.69666636971922]
  We present APRL, a policy regularization framework that modulates the robot's exploration over the course of training. APRL enables a quadrupedal robot to efficiently learn to walk entirely in the real world within minutes.
  arXiv  Detail & Related papers  (2023-10-26T17:51:46Z)
• DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality [64.51295032956118]
  We train a policy that can perform robust dexterous manipulation on an anthropomorphic robot hand. Our work reaffirms the possibilities of sim-to-real transfer for dexterous manipulation in diverse kinds of hardware and simulator setups.
  arXiv  Detail & Related papers  (2022-10-25T01:51:36Z)
• How does AI play football? An analysis of RL and real-world football strategies [2.1628586087680075]
  reinforcement learning (RL) has made it possible to develop sophisticated agents that excel in a wide range of applications. We examine the play-style characteristics of football RL agents and uncover how strategies may develop during training. The learnt strategies are then compared with those of real football players.
  arXiv  Detail & Related papers  (2021-11-24T08:44:23Z)
• Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments [89.04823188871906]
  Generation of diverse realistic scenarios is challenging for real-time strategy (RTS) environments. Most of the existing simulators rely on randomly generating the environments. We introduce the benefits of adopting an existing formal scenario specification language, SCENIC, to assist researchers.
  arXiv  Detail & Related papers  (2021-06-18T21:49:46Z)
• rSoccer: A Framework for Studying Reinforcement Learning in Small and Very Small Size Robot Soccer [0.0]
  This article introduces an open-source simulator for the IEEE Very Small Size Soccer and the Small Size League optimized for reinforcement learning experiments. We also propose a framework for creating OpenAI Gym environments with a set of benchmarks tasks for evaluating single-agent and multi-agent robot soccer skills.
  arXiv  Detail & Related papers  (2021-06-15T01:30:21Z)
• How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned [111.06812202454364]
  We present a number of case studies involving robotic deep RL. We discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting.
  arXiv  Detail & Related papers  (2021-02-04T22:09:28Z)
• Learning to Play Table Tennis From Scratch using Muscular Robots [34.34824536814943]
  This work is the first to (a) fail-safe learn of a safety-critical dynamic task using anthropomorphic robot arms, (b) learn a precision-demanding problem with a PAM-driven system, and (c) train robots to play table tennis without real balls. Videos and datasets are available at muscularTT.embodied.ml.
  arXiv  Detail & Related papers  (2020-06-10T16:43:27Z)
• Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks [70.56451186797436]
  We study how to use meta-reinforcement learning to solve the bulk of the problem in simulation. We demonstrate our approach by training an agent to successfully perform challenging real-world insertion tasks.
  arXiv  Detail & Related papers  (2020-04-29T18:00:22Z)
• Learning to Play Soccer by Reinforcement and Applying Sim-to-Real to Compete in the Real World [1.3114165111679479]
  This work presents an application of Reinforcement Learning (RL) for the complete control of real soccer robots of the IEEE Very Small Size Soccer (VSSS) In the VSSS league, two teams of three small robots play against each other. We propose a simulated environment in which continuous or discrete control policies can be trained, and a Sim-to-Real method to allow using the obtained policies to control a robot in the real world.
  arXiv  Detail & Related papers  (2020-03-24T20:23:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.