An analysis of Reinforcement Learning applied to Coach task in IEEE Very Small Size Soccer

• URL: http://arxiv.org/abs/2011.11785v1
• Date: Mon, 23 Nov 2020 23:10:06 GMT
• Title: An analysis of Reinforcement Learning applied to Coach task in IEEE Very Small Size Soccer
• Authors: Carlos H. C. Pena, Mateus G. Machado, Mariana S. Barros, Jos\'e D. P. Silva, Lucas D. Maciel, Tsang Ing Ren, Edna N. S. Barros, Pedro H. M. Braga, Hansenclever F. Bassani
• Abstract summary: This paper proposes an end-to-end approach for the coaching task based on Reinforcement Learning (RL) We trained two RL policies against three different teams in a simulated environment. Our results were assessed against one of the top teams of the VSSS league.
• Score: 2.5400028272658144
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The IEEE Very Small Size Soccer (VSSS) is a robot soccer competition in which two teams of three small robots play against each other. Traditionally, a deterministic coach agent will choose the most suitable strategy and formation for each adversary's strategy. Therefore, the role of a coach is of great importance to the game. In this sense, this paper proposes an end-to-end approach for the coaching task based on Reinforcement Learning (RL). The proposed system processes the information during the simulated matches to learn an optimal policy that chooses the current formation, depending on the opponent and game conditions. We trained two RL policies against three different teams (balanced, offensive, and heavily offensive) in a simulated environment. Our results were assessed against one of the top teams of the VSSS league, showing promising results after achieving a win/loss ratio of approximately 2.0.

Related papers

• A Minimaximalist Approach to Reinforcement Learning from Human Feedback [49.45285664482369]
  We present Self-Play Preference Optimization (SPO), an algorithm for reinforcement learning from human feedback. Our approach is minimalist in that it does not require training a reward model nor unstable adversarial training. We demonstrate that on a suite of continuous control tasks, we are able to learn significantly more efficiently than reward-model based approaches.
  arXiv  Detail & Related papers  (2024-01-08T17:55:02Z)
• All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization [57.615269148301515]
  In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time. We propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them. Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times.
  arXiv  Detail & Related papers  (2023-10-02T08:11:07Z)
• How does AI play football? An analysis of RL and real-world football strategies [2.1628586087680075]
  reinforcement learning (RL) has made it possible to develop sophisticated agents that excel in a wide range of applications. We examine the play-style characteristics of football RL agents and uncover how strategies may develop during training. The learnt strategies are then compared with those of real football players.
  arXiv  Detail & Related papers  (2021-11-24T08:44:23Z)
• Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition [88.26752130107259]
  In real-world multiagent systems, agents with different capabilities may join or leave without altering the team's overarching goals. We propose COPA, a coach-player framework to tackle this problem. We 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players.
  arXiv  Detail & Related papers  (2021-05-18T17:27:37Z)
• Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competition [1.675857332621569]
  This work proposes a scheme that allows learning complex multi-agent behaviors in a sample efficient manner, applied to 2v2 soccer. The problem is formulated as a Markov game, and solved using deep reinforcement learning. Our results show that high quality soccer play can be obtained with our approach in just under 40M interactions.
  arXiv  Detail & Related papers  (2021-03-09T01:57:16Z)
• L2E: Learning to Exploit Your Opponent [66.66334543946672]
  We propose a novel Learning to Exploit framework for implicit opponent modeling. L2E acquires the ability to exploit opponents by a few interactions with different opponents during training. We propose a novel opponent strategy generation algorithm that produces effective opponents for training automatically.
  arXiv  Detail & Related papers  (2021-02-18T14:27:59Z)
• A Framework for Studying Reinforcement Learning and Sim-to-Real in Robot Soccer [1.1785354380793065]
  This article introduces an open framework, called VSSS-RL, for studying Reinforcement Learning (RL) and sim-to-real in robot soccer. We propose a simulated environment in which continuous or discrete control policies can be trained to control the complete behavior of soccer agents. Our results show that the trained policies learned a broad repertoire of behaviors that are difficult to implement with handcrafted control policies.
  arXiv  Detail & Related papers  (2020-08-18T23:52:32Z)
• Learning to Play Sequential Games versus Unknown Opponents [93.8672371143881]
  We consider a repeated sequential game between a learner, who plays first, and an opponent who responds to the chosen action. We propose a novel algorithm for the learner when playing against an adversarial sequence of opponents. Our results include algorithm's regret guarantees that depend on the regularity of the opponent's response.
  arXiv  Detail & Related papers  (2020-07-10T09:33:05Z)
• Learning from Learners: Adapting Reinforcement Learning Agents to be Competitive in a Card Game [71.24825724518847]
  We present a study on how popular reinforcement learning algorithms can be adapted to learn and to play a real-world implementation of a competitive multiplayer card game. We propose specific training and validation routines for the learning agents, in order to evaluate how the agents learn to be competitive and explain how they adapt to each others' playing style.
  arXiv  Detail & Related papers  (2020-04-08T14:11:05Z)
• Learning to Play Soccer by Reinforcement and Applying Sim-to-Real to Compete in the Real World [1.3114165111679479]
  This work presents an application of Reinforcement Learning (RL) for the complete control of real soccer robots of the IEEE Very Small Size Soccer (VSSS) In the VSSS league, two teams of three small robots play against each other. We propose a simulated environment in which continuous or discrete control policies can be trained, and a Sim-to-Real method to allow using the obtained policies to control a robot in the real world.
  arXiv  Detail & Related papers  (2020-03-24T20:23:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.