From Motor Control to Team Play in Simulated Humanoid Football
- URL: http://arxiv.org/abs/2105.12196v1
- Date: Tue, 25 May 2021 20:17:10 GMT
- Title: From Motor Control to Team Play in Simulated Humanoid Football
- Authors: Siqi Liu, Guy Lever, Zhe Wang, Josh Merel, S. M. Ali Eslami, Daniel
Hennes, Wojciech M. Czarnecki, Yuval Tassa, Shayegan Omidshafiei, Abbas
Abdolmaleki, Noah Y. Siegel, Leonard Hasenclever, Luke Marris, Saran
Tunyasuvunakool, H. Francis Song, Markus Wulfmeier, Paul Muller, Tuomas
Haarnoja, Brendan D. Tracey, Karl Tuyls, Thore Graepel, Nicolas Heess
- Abstract summary: We train teams of physically simulated humanoid avatars to play football in a realistic virtual environment.
In a sequence of stages, players first learn to control a fully articulated body to perform realistic, human-like movements.
They then acquire mid-level football skills such as dribbling and shooting.
Finally, they develop awareness of others and play as a team, bridging the gap between low-level motor control at a timescale of milliseconds.
- Score: 56.86144022071756
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Intelligent behaviour in the physical world exhibits structure at multiple
spatial and temporal scales. Although movements are ultimately executed at the
level of instantaneous muscle tensions or joint torques, they must be selected
to serve goals defined on much longer timescales, and in terms of relations
that extend far beyond the body itself, ultimately involving coordination with
other agents. Recent research in artificial intelligence has shown the promise
of learning-based approaches to the respective problems of complex movement,
longer-term planning and multi-agent coordination. However, there is limited
research aimed at their integration. We study this problem by training teams of
physically simulated humanoid avatars to play football in a realistic virtual
environment. We develop a method that combines imitation learning, single- and
multi-agent reinforcement learning and population-based training, and makes use
of transferable representations of behaviour for decision making at different
levels of abstraction. In a sequence of stages, players first learn to control
a fully articulated body to perform realistic, human-like movements such as
running and turning; they then acquire mid-level football skills such as
dribbling and shooting; finally, they develop awareness of others and play as a
team, bridging the gap between low-level motor control at a timescale of
milliseconds, and coordinated goal-directed behaviour as a team at the
timescale of tens of seconds. We investigate the emergence of behaviours at
different levels of abstraction, as well as the representations that underlie
these behaviours using several analysis techniques, including statistics from
real-world sports analytics. Our work constitutes a complete demonstration of
integrated decision-making at multiple scales in a physically embodied
multi-agent setting. See project video at https://youtu.be/KHMwq9pv7mg.
Related papers
- Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning [17.906144781244336]
We train end-to-end robot soccer policies with fully onboard computation and sensing via egocentric RGB vision.
This paper constitutes a first demonstration of end-to-end training for multi-agent robot soccer.
arXiv Detail & Related papers (2024-05-03T18:41:13Z) - DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervision [26.9579556496875]
Joint manipulation of moving objects and locomotion with legs, such as playing soccer, receive scant attention in the learning community.
We propose a feedback control block to compute the necessary body-level movement accurately and using the outputs as dynamic joint-level locomotion supervision.
We observe that our learning scheme can not only make the policy network converge faster but also enable soccer robots to perform sophisticated maneuvers.
arXiv Detail & Related papers (2024-03-21T11:16:28Z) - Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots.
We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing.
This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
arXiv Detail & Related papers (2024-01-30T10:48:43Z) - Hierarchical generative modelling for autonomous robots [8.023920215148486]
We show how a humanoid robot can autonomously complete a complex task that requires a holistic use of locomotion, manipulation, and grasping.
Specifically, we demonstrate the ability of a humanoid robot that can retrieve and transport a box, open and walk through a door to reach the destination, approach and kick a football, while showing robust performance in presence of body damage and ground irregularities.
arXiv Detail & Related papers (2023-08-15T13:51:03Z) - Persistent-Transient Duality: A Multi-mechanism Approach for Modeling
Human-Object Interaction [58.67761673662716]
Humans are highly adaptable, swiftly switching between different modes to handle different tasks, situations and contexts.
In Human-object interaction (HOI) activities, these modes can be attributed to two mechanisms: (1) the large-scale consistent plan for the whole activity and (2) the small-scale children interactive actions that start and end along the timeline.
This work proposes to model two concurrent mechanisms that jointly control human motion.
arXiv Detail & Related papers (2023-07-24T12:21:33Z) - Action valuation of on- and off-ball soccer players based on multi-agent
deep reinforcement learning [4.477124009148237]
We propose a method of valuing possible actions for on-temporal and off-ball players in a single holistic framework based on multi-agent deep reinforcement learning.
Our approach can assess how multiple players move continuously throughout the game which is difficult to be discretized or labeled.
arXiv Detail & Related papers (2023-05-29T05:14:36Z) - Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning [26.13655448415553]
Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot.
We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game.
The resulting agent exhibits robust and dynamic movement skills such as rapid fall recovery, walking, turning, kicking and more.
arXiv Detail & Related papers (2023-04-26T16:25:54Z) - Bottom-Up Skill Discovery from Unsegmented Demonstrations for
Long-Horizon Robot Manipulation [55.31301153979621]
We tackle real-world long-horizon robot manipulation tasks through skill discovery.
We present a bottom-up approach to learning a library of reusable skills from unsegmented demonstrations.
Our method has shown superior performance over state-of-the-art imitation learning methods in multi-stage manipulation tasks.
arXiv Detail & Related papers (2021-09-28T16:18:54Z) - UniCon: Universal Neural Controller For Physics-based Character Motion [70.45421551688332]
We propose a physics-based universal neural controller (UniCon) that learns to master thousands of motions with different styles by learning on large-scale motion datasets.
UniCon can support keyboard-driven control, compose motion sequences drawn from a large pool of locomotion and acrobatics skills and teleport a person captured on video to a physics-based virtual avatar.
arXiv Detail & Related papers (2020-11-30T18:51:16Z) - Thinking While Moving: Deep Reinforcement Learning with Concurrent
Control [122.49572467292293]
We study reinforcement learning in settings where sampling an action from the policy must be done concurrently with the time evolution of the controlled system.
Much like a person or an animal, the robot must think and move at the same time, deciding on its next action before the previous one has completed.
arXiv Detail & Related papers (2020-04-13T17:49:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.