Hierarchical Decentralized Deep Reinforcement Learning Architecture for
a Simulated Four-Legged Agent
- URL: http://arxiv.org/abs/2210.08003v1
- Date: Wed, 21 Sep 2022 07:55:33 GMT
- Title: Hierarchical Decentralized Deep Reinforcement Learning Architecture for
a Simulated Four-Legged Agent
- Authors: W. Zai El Amri and L. Hermes and M. Schilling
- Abstract summary: In nature, control of movement happens in a hierarchical and decentralized fashion.
We present a novel decentral, hierarchical architecture to control a simulated legged agent.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Legged locomotion is widespread in nature and has inspired the design of
current robots. The controller of these legged robots is often realized as one
centralized instance. However, in nature, control of movement happens in a
hierarchical and decentralized fashion. Introducing these biological design
principles into robotic control systems has motivated this work. We tackle the
question whether decentralized and hierarchical control is beneficial for
legged robots and present a novel decentral, hierarchical architecture to
control a simulated legged agent. Three different tasks varying in complexity
are designed to benchmark five architectures (centralized, decentralized,
hierarchical and two different combinations of hierarchical decentralized
architectures). The results demonstrate that decentralizing the different
levels of the hierarchical architectures facilitates learning of the agent,
ensures more energy efficient movements as well as robustness towards new
unseen environments. Furthermore, this comparison sheds light on the importance
of modularity in hierarchical architectures to solve complex goal-directed
tasks. We provide an open-source code implementation of our architecture
(https://github.com/wzaielamri/hddrl).
Related papers
- Hierarchical generative modelling for autonomous robots [8.023920215148486]
We show how a humanoid robot can autonomously complete a complex task that requires a holistic use of locomotion, manipulation, and grasping.
Specifically, we demonstrate the ability of a humanoid robot that can retrieve and transport a box, open and walk through a door to reach the destination, approach and kick a football, while showing robust performance in presence of body damage and ground irregularities.
arXiv Detail & Related papers (2023-08-15T13:51:03Z) - Creating Multi-Level Skill Hierarchies in Reinforcement Learning [0.0]
We propose an answer based on a graphical representation of how the interaction between an agent and its environment may unfold.
Our approach uses modularity maximisation as a central organising principle to expose the structure of the interaction graph at multiple levels of abstraction.
arXiv Detail & Related papers (2023-06-16T17:23:49Z) - Universal Morphology Control via Contextual Modulation [52.742056836818136]
Learning a universal policy across different robot morphologies can significantly improve learning efficiency and generalization in continuous control.
Existing methods utilize graph neural networks or transformers to handle heterogeneous state and action spaces across different morphologies.
We propose a hierarchical architecture to better model this dependency via contextual modulation.
arXiv Detail & Related papers (2023-02-22T00:04:12Z) - A Framework for Real-World Multi-Robot Systems Running Decentralized
GNN-Based Policies [4.40401067183266]
Graph Neural Networks (GNNs) are a paradigm-shifting neural architecture to facilitate the learning of complex multi-agent behaviors.
Recent work has demonstrated remarkable performance in tasks such as flocking, multi-agent path planning and cooperative coverage.
We present the design of a system that allows for fully decentralized execution of GNN-based policies.
arXiv Detail & Related papers (2021-11-02T17:53:54Z) - Graph Neural Networks for Decentralized Multi-Robot Submodular Action
Selection [101.38634057635373]
We focus on applications where robots are required to jointly select actions to maximize team submodular objectives.
We propose a general-purpose learning architecture towards submodular at scale, with decentralized communications.
We demonstrate the performance of our GNN-based learning approach in a scenario of active target coverage with large networks of robots.
arXiv Detail & Related papers (2021-05-18T15:32:07Z) - Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces.
We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space.
NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
arXiv Detail & Related papers (2020-12-04T18:59:32Z) - Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
We present a deep imitation learning framework for robotic bimanual manipulation.
A core challenge is to generalize the manipulation skills to objects in different locations.
We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
arXiv Detail & Related papers (2020-10-11T01:40:03Z) - Automated Search for Resource-Efficient Branched Multi-Task Networks [81.48051635183916]
We propose a principled approach, rooted in differentiable neural architecture search, to automatically define branching structures in a multi-task neural network.
We show that our approach consistently finds high-performing branching structures within limited resource budgets.
arXiv Detail & Related papers (2020-08-24T09:49:19Z) - Self-organization of multi-layer spiking neural networks [4.859525864236446]
A key mechanism that enables the formation of complex architecture in the developing brain is the emergence of traveling-temporal waves of neuronal activity.
We propose a modular tool-kit in the form of a dynamical system that can be seamlessly stacked to assemble multi-layer neural networks.
Our framework leads to the self-organization of a wide variety of architectures, ranging from multi-layer perceptrons to autoencoders.
arXiv Detail & Related papers (2020-06-12T01:44:48Z) - Decentralized Deep Reinforcement Learning for a Distributed and Adaptive
Locomotion Controller of a Hexapod Robot [0.6193838300896449]
We propose a decentralized organization as found in insect motor control for coordination of different legs.
A concurrent local structure is able to learn better walking behavior.
arXiv Detail & Related papers (2020-05-21T11:40:37Z) - F2A2: Flexible Fully-decentralized Approximate Actor-critic for
Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications.
We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting.
Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
arXiv Detail & Related papers (2020-04-17T14:56:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.