Hierarchical Decentralized Deep Reinforcement Learning Architecture for a Simulated Four-Legged Agent

• URL: http://arxiv.org/abs/2210.08003v1
• Date: Wed, 21 Sep 2022 07:55:33 GMT
• Title: Hierarchical Decentralized Deep Reinforcement Learning Architecture for a Simulated Four-Legged Agent
• Authors: W. Zai El Amri and L. Hermes and M. Schilling
• Abstract summary: In nature, control of movement happens in a hierarchical and decentralized fashion. We present a novel decentral, hierarchical architecture to control a simulated legged agent.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Legged locomotion is widespread in nature and has inspired the design of current robots. The controller of these legged robots is often realized as one centralized instance. However, in nature, control of movement happens in a hierarchical and decentralized fashion. Introducing these biological design principles into robotic control systems has motivated this work. We tackle the question whether decentralized and hierarchical control is beneficial for legged robots and present a novel decentral, hierarchical architecture to control a simulated legged agent. Three different tasks varying in complexity are designed to benchmark five architectures (centralized, decentralized, hierarchical and two different combinations of hierarchical decentralized architectures). The results demonstrate that decentralizing the different levels of the hierarchical architectures facilitates learning of the agent, ensures more energy efficient movements as well as robustness towards new unseen environments. Furthermore, this comparison sheds light on the importance of modularity in hierarchical architectures to solve complex goal-directed tasks. We provide an open-source code implementation of our architecture (https://github.com/wzaielamri/hddrl).

Related papers

• EM-DARTS: Hierarchical Differentiable Architecture Search for Eye Movement Recognition [54.99121380536659]
  Eye movement biometrics have received increasing attention thanks to its high secure identification. Deep learning (DL) models have been recently successfully applied for eye movement recognition. DL architecture still is determined by human prior knowledge. We propose EM-DARTS, a hierarchical differentiable architecture search algorithm to automatically design the DL architecture for eye movement recognition.
  arXiv  Detail & Related papers  (2024-09-22T13:11:08Z)
• Body Transformer: Leveraging Robot Embodiment for Policy Learning [51.531793239586165]
  Body Transformer (BoT) is an architecture that leverages the robot embodiment by providing an inductive bias that guides the learning process. We represent the robot body as a graph of sensors and actuators, and rely on masked attention to pool information throughout the architecture. The resulting architecture outperforms the vanilla transformer, as well as the classical multilayer perceptron, in terms of task completion, scaling properties, and computational efficiency.
  arXiv  Detail & Related papers  (2024-08-12T17:31:28Z)
• Hierarchical learning control for autonomous robots inspired by central nervous system [7.227887302864789]
  We propose a novel hierarchical learning control framework by mimicking the hierarchical structure of the central nervous system. The framework combines the active and passive control systems to improve both the flexibility and reliability of the control system. This study reveals the principle that governs the autonomous behavior in the central nervous system and demonstrates the effectiveness of the hierarchical control approach.
  arXiv  Detail & Related papers  (2024-08-07T03:24:59Z)
• Hierarchical generative modelling for autonomous robots [8.023920215148486]
  We show how a humanoid robot can autonomously complete a complex task that requires a holistic use of locomotion, manipulation, and grasping. Specifically, we demonstrate the ability of a humanoid robot that can retrieve and transport a box, open and walk through a door to reach the destination, approach and kick a football, while showing robust performance in presence of body damage and ground irregularities.
  arXiv  Detail & Related papers  (2023-08-15T13:51:03Z)
• A Framework for Real-World Multi-Robot Systems Running Decentralized GNN-Based Policies [4.40401067183266]
  Graph Neural Networks (GNNs) are a paradigm-shifting neural architecture to facilitate the learning of complex multi-agent behaviors. Recent work has demonstrated remarkable performance in tasks such as flocking, multi-agent path planning and cooperative coverage. We present the design of a system that allows for fully decentralized execution of GNN-based policies.
  arXiv  Detail & Related papers  (2021-11-02T17:53:54Z)
• Graph Neural Networks for Decentralized Multi-Robot Submodular Action Selection [101.38634057635373]
  We focus on applications where robots are required to jointly select actions to maximize team submodular objectives. We propose a general-purpose learning architecture towards submodular at scale, with decentralized communications. We demonstrate the performance of our GNN-based learning approach in a scenario of active target coverage with large networks of robots.
  arXiv  Detail & Related papers  (2021-05-18T15:32:07Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)
• Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
  We present a deep imitation learning framework for robotic bimanual manipulation. A core challenge is to generalize the manipulation skills to objects in different locations. We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
  arXiv  Detail & Related papers  (2020-10-11T01:40:03Z)
• Automated Search for Resource-Efficient Branched Multi-Task Networks [81.48051635183916]
  We propose a principled approach, rooted in differentiable neural architecture search, to automatically define branching structures in a multi-task neural network. We show that our approach consistently finds high-performing branching structures within limited resource budgets.
  arXiv  Detail & Related papers  (2020-08-24T09:49:19Z)
• Self-organization of multi-layer spiking neural networks [4.859525864236446]
  A key mechanism that enables the formation of complex architecture in the developing brain is the emergence of traveling-temporal waves of neuronal activity. We propose a modular tool-kit in the form of a dynamical system that can be seamlessly stacked to assemble multi-layer neural networks. Our framework leads to the self-organization of a wide variety of architectures, ranging from multi-layer perceptrons to autoencoders.
  arXiv  Detail & Related papers  (2020-06-12T01:44:48Z)
• Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot [0.6193838300896449]
  We propose a decentralized organization as found in insect motor control for coordination of different legs. A concurrent local structure is able to learn better walking behavior.
  arXiv  Detail & Related papers  (2020-05-21T11:40:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.