One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control

• URL: http://arxiv.org/abs/2007.04976v1
• Date: Thu, 9 Jul 2020 17:59:35 GMT
• Title: One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control
• Authors: Wenlong Huang, Igor Mordatch, Deepak Pathak
• Abstract summary: We investigate whether there exists a single global policy that can generalize to control a wide variety of agent morphologies. We propose to express this global policy as a collection of identical modular neural networks. We show that a single modular policy can successfully generate locomotion behaviors for several planar agents.
• Score: 47.78262874364569
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning is typically concerned with learning control policies tailored to a particular agent. We investigate whether there exists a single global policy that can generalize to control a wide variety of agent morphologies -- ones in which even dimensionality of state and action spaces changes. We propose to express this global policy as a collection of identical modular neural networks, dubbed as Shared Modular Policies (SMP), that correspond to each of the agent's actuators. Every module is only responsible for controlling its corresponding actuator and receives information from only its local sensors. In addition, messages are passed between modules, propagating information between distant modules. We show that a single modular policy can successfully generate locomotion behaviors for several planar agents with different skeletal structures such as monopod hoppers, quadrupeds, bipeds, and generalize to variants not seen during training -- a process that would normally require training and manual hyperparameter tuning for each morphology. We observe that a wide variety of drastically diverse locomotion styles across morphologies as well as centralized coordination emerges via message passing between decentralized modules purely from the reinforcement learning objective. Videos and code at https://huangwl18.github.io/modular-rl/

Related papers

• Is Modularity Transferable? A Case Study through the Lens of Knowledge Distillation [59.37775534633868]
  We present an extremely straightforward approach to transferring pre-trained, task-specific PEFT modules between same-family PLMs. We also propose a method that allows the transfer of modules between incompatible PLMs without any change in the inference complexity.
  arXiv  Detail & Related papers  (2024-03-27T17:50:00Z)
• FedYolo: Augmenting Federated Learning with Pretrained Transformers [61.56476056444933]
  In this work, we investigate pretrained transformers (PTF) to achieve on-device learning goals. We show that larger scale shrinks the accuracy gaps between alternative approaches and improves robustness. Finally, it enables clients to solve multiple unrelated tasks simultaneously using a single PTF.
  arXiv  Detail & Related papers  (2023-07-10T21:08:52Z)
• Modular Deep Learning [120.36599591042908]
  Transfer learning has recently become the dominant paradigm of machine learning. It remains unclear how to develop models that specialise towards multiple tasks without incurring negative interference. Modular deep learning has emerged as a promising solution to these challenges.
  arXiv  Detail & Related papers  (2023-02-22T18:11:25Z)
• DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body [126.52031472297413]
  We introduce DMAP, a biologically-inspired, attention-based policy network architecture. We show that a control policy based on the proprioceptive state performs poorly with highly variable body configurations. DMAP can be trained end-to-end in all the considered environments, overall matching or surpassing the performance of an oracle agent.
  arXiv  Detail & Related papers  (2022-09-28T16:45:35Z)
• Behavior Trees in Robot Control Systems [0.0]
  Key idea underlying behavior trees is to make use of modularity, hierarchies and feedback. A hierarchy of such modules is natural, since robot tasks can often be decomposed into a hierarchy of sub-tasks. feedback control is a fundamental tool for handling uncertainties and disturbances in any low level control system.
  arXiv  Detail & Related papers  (2022-03-24T14:16:15Z)
• Learning Modular Robot Control Policies [10.503109190599828]
  We construct a modular control policy that handles a broad class of designs. As the modules are physically re-configured, the policy automatically re-configures to match the kinematic structure. We show that the policy can then generalize to a larger set of designs not seen during training.
  arXiv  Detail & Related papers  (2021-05-20T21:54:37Z)
• Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers [84.57980167400513]
  Neural Function Modules (NFM) aims to introduce the same structural capability into deep learning. Most of the work in the context of feed-forward networks combining top-down and bottom-up feedback is limited to classification problems. The key contribution of our work is to combine attention, sparsity, top-down and bottom-up feedback, in a flexible algorithm.
  arXiv  Detail & Related papers  (2020-10-15T20:43:17Z)
• Adapting to Unseen Environments through Explicit Representation of Context [16.8615211682877]
  In order to deploy autonomous agents to domains such as autonomous driving, infrastructure management, health care, and finance, they must be able to adapt safely to unseen situations. This paper proposes a principled approach where a context module is coevolved with a skill module. The Context+Skill approach leads to significantly more robust behavior in environments with previously unseen effects.
  arXiv  Detail & Related papers  (2020-02-13T17:15:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.