DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body

• URL: http://arxiv.org/abs/2209.14218v1
• Date: Wed, 28 Sep 2022 16:45:35 GMT
• Title: DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body
• Authors: Alberto Silvio Chiappa and Alessandro Marin Vargas and Alexander Mathis
• Abstract summary: We introduce DMAP, a biologically-inspired, attention-based policy network architecture. We show that a control policy based on the proprioceptive state performs poorly with highly variable body configurations. DMAP can be trained end-to-end in all the considered environments, overall matching or surpassing the performance of an oracle agent.
• Score: 126.52031472297413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Biological and artificial agents need to deal with constant changes in the real world. We study this problem in four classical continuous control environments, augmented with morphological perturbations. Learning to locomote when the length and the thickness of different body parts vary is challenging, as the control policy is required to adapt to the morphology to successfully balance and advance the agent. We show that a control policy based on the proprioceptive state performs poorly with highly variable body configurations, while an (oracle) agent with access to a learned encoding of the perturbation performs significantly better. We introduce DMAP, a biologically-inspired, attention-based policy network architecture. DMAP combines independent proprioceptive processing, a distributed policy with individual controllers for each joint, and an attention mechanism, to dynamically gate sensory information from different body parts to different controllers. Despite not having access to the (hidden) morphology information, DMAP can be trained end-to-end in all the considered environments, overall matching or surpassing the performance of an oracle agent. Thus DMAP, implementing principles from biological motor control, provides a strong inductive bias for learning challenging sensorimotor tasks. Overall, our work corroborates the power of these principles in challenging locomotion tasks.

Related papers

• Evolving generalist controllers to handle a wide range of morphological variations [1.4425878137951238]
  The study of robustness and generalizability of artificial neural networks (ANNs) has remained limited. Unexpected morphological or environmental changes during operation can risk failure if the ANN controllers are unable to handle these changes. This paper proposes an algorithm that aims to enhance the robustness and generalizability of the controllers.
  arXiv  Detail & Related papers  (2023-09-18T23:06:19Z)
• Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence [59.11038175596807]
  Continual learning aims to empower artificial intelligence with strong adaptability to the real world. Existing advances mainly focus on preserving memory stability to overcome catastrophic forgetting. We propose a generic approach that appropriately attenuates old memories in parameter distributions to improve learning plasticity.
  arXiv  Detail & Related papers  (2023-08-29T02:43:58Z)
• SAR: Generalization of Physiological Agility and Dexterity via Synergistic Action Representation [10.349135207285464]
  We show that modular control via muscle synergies enables organisms to learn muscle control in a simplified and generalizable action space. We use physiologically accurate human hand and leg models as a testbed for determining the extent to which a Synergistic Action Representation (SAR) acquired from simpler tasks facilitates learning more complex tasks. We find in both cases that SAR-exploiting policies significantly outperform end-to-end reinforcement learning.
  arXiv  Detail & Related papers  (2023-07-07T17:07:41Z)
• Latent Exploration for Reinforcement Learning [87.42776741119653]
  In Reinforcement Learning, agents learn policies by exploring and interacting with the environment. We propose LATent TIme-Correlated Exploration (Lattice), a method to inject temporally-correlated noise into the latent state of the policy network.
  arXiv  Detail & Related papers  (2023-05-31T17:40:43Z)
• Universal Morphology Control via Contextual Modulation [52.742056836818136]
  Learning a universal policy across different robot morphologies can significantly improve learning efficiency and generalization in continuous control. Existing methods utilize graph neural networks or transformers to handle heterogeneous state and action spaces across different morphologies. We propose a hierarchical architecture to better model this dependency via contextual modulation.
  arXiv  Detail & Related papers  (2023-02-22T00:04:12Z)
• MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control [7.856809409051587]
  MyoSuite is a suite of physiologically accurate biomechanical models of elbow, wrist, and hand, with physical contact capabilities. We provide diverse motor-control challenges: from simple postural control to skilled hand-object interactions.
  arXiv  Detail & Related papers  (2022-05-26T20:11:23Z)
• Collective control of modular soft robots via embodied Spiking Neural Cellular Automata [1.0323063834827415]
  Voxel-based Soft Robots (VSRs) are a form of modular soft robots, composed of several deformable cubes, i.e., voxels. We propose a novel form of collective control, influenced by Neural Cellular Automata (NCA) and based on the bio-inspired Spiking Neural Networks: the embodied Spiking NCA.
  arXiv  Detail & Related papers  (2022-04-05T10:42:57Z)
• Learning Reactive and Predictive Differentiable Controllers for Switching Linear Dynamical Models [7.653542219337937]
  We present a framework for learning composite dynamical behaviors from expert demonstrations. We learn a switching linear dynamical model with contacts encoded in switching conditions as a close approximation of our system dynamics. We then use discrete-time LQR as the differentiable policy class for data-efficient learning of control to develop a control strategy.
  arXiv  Detail & Related papers  (2021-03-26T04:40:24Z)
• One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control [47.78262874364569]
  We investigate whether there exists a single global policy that can generalize to control a wide variety of agent morphologies. We propose to express this global policy as a collection of identical modular neural networks. We show that a single modular policy can successfully generate locomotion behaviors for several planar agents.
  arXiv  Detail & Related papers  (2020-07-09T17:59:35Z)
• Learning Compliance Adaptation in Contact-Rich Manipulation [81.40695846555955]
  We propose a novel approach for learning predictive models of force profiles required for contact-rich tasks. The approach combines an anomaly detection based on Bidirectional Gated Recurrent Units (Bi-GRU) and an adaptive force/impedance controller.
  arXiv  Detail & Related papers  (2020-05-01T05:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.