DeepCPG Policies for Robot Locomotion

• URL: http://arxiv.org/abs/2302.13191v1
• Date: Sat, 25 Feb 2023 23:16:57 GMT
• Title: DeepCPG Policies for Robot Locomotion
• Authors: Aditya M. Deshpande and Eric Hurd and Ali A. Minai and Manish Kumar
• Abstract summary: novel DeepCPG policies that embed CPGs as a layer in a larger neural network. We show that, compared to traditional approaches, DeepCPG policies allow sample-efficient end-to-end learning of effective locomotion strategies. Results suggest that gradual complexification with embedded priors of these policies in a modular fashion could achieve non-trivial sensor and motor integration on a robot platform.
• Score: 1.0057838324294686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Central Pattern Generators (CPGs) form the neural basis of the observed rhythmic behaviors for locomotion in legged animals. The CPG dynamics organized into networks allow the emergence of complex locomotor behaviors. In this work, we take this inspiration for developing walking behaviors in multi-legged robots. We present novel DeepCPG policies that embed CPGs as a layer in a larger neural network and facilitate end-to-end learning of locomotion behaviors in deep reinforcement learning (DRL) setup. We demonstrate the effectiveness of this approach on physics engine-based insectoid robots. We show that, compared to traditional approaches, DeepCPG policies allow sample-efficient end-to-end learning of effective locomotion strategies even in the case of high-dimensional sensor spaces (vision). We scale the DeepCPG policies using a modular robot configuration and multi-agent DRL. Our results suggest that gradual complexification with embedded priors of these policies in a modular fashion could achieve non-trivial sensor and motor integration on a robot platform. These results also indicate the efficacy of bootstrapping more complex intelligent systems from simpler ones based on biological principles. Finally, we present the experimental results for a proof-of-concept insectoid robot system for which DeepCPG learned policies initially using the simulation engine and these were afterwards transferred to real-world robots without any additional fine-tuning.

Related papers

• Astrocyte Regulated Neuromorphic Central Pattern Generator Control of Legged Robotic Locomotion [3.7814142008074954]
  This paper introduces an astrocyte regulated Spiking Neural Network (SNN)-based CPG for learning locomotion gait through Reward-Modulated STDP for quadruped robots. The SNN-based CPG is simulated on a multi-object physics simulation platform resulting in the emergence of a trotting gait while running the robot on flat ground. $23.3times$ computational power savings is observed in comparison to a state-of-the-art reinforcement learning based robot control algorithm.
  arXiv  Detail & Related papers  (2023-12-25T20:33:16Z)
• Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems [79.07468367923619]
  We propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC) We design an agent built completely from powerful predictive coding/processing circuits that facilitate dynamic, online learning from sparse rewards. We show that our proposed ActPC agent performs well in the face of sparse (extrinsic) reward signals and is competitive with or outperforms several powerful backprop-based RL approaches.
  arXiv  Detail & Related papers  (2022-09-19T16:49:32Z)
• Human-AI Shared Control via Frequency-based Policy Dissection [34.0399894373716]
  Human-AI shared control allows human to interact and collaborate with AI to accomplish control tasks in complex environments. Previous Reinforcement Learning (RL) methods attempt the goal-conditioned design to achieve human-controllable policies. We develop a simple yet effective frequency-based approach called textitPolicy Dissection to align the intermediate representation of the learned neural controller with the kinematic attributes of the agent behavior.
  arXiv  Detail & Related papers  (2022-05-31T23:57:55Z)
• Model Predictive Control for Fluid Human-to-Robot Handovers [50.72520769938633]
  Planning motions that take human comfort into account is not a part of the human-robot handover process. We propose to generate smooth motions via an efficient model-predictive control framework. We conduct human-to-robot handover experiments on a diverse set of objects with several users.
  arXiv  Detail & Related papers  (2022-03-31T23:08:20Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• GEM: Group Enhanced Model for Learning Dynamical Control Systems [78.56159072162103]
  We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
  arXiv  Detail & Related papers  (2021-04-07T01:08:18Z)
• An Energy-Saving Snake Locomotion Gait Policy Using Deep Reinforcement Learning [0.0]
  In this work, a snake locomotion gait policy is developed via deep reinforcement learning (DRL) for energy-efficient control. We apply proximal policy optimization (PPO) to each joint motor parameterized by angular velocity and the DRL agent learns the standard serpenoid curve at each timestep. Comparing to conventional control strategies, the snake robots controlled by the trained PPO agent can achieve faster movement and more energy-efficient locomotion gait.
  arXiv  Detail & Related papers  (2021-03-08T02:06:44Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)
• An Astrocyte-Modulated Neuromorphic Central Pattern Generator for Hexapod Robot Locomotion on Intel's Loihi [0.0]
  Locomotion is a crucial challenge for legged robots that is addressed "effortlessly" by biological networks abundant in nature, named central pattern generators (CPG) Here, we propose a brain-morphic CPG controler based on a comprehensive spiking neural-astrocytic network that generates two gait patterns for a hexapod robot. Our results pave the way for scaling this and other approaches towards Loihi-controlled locomotion in autonomous mobile robots.
  arXiv  Detail & Related papers  (2020-06-08T17:35:48Z)
• Populations of Spiking Neurons for Reservoir Computing: Closed Loop Control of a Compliant Quadruped [64.64924554743982]
  We present a framework for implementing central pattern generators with spiking neural networks to obtain closed loop robot control. We demonstrate the learning of predefined gait patterns, speed control and gait transition on a simulated model of a compliant quadrupedal robot.
  arXiv  Detail & Related papers  (2020-04-09T14:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.