Learning to Walk: Spike Based Reinforcement Learning for Hexapod Robot Central Pattern Generation

• URL: http://arxiv.org/abs/2003.10026v1
• Date: Sun, 22 Mar 2020 23:45:32 GMT
• Title: Learning to Walk: Spike Based Reinforcement Learning for Hexapod Robot Central Pattern Generation
• Authors: Ashwin Sanjay Lele, Yan Fang, Justin Ting, Arijit Raychowdhury
• Abstract summary: Methods such as gradient, deep reinforcement learning (RL) have been explored for bipeds, quadrupeds and hexapods. Recent advances in spiking neural networks (SNNs) promise a significant reduction in computing owing to the sparse firing of neuros. We propose a reinforcement based weight update technique for training a spiking pattern generator.
• Score: 2.4603302139672003
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning to walk -- i.e., learning locomotion under performance and energy constraints continues to be a challenge in legged robotics. Methods such as stochastic gradient, deep reinforcement learning (RL) have been explored for bipeds, quadrupeds and hexapods. These techniques are computationally intensive and often prohibitive for edge applications. These methods rely on complex sensors and pre-processing of data, which further increases energy and latency. Recent advances in spiking neural networks (SNNs) promise a significant reduction in computing owing to the sparse firing of neuros and has been shown to integrate reinforcement learning mechanisms with biologically observed spike time dependent plasticity (STDP). However, training a legged robot to walk by learning the synchronization patterns of central pattern generators (CPG) in an SNN framework has not been shown. This can marry the efficiency of SNNs with synchronized locomotion of CPG based systems providing breakthrough end-to-end learning in mobile robotics. In this paper, we propose a reinforcement based stochastic weight update technique for training a spiking CPG. The whole system is implemented on a lightweight raspberry pi platform with integrated sensors, thus opening up exciting new possibilities.

Related papers

• Adaptive Robotic Arm Control with a Spiking Recurrent Neural Network on a Digital Accelerator [41.60361484397962]
  We present an overview of the system, and a Python framework to use it on a Pynq ZU platform. We show how the simulated accuracy is preserved with a peak performance of 3.8M events processed per second.
  arXiv  Detail & Related papers  (2024-05-21T14:59:39Z)
• Astrocyte Regulated Neuromorphic Central Pattern Generator Control of Legged Robotic Locomotion [3.7814142008074954]
  This paper introduces an astrocyte regulated Spiking Neural Network (SNN)-based CPG for learning locomotion gait through Reward-Modulated STDP for quadruped robots. The SNN-based CPG is simulated on a multi-object physics simulation platform resulting in the emergence of a trotting gait while running the robot on flat ground. $23.3times$ computational power savings is observed in comparison to a state-of-the-art reinforcement learning based robot control algorithm.
  arXiv  Detail & Related papers  (2023-12-25T20:33:16Z)
• SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence [51.6943465041708]
  Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency. We contribute a full-stack toolkit for pre-processing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips.
  arXiv  Detail & Related papers  (2023-10-25T13:15:17Z)
• Neuromorphic analog circuits for robust on-chip always-on learning in spiking neural networks [1.9809266426888898]
  Mixed-signal neuromorphic systems represent a promising solution for solving extreme-edge computing tasks. Their spiking neural network circuits are optimized for processing sensory data on-line in continuous-time. We design on-chip learning circuits with short-term analog dynamics and long-term tristate discretization mechanisms.
  arXiv  Detail & Related papers  (2023-07-12T11:14:25Z)
• Evolving Connectivity for Recurrent Spiking Neural Networks [8.80300633999542]
  Recurrent neural networks (RSNNs) hold great potential for advancing artificial general intelligence. We propose the evolving connectivity (EC) framework, an inference-only method for training RSNNs.
  arXiv  Detail & Related papers  (2023-05-28T07:08:25Z)
• SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks [56.35403810762512]
  Spiking neural networks (SNNs) with event-based computation are promising brain-inspired models for energy-efficient applications on neuromorphic hardware. We study spike-based implicit differentiation on the equilibrium state (SPIDE) that extends the recently proposed training method.
  arXiv  Detail & Related papers  (2023-02-01T04:22:59Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Online Training Through Time for Spiking Neural Networks [66.7744060103562]
  Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Recent progress in training methods has enabled successful deep SNNs on large-scale tasks with low latency. We propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-time learning.
  arXiv  Detail & Related papers  (2022-10-09T07:47:56Z)
• Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems [79.07468367923619]
  We propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC) We design an agent built completely from powerful predictive coding/processing circuits that facilitate dynamic, online learning from sparse rewards. We show that our proposed ActPC agent performs well in the face of sparse (extrinsic) reward signals and is competitive with or outperforms several powerful backprop-based RL approaches.
  arXiv  Detail & Related papers  (2022-09-19T16:49:32Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)
• Indirect and Direct Training of Spiking Neural Networks for End-to-End Control of a Lane-Keeping Vehicle [12.137685936113384]
  Building spiking neural networks (SNNs) based on biological synaptic plasticities holds a promising potential for accomplishing fast and energy-efficient computing. In this paper, we introduce both indirect and direct end-to-end training methods of SNNs for a lane-keeping vehicle.
  arXiv  Detail & Related papers  (2020-03-10T09:35:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.