Training slow silicon neurons to control extremely fast robots with spiking reinforcement learning

• URL: http://arxiv.org/abs/2601.21548v1
• Date: Thu, 29 Jan 2026 11:05:23 GMT
• Title: Training slow silicon neurons to control extremely fast robots with spiking reinforcement learning
• Authors: Irene Ambrosini, Ingo Blakowski, Dmitrii Zendrikov, Cristiano Capone, Luna Gava, Giacomo Indiveri, Chiara De Luca, Chiara Bartolozzi,
• Abstract summary: A network of spiking neurons runs on a mixed-signal analog/digital neuromorphic processor.<n>We train the system to achieve successful puck interactions through reinforcement learning in a small number of trials.<n>This work bridges neuroscience-inspired hardware with real-world robotic control, showing that brain-inspired approaches can tackle fast-paced interaction tasks.
• Score: 7.709259343105974
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Air hockey demands split-second decisions at high puck velocities, a challenge we address with a compact network of spiking neurons running on a mixed-signal analog/digital neuromorphic processor. By co-designing hardware and learning algorithms, we train the system to achieve successful puck interactions through reinforcement learning in a remarkably small number of trials. The network leverages fixed random connectivity to capture the task's temporal structure and adopts a local e-prop learning rule in the readout layer to exploit event-driven activity for fast and efficient learning. The result is real-time learning with a setup comprising a computer and the neuromorphic chip in-the-loop, enabling practical training of spiking neural networks for robotic autonomous systems. This work bridges neuroscience-inspired hardware with real-world robotic control, showing that brain-inspired approaches can tackle fast-paced interaction tasks while supporting always-on learning in intelligent machines.

Related papers

• Neuromorphic dreaming: A pathway to efficient learning in artificial agents [2.6542148964152923]
  We present a hardware implementation of model-based reinforcement learning (MBRL) using spiking neural networks (SNNs) on mixed-signal analog/digital neuromorphic hardware. This approach leverages the energy efficiency of mixed-signal neuromorphic chips while achieving high sample efficiency. We validate the model by training the hardware implementation to play the Atari game Pong.
  arXiv  Detail & Related papers  (2024-05-24T15:03:56Z)
• LPAC: Learnable Perception-Action-Communication Loops with Applications to Coverage Control [72.81786007015471]
  We propose a learnable Perception-Action-Communication (LPAC) architecture for the problem.<n>CNN processes localized perception; a graph neural network (GNN) facilitates robot communications.<n> Evaluations show that the LPAC models outperform standard decentralized and centralized coverage control algorithms.
  arXiv  Detail & Related papers  (2024-01-10T00:08:00Z)
• Fully Spiking Neural Network for Legged Robots [6.974746966671198]
  Spiking Neural Network (SNN) for legged robots shows exceptional performance in simulated terrains. SNNs provide natural advantages in inference speed and energy consumption. This study presents a highly efficient SNN for legged robots that can be seamless integrated into other learning models.
  arXiv  Detail & Related papers  (2023-10-08T05:48:30Z)
• Incremental procedural and sensorimotor learning in cognitive humanoid robots [52.77024349608834]
  This work presents a cognitive agent that can learn procedures incrementally. We show the cognitive functions required in each substage and how adding new functions helps address tasks previously unsolved by the agent. Results show that this approach is capable of solving complex tasks incrementally.
  arXiv  Detail & Related papers  (2023-04-30T22:51:31Z)
• Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance [71.36749876465618]
  We describe a system for vision-based dexterous manipulation that provides a "programming-free" approach for users to define new tasks. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples. experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world.
  arXiv  Detail & Related papers  (2022-12-19T22:50:40Z)
• Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems [79.07468367923619]
  We propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC) We design an agent built completely from powerful predictive coding/processing circuits that facilitate dynamic, online learning from sparse rewards. We show that our proposed ActPC agent performs well in the face of sparse (extrinsic) reward signals and is competitive with or outperforms several powerful backprop-based RL approaches.
  arXiv  Detail & Related papers  (2022-09-19T16:49:32Z)
• Versatile modular neural locomotion control with fast learning [6.85316573653194]
  Legged robots have significant potential to operate in highly unstructured environments. Currently, controllers must be either manually designed for specific robots or automatically designed via machine learning methods. We propose a simple yet versatile modular neural control structure with fast learning.
  arXiv  Detail & Related papers  (2021-07-16T12:12:28Z)
• Task-relevant Representation Learning for Networked Robotic Perception [74.0215744125845]
  This paper presents an algorithm to learn task-relevant representations of sensory data that are co-designed with a pre-trained robotic perception model's ultimate objective. Our algorithm aggressively compresses robotic sensory data by up to 11x more than competing methods.
  arXiv  Detail & Related papers  (2020-11-06T07:39:08Z)
• Robust trajectory generation for robotic control on the neuromorphic research chip Loihi [0.0]
  We exploit a recently developed spiking neural network model, the so-called anisotropic network. We show that the anisotropic network on Loihi reliably encodes sequential patterns of neural activity. Taken together, our study presents a new algorithm that allows the generation of complex robotic movements.
  arXiv  Detail & Related papers  (2020-08-26T16:02:39Z)
• Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot [0.6193838300896449]
  We propose a decentralized organization as found in insect motor control for coordination of different legs. A concurrent local structure is able to learn better walking behavior.
  arXiv  Detail & Related papers  (2020-05-21T11:40:37Z)
• Learning Agile Robotic Locomotion Skills by Imitating Animals [72.36395376558984]
  Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics. We present an imitation learning system that enables legged robots to learn agile locomotion skills by imitating real-world animals.
  arXiv  Detail & Related papers  (2020-04-02T02:56:16Z)
• Self learning robot using real-time neural networks [7.347989843033033]
  This paper involves research, development and experimental analysis of a neural network implemented on a robot with an arm. The neural network learns using the algorithms of Gradient Descent and Backpropagation. Both the implementation and training of the neural network is done locally on the robot on a raspberry pi 3 so that its learning process is completely independent.
  arXiv  Detail & Related papers  (2020-01-06T13:13:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.