Closed-form control with spike coding networks
- URL: http://arxiv.org/abs/2212.12887v2
- Date: Mon, 17 Jul 2023 09:12:06 GMT
- Title: Closed-form control with spike coding networks
- Authors: Filip S. Slijkhuis, Sander W. Keemink, Pablo Lanillos
- Abstract summary: Efficient and robust control using spiking neural networks (SNNs) is still an open problem.
We extend neuroscience theory of Spike Coding Networks (SCNs) by incorporating closed-form optimal estimation and control.
We demonstrate robust spiking control of simulated spring-mass-damper and cart-pole systems.
- Score: 1.1470070927586016
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Efficient and robust control using spiking neural networks (SNNs) is still an
open problem. Whilst behaviour of biological agents is produced through sparse
and irregular spiking patterns, which provide both robust and efficient
control, the activity patterns in most artificial spiking neural networks used
for control are dense and regular -- resulting in potentially less efficient
codes. Additionally, for most existing control solutions network training or
optimization is necessary, even for fully identified systems, complicating
their implementation in on-chip low-power solutions. The neuroscience theory of
Spike Coding Networks (SCNs) offers a fully analytical solution for
implementing dynamical systems in recurrent spiking neural networks -- while
maintaining irregular, sparse, and robust spiking activity -- but it's not
clear how to directly apply it to control problems. Here, we extend SCN theory
by incorporating closed-form optimal estimation and control. The resulting
networks work as a spiking equivalent of a linear-quadratic-Gaussian
controller. We demonstrate robust spiking control of simulated
spring-mass-damper and cart-pole systems, in the face of several perturbations,
including input- and system-noise, system disturbances, and neural silencing.
As our approach does not need learning or optimization, it offers opportunities
for deploying fast and efficient task-specific on-chip spiking controllers with
biologically realistic activity.
Related papers
- Communication-Control Codesign for Large-Scale Wireless Networked Control Systems [80.30532872347668]
Wireless Networked Control Systems (WNCSs) are essential to Industry 4.0, enabling flexible control in applications, such as drone swarms and autonomous robots.
We propose a practical WNCS model that captures correlated dynamics among multiple control loops with spatially distributed sensors and actuators sharing limited wireless resources over multi-state Markov block-fading channels.
We develop a Deep Reinforcement Learning (DRL) algorithm that efficiently handles the hybrid action space, captures communication-control correlations, and ensures robust training despite sparse cross-domain variables and floating control inputs.
arXiv Detail & Related papers (2024-10-15T06:28:21Z) - Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency.
In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution.
Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
arXiv Detail & Related papers (2024-04-05T17:58:37Z) - Fully Spiking Actor Network with Intra-layer Connections for
Reinforcement Learning [51.386945803485084]
We focus on the task where the agent needs to learn multi-dimensional deterministic policies to control.
Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected layer.
To develop a fully spiking actor network without any floating-point matrix operations, we draw inspiration from the non-spiking interneurons found in insects.
arXiv Detail & Related papers (2024-01-09T07:31:34Z) - How neural networks learn to classify chaotic time series [77.34726150561087]
We study the inner workings of neural networks trained to classify regular-versus-chaotic time series.
We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
arXiv Detail & Related papers (2023-06-04T08:53:27Z) - POLAR-Express: Efficient and Precise Formal Reachability Analysis of
Neural-Network Controlled Systems [18.369115196505657]
We present POLAR-Express, an efficient and precise formal reachability analysis tool for verifying the safety of neural-network controlled systems (NNCSs)
POLAR-Express uses Taylor model arithmetic to propagate Taylor models across a neural network layer-by-layer to compute an overapproximation of the neural-network function.
We also present a novel approach to propagate TMs more efficiently and precisely across ReLU activation functions.
arXiv Detail & Related papers (2023-03-31T06:51:36Z) - Pontryagin Optimal Control via Neural Networks [19.546571122359534]
We integrate Neural Networks with the Pontryagin's Maximum Principle (PMP), and propose a sample efficient framework NN-PMP-Gradient.
The resulting controller can be implemented for systems with unknown and complex dynamics.
Compared with the widely applied model-free and model-based reinforcement learning (RL) algorithms, our NN-PMP-Gradient achieves higher sample-efficiency and performance in terms of control objectives.
arXiv Detail & Related papers (2022-12-30T06:47:03Z) - Quantization-aware Interval Bound Propagation for Training Certifiably
Robust Quantized Neural Networks [58.195261590442406]
We study the problem of training and certifying adversarially robust quantized neural networks (QNNs)
Recent work has shown that floating-point neural networks that have been verified to be robust can become vulnerable to adversarial attacks after quantization.
We present quantization-aware interval bound propagation (QA-IBP), a novel method for training robust QNNs.
arXiv Detail & Related papers (2022-11-29T13:32:38Z) - Near-optimal control of dynamical systems with neural ordinary
differential equations [0.0]
Recent advances in deep learning and neural network-based optimization have contributed to the development of methods that can help solve control problems involving high-dimensional dynamical systems.
We first analyze how truncated and non-truncated backpropagation through time affect runtime performance and the ability of neural networks to learn optimal control functions.
arXiv Detail & Related papers (2022-06-22T14:11:11Z) - Distributed neural network control with dependability guarantees: a
compositional port-Hamiltonian approach [0.0]
Large-scale cyber-physical systems require that control policies are distributed, that is, that they only rely on local real-time measurements and communication with neighboring agents.
Recent work has proposed training Neural Network (NN) distributed controllers.
A main challenge of NN controllers is that they are not dependable during and after training, that is, the closed-loop system may be unstable, and the training may fail due to vanishing and exploding gradients.
arXiv Detail & Related papers (2021-12-16T17:37:11Z) - Generating Probabilistic Safety Guarantees for Neural Network
Controllers [30.34898838361206]
We use a dynamics model to determine the output properties that must hold for a neural network controller to operate safely.
We develop an adaptive verification approach to efficiently generate an overapproximation of the neural network policy.
We show that our method is able to generate meaningful probabilistic safety guarantees for aircraft collision avoidance neural networks.
arXiv Detail & Related papers (2021-03-01T18:48:21Z) - Decentralized Control with Graph Neural Networks [147.84766857793247]
We propose a novel framework using graph neural networks (GNNs) to learn decentralized controllers.
GNNs are well-suited for the task since they are naturally distributed architectures and exhibit good scalability and transferability properties.
The problems of flocking and multi-agent path planning are explored to illustrate the potential of GNNs in learning decentralized controllers.
arXiv Detail & Related papers (2020-12-29T18:59:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.