Neural optimal feedback control with local learning rules

• URL: http://arxiv.org/abs/2111.06920v1
• Date: Fri, 12 Nov 2021 20:02:00 GMT
• Title: Neural optimal feedback control with local learning rules
• Authors: Johannes Friedrich, Siavash Golkar, Shiva Farashahi, Alexander Genkin, Anirvan M. Sengupta, Dmitri B. Chklovskii
• Abstract summary: A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. We introduce a novel online algorithm which combines adaptive Kalman filtering with a model free control approach.
• Score: 67.5926699124528
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. A prominent framework for addressing such control problems is Optimal Feedback Control (OFC). OFC generates control actions that optimize behaviorally relevant criteria by integrating noisy sensory stimuli and the predictions of an internal model using the Kalman filter or its extensions. However, a satisfactory neural model of Kalman filtering and control is lacking because existing proposals have the following limitations: not considering the delay of sensory feedback, training in alternating phases, and requiring knowledge of the noise covariance matrices, as well as that of systems dynamics. Moreover, the majority of these studies considered Kalman filtering in isolation, and not jointly with control. To address these shortcomings, we introduce a novel online algorithm which combines adaptive Kalman filtering with a model free control approach (i.e., policy gradient algorithm). We implement this algorithm in a biologically plausible neural network with local synaptic plasticity rules. This network performs system identification and Kalman filtering, without the need for multiple phases with distinct update rules or the knowledge of the noise covariances. It can perform state estimation with delayed sensory feedback, with the help of an internal model. It learns the control policy without requiring any knowledge of the dynamics, thus avoiding the need for weight transport. In this way, our implementation of OFC solves the credit assignment problem needed to produce the appropriate sensory-motor control in the presence of stimulus delay.

Related papers

• Distributed Leader Follower Formation Control of Mobile Robots based on Bioinspired Neural Dynamics and Adaptive Sliding Innovation Filter [14.66072990853587]
  We propose a bioinspired neural dynamic based backstepping and sliding mode control hybrid formation control method. The proposed control strategy resolves the impractical speed jump issue that exists in the conventional backstepping design. We performed multiple simulations to demonstrate the efficiency and effectiveness of the proposed formation control strategy.
  arXiv  Detail & Related papers  (2023-05-03T17:29:46Z)
• Neuromorphic Control using Input-Weighted Threshold Adaptation [13.237124392668573]
  It is still challenging to replicate even basic low-level controllers such as proportional-integral-derivative (PID) controllers. We propose a neuromorphic controller that incorporates proportional, integral, and derivative pathways during learning. We demonstrate the stability of our bio-inspired algorithm with flights in the presence of disturbances.
  arXiv  Detail & Related papers  (2023-04-18T07:21:24Z)
• Incorporating Recurrent Reinforcement Learning into Model Predictive Control for Adaptive Control in Autonomous Driving [11.67417895998434]
  Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique. In this paper, we reformulate the problem as a Partially Observed Markov Decision Process (POMDP) We then learn a recurrent policy continually adapting the parameters of the dynamics model via Recurrent Reinforcement Learning (RRL) for optimal and adaptive control.
  arXiv  Detail & Related papers  (2023-01-30T22:11:07Z)
• Design of a Supervisory Control System for Autonomous Operation of Advanced Reactors [0.0]
  This work focuses on the control aspect of autonomous operation. Within the system, data-driven modeling, physics-based state observation, and classical control algorithms are integrated. A 320 MW Fluoride-cooled High-temperature Pebble-bed Reactor is the design basis for demonstrating the control system.
  arXiv  Detail & Related papers  (2022-09-09T14:48:34Z)
• Adaptation through prediction: multisensory active inference torque control [0.0]
  We present a novel multisensory active inference torque controller for industrial arms. Our controller, inspired by the predictive brain hypothesis, improves the capabilities of current active inference approaches.
  arXiv  Detail & Related papers  (2021-12-13T16:03:18Z)
• Adaptive Low-Pass Filtering using Sliding Window Gaussian Processes [71.23286211775084]
  We propose an adaptive low-pass filter based on Gaussian process regression. We show that the estimation error of the proposed method is uniformly bounded.
  arXiv  Detail & Related papers  (2021-11-05T17:06:59Z)
• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics [84.18625250574853]
  We present KalmanNet, a real-time state estimator that learns from data to carry out Kalman filtering under non-linear dynamics. We numerically demonstrate that KalmanNet overcomes nonlinearities and model mismatch, outperforming classic filtering methods.
  arXiv  Detail & Related papers  (2021-07-21T12:26:46Z)
• Neural Kalman Filtering [62.997667081978825]
  We show that a gradient-descent approximation to the Kalman filter requires only local computations with variance weighted prediction errors. We also show that it is possible under the same scheme to adaptively learn the dynamics model with a learning rule that corresponds directly to Hebbian plasticity.
  arXiv  Detail & Related papers  (2021-02-19T16:43:15Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.