Backprop-Free Reinforcement Learning with Active Neural Generative Coding

• URL: http://arxiv.org/abs/2107.07046v1
• Date: Sat, 10 Jul 2021 19:02:27 GMT
• Title: Backprop-Free Reinforcement Learning with Active Neural Generative Coding
• Authors: Alexander Ororbia, Ankur Mali
• Abstract summary: We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
• Score: 84.11376568625353
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In humans, perceptual awareness facilitates the fast recognition and extraction of information from sensory input. This awareness largely depends on how the human agent interacts with the environment. In this work, we propose active neural generative coding, a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. Specifically, we develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. We demonstrate on several control problems, in the online learning setting, that our proposed modeling framework performs competitively with deep Q-learning models. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.

Related papers

• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Manipulating Feature Visualizations with Gradient Slingshots [54.31109240020007]
  We introduce a novel method for manipulating Feature Visualization (FV) without significantly impacting the model's decision-making process. We evaluate the effectiveness of our method on several neural network models and demonstrate its capabilities to hide the functionality of arbitrarily chosen neurons.
  arXiv  Detail & Related papers  (2024-01-11T18:57:17Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Reinforcement Learning in an Adaptable Chess Environment for Detecting Human-understandable Concepts [0.0]
  We show a method for probing which concepts self-learning agents internalise in the course of their training. For demonstration, we use a chess playing agent in a fast and light environment developed specifically to be suitable for research groups.
  arXiv  Detail & Related papers  (2022-11-10T11:48:10Z)
• Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems [79.07468367923619]
  We propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC) We design an agent built completely from powerful predictive coding/processing circuits that facilitate dynamic, online learning from sparse rewards. We show that our proposed ActPC agent performs well in the face of sparse (extrinsic) reward signals and is competitive with or outperforms several powerful backprop-based RL approaches.
  arXiv  Detail & Related papers  (2022-09-19T16:49:32Z)
• Inference of Affordances and Active Motor Control in Simulated Agents [0.5161531917413706]
  We introduce an output-probabilistic, temporally predictive, modular artificial neural network architecture. We show that our architecture develops latent states that can be interpreted as affordance maps. In combination with active inference, we show that flexible, goal-directed behavior can be invoked.
  arXiv  Detail & Related papers  (2022-02-23T14:13:04Z)
• A neural net architecture based on principles of neural plasticity and development evolves to effectively catch prey in a simulated environment [2.834895018689047]
  A profound challenge for A-Life is to construct agents whose behavior is 'life-like' in a deep way. We propose an architecture and approach to constructing networks driving artificial agents, using processes analogous to the processes that construct and sculpt the brains of animals. We think this architecture may be useful for controlling small autonomous robots or drones, because it allows for a rapid response to changes in sensor inputs.
  arXiv  Detail & Related papers  (2022-01-28T05:10:56Z)
• Generative Adversarial Reward Learning for Generalized Behavior Tendency Inference [71.11416263370823]
  We propose a generative inverse reinforcement learning for user behavioral preference modelling. Our model can automatically learn the rewards from user's actions based on discriminative actor-critic network and Wasserstein GAN.
  arXiv  Detail & Related papers  (2021-05-03T13:14:25Z)
• Uncertainty-based Modulation for Lifelong Learning [1.3334365645271111]
  We present an algorithm inspired by neuromodulatory mechanisms in the human brain that integrates and expands upon Stephen Grossberg's Adaptive Resonance Theory proposals. Specifically, it builds on the concept of uncertainty, and employs a series of neuromodulatory mechanisms to enable continuous learning. We demonstrate the critical role of developing these systems in a closed-loop manner where the environment and the agent's behaviors constrain and guide the learning process.
  arXiv  Detail & Related papers  (2020-01-27T14:34:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.