Biological Neurons Compete with Deep Reinforcement Learning in Sample Efficiency in a Simulated Gameworld

• URL: http://arxiv.org/abs/2405.16946v1
• Date: Mon, 27 May 2024 08:38:17 GMT
• Title: Biological Neurons Compete with Deep Reinforcement Learning in Sample Efficiency in a Simulated Gameworld
• Authors: Moein Khajehnejad, Forough Habibollahi, Aswin Paul, Adeel Razi, Brett J. Kagan,
• Abstract summary: We compare the learning efficiency of in vitro biological neural networks to the state-of-the-art deep reinforcement learning (RL) algorithms in a simplified simulation of the game Pong' Even when tested across multiple types of information input, biological neurons showcased faster learning than all deep reinforcement learning agents.
• Score: 2.003941363902692
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How do biological systems and machine learning algorithms compare in the number of samples required to show significant improvements in completing a task? We compared the learning efficiency of in vitro biological neural networks to the state-of-the-art deep reinforcement learning (RL) algorithms in a simplified simulation of the game `Pong'. Using DishBrain, a system that embodies in vitro neural networks with in silico computation using a high-density multi-electrode array, we contrasted the learning rate and the performance of these biological systems against time-matched learning from three state-of-the-art deep RL algorithms (i.e., DQN, A2C, and PPO) in the same game environment. This allowed a meaningful comparison between biological neural systems and deep RL. We find that when samples are limited to a real-world time course, even these very simple biological cultures outperformed deep RL algorithms across various game performance characteristics, implying a higher sample efficiency. Ultimately, even when tested across multiple types of information input to assess the impact of higher dimensional data input, biological neurons showcased faster learning than all deep reinforcement learning agents.

Related papers

• Memory Networks: Towards Fully Biologically Plausible Learning [2.7013801448234367]
  Current artificial neural networks rely on techniques like backpropagation and weight sharing, which do not align with the brain's natural information processing methods. We propose the Memory Network, a model inspired by biological principles that avoids backpropagation and convolutions, and operates in a single pass.
  arXiv  Detail & Related papers  (2024-09-18T06:01:35Z)
• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Towards Biologically Plausible Computing: A Comprehensive Comparison [24.299920289520013]
  Backpropagation is a cornerstone algorithm in training neural networks for supervised learning. The biological plausibility of backpropagation is questioned due to its requirements for weight symmetry, global error computation, and dual-phase training. In this study, we establish criteria for biological plausibility that a desirable learning algorithm should meet.
  arXiv  Detail & Related papers  (2024-06-23T09:51:20Z)
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment [65.268245109828]
  We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology. We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors. The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
  arXiv  Detail & Related papers  (2023-12-01T05:20:57Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Efficient, probabilistic analysis of combinatorial neural codes [0.0]
  neural networks encode inputs in the form of combinations of individual neurons' activities. These neural codes present a computational challenge due to their high dimensionality and often large volumes of data. We apply methods previously applied to small examples and apply them to large neural codes generated by experiments.
  arXiv  Detail & Related papers  (2022-10-19T11:58:26Z)
• aSTDP: A More Biologically Plausible Learning [0.0]
  We introduce approximate STDP, a new neural networks learning framework. It uses only STDP rules for supervised and unsupervised learning. It can make predictions or generate patterns in one model without additional configuration.
  arXiv  Detail & Related papers  (2022-05-22T08:12:50Z)
• Credit Assignment in Neural Networks through Deep Feedback Control [59.14935871979047]
  Deep Feedback Control (DFC) is a new learning method that uses a feedback controller to drive a deep neural network to match a desired output target and whose control signal can be used for credit assignment. The resulting learning rule is fully local in space and time and approximates Gauss-Newton optimization for a wide range of connectivity patterns. To further underline its biological plausibility, we relate DFC to a multi-compartment model of cortical pyramidal neurons with a local voltage-dependent synaptic plasticity rule, consistent with recent theories of dendritic processing.
  arXiv  Detail & Related papers  (2021-06-15T05:30:17Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Predictive Coding Can Do Exact Backpropagation on Any Neural Network [40.51949948934705]
  We generalize (IL and) Z-IL by directly defining them on computational graphs. This is the first biologically plausible algorithm that is shown to be equivalent to BP in the way of updating parameters on any neural network.
  arXiv  Detail & Related papers  (2021-03-08T11:52:51Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.