Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Network

• URL: http://arxiv.org/abs/2301.10292v2
• Date: Mon, 25 Nov 2024 15:11:06 GMT
• Title: Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Network
• Authors: Duzhen Zhang, Tielin Zhang, Shuncheng Jia, Qingyu Wang, Bo Xu,
• Abstract summary: Modern deep reinforcement learning (DRL) explores a computational approach to learning from interaction. We optimized a spiking policy network (SPN) using a genetic algorithm as an energy-efficient alternative to DRL. Inspired by biological research showing that the brain forms memories by creating new synaptic connections, we tuned the synaptic connections instead of weights in the SPN to solve given tasks.
• Score: 16.876474167808784
• License:
• Abstract: Learning from interaction is the primary way that biological agents acquire knowledge about their environment and themselves. Modern deep reinforcement learning (DRL) explores a computational approach to learning from interaction and has made significant progress in solving various tasks. However, despite its power, DRL still falls short of biological agents in terms of energy efficiency. Although the underlying mechanisms are not fully understood, we believe that the integration of spiking communication between neurons and biologically-plausible synaptic plasticity plays a prominent role in achieving greater energy efficiency. Following this biological intuition, we optimized a spiking policy network (SPN) using a genetic algorithm as an energy-efficient alternative to DRL. Our SPN mimics the sensorimotor neuron pathway of insects and communicates through event-based spikes. Inspired by biological research showing that the brain forms memories by creating new synaptic connections and rewiring these connections based on new experiences, we tuned the synaptic connections instead of weights in the SPN to solve given tasks. Experimental results on several robotic control tasks demonstrate that our method can achieve the same level of performance as mainstream DRL methods while exhibiting significantly higher energy efficiency.

Related papers

• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Research Advances and New Paradigms for Biology-inspired Spiking Neural Networks [8.315801422499861]
  Spiking neural networks (SNNs) are gaining popularity in the computational simulation and artificial intelligence fields. This paper explores the historical development of SNN and concludes that these two fields are intersecting and merging rapidly.
  arXiv  Detail & Related papers  (2024-08-26T03:37:48Z)
• Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling [52.06722364186432]
  We propose a biologically-informed framework for enhancing artificial neural networks (ANNs) Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors. We outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, bioinspiration and complexity.
  arXiv  Detail & Related papers  (2024-07-05T14:11:28Z)
• Biologically-Plausible Topology Improved Spiking Actor Network for Efficient Deep Reinforcement Learning [15.143466733327566]
  Recent advances in neuroscience have unveiled that the human brain achieves efficient reward-based learning. The success of Deep Reinforcement Learning (DRL) is largely attributed to utilizing Artificial Neural Networks (ANNs) as function approximators. We propose a novel alternative for function approximator, the Biologically-Plausible Topology improved Spiking Actor Network (BPT-SAN)
  arXiv  Detail & Related papers  (2024-03-29T13:25:19Z)
• Single Neuromorphic Memristor closely Emulates Multiple Synaptic Mechanisms for Energy Efficient Neural Networks [71.79257685917058]
  We demonstrate memristive nano-devices based on SrTiO3 that inherently emulate all these synaptic functions. These memristors operate in a non-filamentary, low conductance regime, which enables stable and energy efficient operation.
  arXiv  Detail & Related papers  (2024-02-26T15:01:54Z)
• Learning with Chemical versus Electrical Synapses -- Does it Make a Difference? [61.85704286298537]
  Bio-inspired neural networks have the potential to advance our understanding of neural computation and improve the state-of-the-art of AI systems. We conduct experiments with autonomous lane-keeping through a photorealistic autonomous driving simulator to evaluate their performance under diverse conditions.
  arXiv  Detail & Related papers  (2023-11-21T13:07:20Z)
• A Spiking Binary Neuron -- Detector of Causal Links [0.0]
  Causal relationship recognition is a fundamental operation in neural networks aimed at learning behavior, action planning, and inferring external world dynamics. This research paper presents a novel approach to realize causal relationship recognition using a simple spiking binary neuron.
  arXiv  Detail & Related papers  (2023-09-15T15:34:17Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Credit Assignment in Neural Networks through Deep Feedback Control [59.14935871979047]
  Deep Feedback Control (DFC) is a new learning method that uses a feedback controller to drive a deep neural network to match a desired output target and whose control signal can be used for credit assignment. The resulting learning rule is fully local in space and time and approximates Gauss-Newton optimization for a wide range of connectivity patterns. To further underline its biological plausibility, we relate DFC to a multi-compartment model of cortical pyramidal neurons with a local voltage-dependent synaptic plasticity rule, consistent with recent theories of dendritic processing.
  arXiv  Detail & Related papers  (2021-06-15T05:30:17Z)
• Embodied Synaptic Plasticity with Online Reinforcement learning [5.6006805285925445]
  This paper contributes to bringing the fields of computational neuroscience and robotics closer together by integrating open-source software components from these two fields. We demonstrate this framework to evaluate Synaptic Plasticity with Online REinforcement learning (SPORE), a reward-learning rule based on synaptic sampling, on two visuomotor tasks.
  arXiv  Detail & Related papers  (2020-03-03T10:29:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.