The Dormant Neuron Phenomenon in Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2302.12902v2
• Date: Tue, 13 Jun 2023 15:16:55 GMT
• Title: The Dormant Neuron Phenomenon in Deep Reinforcement Learning
• Authors: Ghada Sokar, Rishabh Agarwal, Pablo Samuel Castro, Utku Evci
• Abstract summary: We identify the dormant neuron phenomenon in deep reinforcement learning, where an agent's network suffers from an increasing number of inactive neurons. We propose a simple and effective method (ReDo) that Recycles Dormant neurons throughout training. Our experiments demonstrate that ReDo maintains the expressive power of networks by reducing the number of dormant neurons and results in improved performance.
• Score: 26.09145694804957
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agent's network suffers from an increasing number of inactive neurons, thereby affecting network expressivity. We demonstrate the presence of this phenomenon across a variety of algorithms and environments, and highlight its effect on learning. To address this issue, we propose a simple and effective method (ReDo) that Recycles Dormant neurons throughout training. Our experiments demonstrate that ReDo maintains the expressive power of networks by reducing the number of dormant neurons and results in improved performance.

Related papers

• Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning [22.627302782393865]
  This paper investigates the relationship between large language models' (LLMs) ability to recognize repetitive input patterns and their performance on in-context learning (ICL)<n>Our experiments reveal that the impact of repetition neurons on ICL performance varies depending on the depth of the layer in which they reside.
  arXiv  Detail & Related papers  (2025-07-10T14:40:31Z)
• Neural Learning Rules from Associative Networks Theory [0.0]
  Associative networks theory is providing tools to interpret update rules of artificial neural networks. deriving neural learning rules from a solid theory remains a fundamental challenge.
  arXiv  Detail & Related papers  (2025-03-11T11:44:04Z)
• Discovering Chunks in Neural Embeddings for Interpretability [53.80157905839065]
  We propose leveraging the principle of chunking to interpret artificial neural population activities. We first demonstrate this concept in recurrent neural networks (RNNs) trained on artificial sequences with imposed regularities. We identify similar recurring embedding states corresponding to concepts in the input, with perturbations to these states activating or inhibiting the associated concepts.
  arXiv  Detail & Related papers  (2025-02-03T20:30:46Z)
• Artificial Kuramoto Oscillatory Neurons [65.16453738828672]
  We introduce Artificial Kuramotoy Neurons (AKOrN) as a dynamical alternative to threshold units. We show that this idea provides performance improvements across a wide spectrum of tasks. We believe that these empirical results show the importance of our assumptions at the most basic neuronal level of neural representation.
  arXiv  Detail & Related papers  (2024-10-17T17:47:54Z)
• Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling [52.06722364186432]
  We propose a biologically-informed framework for enhancing artificial neural networks (ANNs) Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors. We outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, bioinspiration and complexity.
  arXiv  Detail & Related papers  (2024-07-05T14:11:28Z)
• Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks [59.38765771221084]
  We present a physiologically inspired speech recognition architecture compatible and scalable with deep learning frameworks. We show end-to-end gradient descent training leads to the emergence of neural oscillations in the central spiking neural network. Our findings highlight the crucial inhibitory role of feedback mechanisms, such as spike frequency adaptation and recurrent connections, in regulating and synchronising neural activity to improve recognition performance.
  arXiv  Detail & Related papers  (2024-04-22T09:40:07Z)
• Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks [74.3099028063756]
  We develop a new method with neuronal operations based on lateral connections and Hebbian learning. We show that Hebbian and anti-Hebbian learning on recurrent lateral connections can effectively extract the principal subspace of neural activities. Our method consistently solves for spiking neural networks with nearly zero forgetting.
  arXiv  Detail & Related papers  (2024-02-19T09:29:37Z)
• Seeking Next Layer Neurons' Attention for Error-Backpropagation-Like Training in a Multi-Agent Network Framework [6.446189857311325]
  We propose a local objective for neurons that align them to exhibit similarities to error-backpropagation. We examine a neural network comprising decentralized, self-interested neurons seeking to maximize their local objective. We demonstrate the learning capacity of these multi-agent neural networks through experiments on three datasets.
  arXiv  Detail & Related papers  (2023-10-15T21:07:09Z)
• Expressivity of Spiking Neural Networks [15.181458163440634]
  We study the capabilities of spiking neural networks where information is encoded in the firing time of neurons. In contrast to ReLU networks, we prove that spiking neural networks can realize both continuous and discontinuous functions.
  arXiv  Detail & Related papers  (2023-08-16T08:45:53Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• Biologically-inspired neuronal adaptation improves learning in neural networks [0.7734726150561086]
  Humans still outperform artificial neural networks on many tasks. We draw inspiration from the brain to improve machine learning algorithms. We add adaptation to multilayer perceptrons and convolutional neural networks trained on MNIST and CIFAR-10.
  arXiv  Detail & Related papers  (2022-04-08T16:16:02Z)
• Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks [0.9790524827475205]
  We show how a novel type of adaptive spiking recurrent neural network (SRNN) is able to achieve state-of-the-art performance. We calculate a $>$100x energy improvement for our SRNNs over classical RNNs on the harder tasks.
  arXiv  Detail & Related papers  (2020-05-24T01:04:53Z)
• Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations [0.3441021278275805]
  We shed light on the roles of single neurons and groups of neurons within the network fulfilling a learned task. We find that neither a neuron's magnitude or selectivity of activation, nor its impact on network performance are sufficient stand-alone indicators.
  arXiv  Detail & Related papers  (2020-04-02T20:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.