Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2504.17490v1
• Date: Thu, 24 Apr 2025 12:32:13 GMT
• Title: Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learning
• Authors: Mingqi Yuan, Qi Wang, Guozheng Ma, Bo Li, Xin Jin, Yunbo Wang, Xiaokang Yang, Wenjun Zeng, Dacheng Tao,
• Abstract summary: Plasticine is an open-source framework for benchmarking plasticity optimization in deep reinforcement learning.<n>Plasticine provides single-file implementations of over 13 mitigation methods, 10 evaluation metrics, and learning scenarios.
• Score: 122.67854581396578
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Developing lifelong learning agents is crucial for artificial general intelligence. However, deep reinforcement learning (RL) systems often suffer from plasticity loss, where neural networks gradually lose their ability to adapt during training. Despite its significance, this field lacks unified benchmarks and evaluation protocols. We introduce Plasticine, the first open-source framework for benchmarking plasticity optimization in deep RL. Plasticine provides single-file implementations of over 13 mitigation methods, 10 evaluation metrics, and learning scenarios with increasing non-stationarity levels from standard to open-ended environments. This framework enables researchers to systematically quantify plasticity loss, evaluate mitigation strategies, and analyze plasticity dynamics across different contexts. Our documentation, examples, and source code are available at https://github.com/RLE-Foundation/Plasticine.

Related papers

• Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining [55.262510814326035]
  Existing reweighting strategies primarily focus on group-level data importance. We introduce novel algorithms for dynamic, instance-level data reweighting. Our framework allows us to devise reweighting strategies deprioritizing redundant or uninformative data.
  arXiv  Detail & Related papers  (2025-02-10T17:57:15Z)
• Plasticity Loss in Deep Reinforcement Learning: A Survey [15.525552360867367]
  plasticity is crucial for deep Reinforcement Learning (RL) agents. Once plasticity is lost, an agent's performance will plateau because it cannot improve its policy to account for changes in the data distribution. Loss of plasticity can be connected to many other issues plaguing deep RL, such as training instabilities, scaling failures, overestimation bias, and insufficient exploration.
  arXiv  Detail & Related papers  (2024-11-07T16:13:54Z)
• Disentangling the Causes of Plasticity Loss in Neural Networks [55.23250269007988]
  We show that loss of plasticity can be decomposed into multiple independent mechanisms. We show that a combination of layer normalization and weight decay is highly effective at maintaining plasticity in a variety of synthetic nonstationary learning tasks.
  arXiv  Detail & Related papers  (2024-02-29T00:02:33Z)
• A Discrepancy Aware Framework for Robust Anomaly Detection [51.710249807397695]
  We present a Discrepancy Aware Framework (DAF), which demonstrates robust performance consistently with simple and cheap strategies. Our method leverages an appearance-agnostic cue to guide the decoder in identifying defects, thereby alleviating its reliance on synthetic appearance. Under the simple synthesis strategies, it outperforms existing methods by a large margin. Furthermore, it also achieves the state-of-the-art localization performance.
  arXiv  Detail & Related papers  (2023-10-11T15:21:40Z)
• Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages [56.98243487769916]
  Plasticity, the ability of a neural network to evolve with new data, is crucial for high-performance and sample-efficient visual reinforcement learning. We propose Adaptive RR, which dynamically adjusts the replay ratio based on the critic's plasticity level.
  arXiv  Detail & Related papers  (2023-10-11T12:05:34Z)
• PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning [54.409634256153154]
  In Reinforcement Learning (RL), enhancing sample efficiency is crucial. In principle, off-policy RL algorithms can improve sample efficiency by allowing multiple updates per environment interaction. Our study investigates the underlying causes of this phenomenon by dividing plasticity into two aspects.
  arXiv  Detail & Related papers  (2023-06-19T06:14:51Z)
• Deep Reinforcement Learning with Plasticity Injection [37.19742321534183]
  Evidence suggests that in deep reinforcement learning (RL) networks gradually lose their plasticity. plasticity injection increases the network plasticity without changing the number of parameters. plasticity injection attains stronger performance compared to alternative methods.
  arXiv  Detail & Related papers  (2023-05-24T20:41:35Z)
• Understanding plasticity in neural networks [41.79540750236036]
  Plasticity is the ability of a neural network to quickly change its predictions in response to new information. Deep neural networks are known to lose plasticity over the course of training even in relatively simple learning problems.
  arXiv  Detail & Related papers  (2023-03-02T18:47:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.