Natural Mitigation of Catastrophic Interference: Continual Learning in Power-Law Learning Environments

• URL: http://arxiv.org/abs/2401.10393v3
• Date: Mon, 26 Aug 2024 23:10:59 GMT
• Title: Natural Mitigation of Catastrophic Interference: Continual Learning in Power-Law Learning Environments
• Authors: Atith Gandhi, Raj Sanjay Shah, Vijay Marupudi, Sashank Varma,
• Abstract summary: We show that in naturalistic environments, the probability of encountering a task decreases as a power-law of the time since it was last performed. We evaluate the extent of the natural mitigation of CI when training models in power-law environments, similar to those humans face.
• Score: 2.714641498775159
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Neural networks often suffer from catastrophic interference (CI): performance on previously learned tasks drops off significantly when learning a new task. This contrasts strongly with humans, who can continually learn new tasks without appreciably forgetting previous tasks. Prior work has explored various techniques for mitigating CI and promoting continual learning such as regularization, rehearsal, generative replay, and context-specific components. This paper takes a different approach, one guided by cognitive science research showing that in naturalistic environments, the probability of encountering a task decreases as a power-law of the time since it was last performed. We argue that techniques for mitigating CI should be compared against the intrinsic mitigation in simulated naturalistic learning environments. Thus, we evaluate the extent of the natural mitigation of CI when training models in power-law environments, similar to those humans face. Our results show that natural rehearsal environments are better at mitigating CI than existing methods, calling for the need for better evaluation processes. The benefits of this environment include simplicity, rehearsal that is agnostic to both tasks and models, and the lack of a need for extra neural circuitry. In addition, we explore popular mitigation techniques in power-law environments to create new baselines for continual learning research.

Related papers

• No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery [53.08822154199948]
  Unsupervised Environment Design (UED) methods have gained recent attention as their adaptive curricula promise to enable agents to be robust to in- and out-of-distribution tasks. This work investigates how existing UED methods select training environments, focusing on task prioritisation metrics. We develop a method that directly trains on scenarios with high learnability.
  arXiv  Detail & Related papers  (2024-08-27T14:31:54Z)
• Discovering Minimal Reinforcement Learning Environments [24.6408931194983]
  Reinforcement learning (RL) agents are commonly trained and evaluated in the same environment. Humans often train in a specialized environment before being evaluated, such as studying a book before taking an exam.
  arXiv  Detail & Related papers  (2024-06-18T13:19:26Z)
• Decoding Neural Activity to Assess Individual Latent State in Ecologically Valid Contexts [1.1059590443280727]
  We use data from two highly controlled laboratory paradigms to train two separate domain-generalized models. We derive estimates of the underlying latent state and associated patterns of neural activity.
  arXiv  Detail & Related papers  (2023-04-18T15:15:00Z)
• Continual Predictive Learning from Videos [100.27176974654559]
  We study a new continual learning problem in the context of video prediction. We propose the continual predictive learning (CPL) approach, which learns a mixture world model via predictive experience replay. We construct two new benchmarks based on RoboNet and KTH, in which different tasks correspond to different physical robotic environments or human actions.
  arXiv  Detail & Related papers  (2022-04-12T08:32:26Z)
• Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
  Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world. Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts. This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
  arXiv  Detail & Related papers  (2021-12-17T16:28:06Z)
• Natural continual learning: success is a journey, not (just) a destination [9.462808515258464]
  Natural Continual Learning (NCL) is a new method that unifies weight regularization and projected gradient descent. Our method outperforms both standard weight regularization techniques and projection based approaches when applied to continual learning problems in RNNs. The trained networks evolve task-specific dynamics that are strongly preserved as new tasks are learned, similar to experimental findings in biological circuits.
  arXiv  Detail & Related papers  (2021-06-15T12:24:53Z)
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment [55.91291559442884]
  This work develops a methodology that enables data-driven methods to continuously learn and optimize in a dynamic environment. We propose to build the notion of continual learning into the modeling process of learning wireless systems. Our design is based on a novel min-max formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2020-11-16T08:24:34Z)
• Ecological Reinforcement Learning [76.9893572776141]
  We study the kinds of environment properties that can make learning under such conditions easier. understanding how properties of the environment impact the performance of reinforcement learning agents can help us to structure our tasks in ways that make learning tractable.
  arXiv  Detail & Related papers  (2020-06-22T17:55:03Z)
• Deep Reinforcement Learning amidst Lifelong Non-Stationarity [67.24635298387624]
  We show that an off-policy RL algorithm can reason about and tackle lifelong non-stationarity. Our method leverages latent variable models to learn a representation of the environment from current and past experiences. We also introduce several simulation environments that exhibit lifelong non-stationarity, and empirically find that our approach substantially outperforms approaches that do not reason about environment shift.
  arXiv  Detail & Related papers  (2020-06-18T17:34:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.