Catastrophic Forgetting in Continual Reinforcement Learning

• URL: http://arxiv.org/abs/2608.08673v1
• Date: Sun, 09 Aug 2026 12:39:07 GMT
• Title: Catastrophic Forgetting in Continual Reinforcement Learning
• Abstract summary: Catastrophic forgetting is a significant impediment to continual learning.<n>This study aims to understand the extent to which the similarity of a new task influences the performance on the previous task.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work explores the relationship between task similarity and catastrophic forgetting in reinforcement learning. Catastrophic forgetting, the phenomenon in machine learning of losing the ability to effectively perform on previous tasks, is a significant impediment to continual learning. This study aims to understand the extent to which the similarity of a new task influences the performance on the previous task. Interpretable reinforcement learning, specifically Q-learning, is employed on graph-based tasks with the objective of minimising the number of steps to reach a goal. The study investigates the performance on a previously learned task after training on a new task, for tasks of varying relative levels of complexity. The experimental results reveal a complex dynamic between task similarity and forgetting, with significant fluctuations in forgetting severity observed across degrees of task similarities and task complexities, and are suggestive of an interdependence of forgetting on the similarity and complexity of tasks. The observations were accompanied by observations of high degrees of variability in forgetting and an uneven distribution of task similarity measures. The relationship between these variables remains unclear and no evidence of statistical significance that task similarity has an effect, independently, on forgetting is found in continual reinforcement learning. Further research is warranted to gain a comprehensive understanding of the potential interplay between task similarity and catastrophic forgetting.

Related papers

• The Importance of Being Lazy: Scaling Limits of Continual Learning [60.97756735877614]
  We show that increasing model width is only beneficial when it reduces the amount of feature learning, yielding more laziness.<n>We study the intricate relationship between feature learning, task non-stationarity, and forgetting, finding that high feature learning is only beneficial with highly similar tasks.
  arXiv  Detail & Related papers  (2025-06-20T10:12:38Z)
• Disentangling and Mitigating the Impact of Task Similarity for Continual Learning [1.3597551064547502]
  Continual learning of partially similar tasks poses a challenge for artificial neural networks. High input feature similarity coupled with low readout similarity is catastrophic for both knowledge transfer and retention. Weight regularization based on the Fisher information metric significantly improves retention, regardless of task similarity.
  arXiv  Detail & Related papers  (2024-05-30T16:40:07Z)
• Exploring Correlations of Self-Supervised Tasks for Graphs [6.977921096191354]
  This paper aims to provide a fresh understanding of graph self-supervised learning based on task correlations. We evaluate the performance of the representations trained by one specific task on other tasks and define correlation values to quantify task correlations. We propose Graph Task Correlation Modeling (GraphTCM) to illustrate the task correlations and utilize it to enhance graph self-supervised training.
  arXiv  Detail & Related papers  (2024-05-07T12:02:23Z)
• Multitask Learning with No Regret: from Improved Confidence Bounds to Active Learning [79.07658065326592]
  Quantifying uncertainty in the estimated tasks is of pivotal importance for many downstream applications, such as online or active learning. We provide novel multitask confidence intervals in the challenging setting when neither the similarity between tasks nor the tasks' features are available to the learner. We propose a novel online learning algorithm that achieves such improved regret without knowing this parameter in advance.
  arXiv  Detail & Related papers  (2023-08-03T13:08:09Z)
• Composite Learning for Robust and Effective Dense Predictions [81.2055761433725]
  Multi-task learning promises better model generalization on a target task by jointly optimizing it with an auxiliary task. We find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks.
  arXiv  Detail & Related papers  (2022-10-13T17:59:16Z)
• Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering [58.82325933356066]
  Continual learning aims to train a model incrementally on a sequence of tasks without forgetting previous knowledge. We present a detailed study of how different settings affect performance for Visual Question Answering.
  arXiv  Detail & Related papers  (2022-09-30T19:12:58Z)
• Saliency-Regularized Deep Multi-Task Learning [7.3810864598379755]
  Multitask learning enforces multiple learning tasks to share knowledge to improve their generalization abilities. Modern deep multitask learning can jointly learn latent features and task sharing, but they are obscure in task relation. This paper proposes a new multitask learning framework that jointly learns latent features and explicit task relations.
  arXiv  Detail & Related papers  (2022-07-03T20:26:44Z)
• The Effect of Task Ordering in Continual Learning [12.571389210876315]
  We show that reordering tasks significantly affects the amount of catastrophic forgetting. We show that the effect of task ordering can be exploited to modify continual learning performance.
  arXiv  Detail & Related papers  (2022-05-26T12:56:15Z)
• Continual Learning in the Teacher-Student Setup: Impact of Task Similarity [5.1135133995376085]
  We study catastrophic forgetting in two-layer networks in the teacher-student setup. We find that when tasks depend on similar features, intermediate task similarity leads to greatest forgetting. We find a complex interplay between both types of similarity, initial transfer/forgetting rates, maximum transfer/forgetting, and long-term transfer/forgetting.
  arXiv  Detail & Related papers  (2021-07-09T12:30:39Z)
• Seeing Differently, Acting Similarly: Imitation Learning with Heterogeneous Observations [126.78199124026398]
  In many real-world imitation learning tasks, the demonstrator and the learner have to act in different but full observation spaces. In this work, we model the above learning problem as Heterogeneous Observations Learning (HOIL) We propose the Importance Weighting with REjection (IWRE) algorithm based on the techniques of importance-weighting, learning with rejection, and active querying to solve the key challenge of occupancy measure matching.
  arXiv  Detail & Related papers  (2021-06-17T05:44:04Z)
• Robust Learning Through Cross-Task Consistency [92.42534246652062]
  We propose a broadly applicable and fully computational method for augmenting learning with Cross-Task Consistency. We observe that learning with cross-task consistency leads to more accurate predictions and better generalization to out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-06-07T09:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.