Continual Reinforcement Learning via Autoencoder-Driven Task and New Environment Recognition

• URL: http://arxiv.org/abs/2505.09003v1
• Date: Tue, 13 May 2025 22:38:54 GMT
• Title: Continual Reinforcement Learning via Autoencoder-Driven Task and New Environment Recognition
• Authors: Zeki Doruk Erden, Donia Gasmi, Boi Faltings,
• Abstract summary: We study the effectiveness of autoencoders in detecting new tasks and matching observed environments to previously encountered ones.<n>Our approach integrates policy optimization with familiarity autoencoders within an end-to-end continual learning system.
• Score: 13.681245711959543
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning for reinforcement learning agents remains a significant challenge, particularly in preserving and leveraging existing information without an external signal to indicate changes in tasks or environments. In this study, we explore the effectiveness of autoencoders in detecting new tasks and matching observed environments to previously encountered ones. Our approach integrates policy optimization with familiarity autoencoders within an end-to-end continual learning system. This system can recognize and learn new tasks or environments while preserving knowledge from earlier experiences and can selectively retrieve relevant knowledge when re-encountering a known environment. Initial results demonstrate successful continual learning without external signals to indicate task changes or reencounters, showing promise for this methodology.

Related papers

• Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning [19.463863037999054]
  We consider a Continual Reinforcement Learning setup, where a learning agent must continuously adapt to new tasks while retaining previously acquired skill sets.<n>We introduce HiSPO, a novel hierarchical framework designed specifically for continual learning in navigation settings from offline data.<n>We demonstrate, through a careful experimental study, the effectiveness of our method in both classical MuJoCo maze environments and complex video game-like navigation simulations.
  arXiv  Detail & Related papers  (2024-12-19T14:00:03Z)
• No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery [53.08822154199948]
  Unsupervised Environment Design (UED) methods have gained recent attention as their adaptive curricula promise to enable agents to be robust to in- and out-of-distribution tasks. This work investigates how existing UED methods select training environments, focusing on task prioritisation metrics. We develop a method that directly trains on scenarios with high learnability.
  arXiv  Detail & Related papers  (2024-08-27T14:31:54Z)
• Recall-Oriented Continual Learning with Generative Adversarial Meta-Model [5.710971447109951]
  We propose a recall-oriented continual learning framework to address the stability-plasticity dilemma. Inspired by the human brain's ability to separate the mechanisms responsible for stability and plasticity, our framework consists of a two-level architecture. We show that our framework not only effectively learns new knowledge without any disruption but also achieves high stability of previous knowledge.
  arXiv  Detail & Related papers  (2024-03-05T16:08:59Z)
• Online Continual Learning via the Knowledge Invariant and Spread-out Properties [4.109784267309124]
  Key challenge in continual learning is catastrophic forgetting. We propose a new method, named Online Continual Learning via the Knowledge Invariant and Spread-out Properties (OCLKISP) We empirically evaluate our proposed method on four popular benchmarks for continual learning: Split CIFAR 100, Split SVHN, Split CUB200 and Split Tiny-Image-Net.
  arXiv  Detail & Related papers  (2023-02-02T04:03:38Z)
• Learning Task Automata for Reinforcement Learning using Hidden Markov Models [37.69303106863453]
  This paper proposes a novel pipeline for learning non-Markovian task specifications as succinct finite-state task automata' We learn a product MDP, a model composed of the specification's automaton and the environment's MDP, by treating the product MDP as a partially observable MDP and using the well-known Baum-Welch algorithm for learning hidden Markov models. Our learnt task automaton enables the decomposition of a task into its constituent sub-tasks, which improves the rate at which an RL agent can later synthesise an optimal policy.
  arXiv  Detail & Related papers  (2022-08-25T02:58:23Z)
• Learning with Recoverable Forgetting [77.56338597012927]
  Learning wIth Recoverable Forgetting explicitly handles the task- or sample-specific knowledge removal and recovery. Specifically, LIRF brings in two innovative schemes, namely knowledge deposit and withdrawal. We conduct experiments on several datasets, and demonstrate that the proposed LIRF strategy yields encouraging results with gratifying generalization capability.
  arXiv  Detail & Related papers  (2022-07-17T16:42:31Z)
• L2Explorer: A Lifelong Reinforcement Learning Assessment Environment [49.40779372040652]
  Reinforcement learning solutions tend to generalize poorly when exposed to new tasks outside of the data distribution they are trained on. We introduce a framework for continual reinforcement-learning development and assessment using Lifelong Learning Explorer (L2Explorer) L2Explorer is a new, Unity-based, first-person 3D exploration environment that can be continuously reconfigured to generate a range of tasks and task variants structured into complex evaluation curricula.
  arXiv  Detail & Related papers  (2022-03-14T19:20:26Z)
• Continual Prompt Tuning for Dialog State Tracking [58.66412648276873]
  A desirable dialog system should be able to continually learn new skills without forgetting old ones. We present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks.
  arXiv  Detail & Related papers  (2022-03-13T13:22:41Z)
• Transferability in Deep Learning: A Survey [80.67296873915176]
  The ability to acquire and reuse knowledge is known as transferability in deep learning. We present this survey to connect different isolated areas in deep learning with their relation to transferability. We implement a benchmark and an open-source library, enabling a fair evaluation of deep learning methods in terms of transferability.
  arXiv  Detail & Related papers  (2022-01-15T15:03:17Z)
• Relational Experience Replay: Continual Learning by Adaptively Tuning Task-wise Relationship [54.73817402934303]
  We propose Experience Continual Replay (ERR), a bi-level learning framework to adaptively tune task-wise to achieve a better stability plasticity' tradeoff. ERR can consistently improve the performance of all baselines and surpass current state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-31T12:05:22Z)
• AFEC: Active Forgetting of Negative Transfer in Continual Learning [37.03139674884091]
  We show that biological neural networks can actively forget the old knowledge that conflicts with the learning of a new experience. Inspired by the biological active forgetting, we propose to actively forget the old knowledge that limits the learning of new tasks to benefit continual learning.
  arXiv  Detail & Related papers  (2021-10-23T10:03:19Z)
• Self-Attention Meta-Learner for Continual Learning [5.979373021392084]
  Self-Attention Meta-Learner (SAM) learns a prior knowledge for continual learning that permits learning a sequence of tasks. SAM incorporates an attention mechanism that learns to select the particular relevant representation for each future task. We evaluate the proposed method on the Split CIFAR-10/100 and Split MNIST benchmarks in the task inference.
  arXiv  Detail & Related papers  (2021-01-28T17:35:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.