Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learning

• URL: http://arxiv.org/abs/2603.00903v1
• Date: Sun, 01 Mar 2026 03:49:20 GMT
• Title: Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learning
• Authors: Ke Sun, Hongming Zhang, Jun Jin, Chao Gao, Xi Chen, Wulong Liu, Linglong Kong,
• Abstract summary: We propose a dual-learner framework comprising a fast learner and a meta learner to address continual Reinforcement Learning(RL) problems.<n>These two learners are coupled to perform distinct yet complementary roles: the fast learner focuses on knowledge transfer, while the meta learner ensures knowledge integration.
• Score: 39.70127895481068
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inspired by the human learning and memory system, particularly the interplay between the hippocampus and cerebral cortex, this study proposes a dual-learner framework comprising a fast learner and a meta learner to address continual Reinforcement Learning~(RL) problems. These two learners are coupled to perform distinct yet complementary roles: the fast learner focuses on knowledge transfer, while the meta learner ensures knowledge integration. In contrast to traditional multi-task RL approaches that share knowledge through average return maximization, our meta learner incrementally integrates new experiences by explicitly minimizing catastrophic forgetting, thereby supporting efficient cumulative knowledge transfer for the fast learner. To facilitate rapid adaptation in new environments, we introduce an adaptive meta warm-up mechanism that selectively harnesses past knowledge. We conduct experiments in various pixel-based and continuous control benchmarks, revealing the superior performance of continual learning for our proposed dual-learner approach relative to baseline methods. The code is released in https://github.com/datake/FAME.

Related papers

• Continual Knowledge Adaptation for Reinforcement Learning [37.4253231932861]
  Reinforcement Learning enables agents to learn optimal behaviors through interactions with environments.<n>We propose Continual Knowledge Adaptation for Reinforcement Learning (CKA-RL), which enables the accumulation and effective utilization of historical knowledge.<n> Experiments on three benchmarks demonstrate that the proposed CKA-RL outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-22T07:25:41Z)
• How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training [92.88889953768455]
  Large Language Models (LLMs) face a critical gap in understanding how they internalize new knowledge.<n>We identify computational subgraphs that facilitate knowledge storage and processing.
  arXiv  Detail & Related papers  (2025-02-16T16:55:43Z)
• Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning [79.46570165281084]
  We propose a Multi-Stage Knowledge Integration network (MulKI) to emulate the human learning process in distillation methods. MulKI achieves this through four stages, including Eliciting Ideas, Adding New Ideas, Distinguishing Ideas, and Making Connections. Our method demonstrates significant improvements in maintaining zero-shot capabilities while supporting continual learning across diverse downstream tasks.
  arXiv  Detail & Related papers  (2024-11-11T07:36:19Z)
• Towards Automated Knowledge Integration From Human-Interpretable Representations [55.2480439325792]
  We introduce and motivate theoretically the principles of informed meta-learning enabling automated and controllable inductive bias selection.<n>We empirically demonstrate the potential benefits and limitations of informed meta-learning in improving data efficiency and generalisation.
  arXiv  Detail & Related papers  (2024-02-25T15:08:37Z)
• Anti-Retroactive Interference for Lifelong Learning [65.50683752919089]
  We design a paradigm for lifelong learning based on meta-learning and associative mechanism of the brain. It tackles the problem from two aspects: extracting knowledge and memorizing knowledge. It is theoretically analyzed that the proposed learning paradigm can make the models of different tasks converge to the same optimum.
  arXiv  Detail & Related papers  (2022-08-27T09:27:36Z)
• Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System [13.041607703862724]
  We propose CLS-ER, a novel dual memory experience replay (ER) method. New knowledge is acquired while aligning the decision boundaries with the semantic memories. Our approach achieves state-of-the-art performance on standard benchmarks.
  arXiv  Detail & Related papers  (2022-01-29T15:15:23Z)
• Mixture-of-Variational-Experts for Continual Learning [0.0]
  We propose an optimality principle that facilitates a trade-off between learning and forgetting. We propose a neural network layer for continual learning, called Mixture-of-Variational-Experts (MoVE) Our experiments on variants of the MNIST and CIFAR10 datasets demonstrate the competitive performance of MoVE layers.
  arXiv  Detail & Related papers  (2021-10-25T06:32:06Z)
• Meta Learning for Knowledge Distillation [12.716258111815312]
  We show the teacher network can learn to better transfer knowledge to the student network. We introduce a pilot update mechanism to improve the alignment between the inner-learner and meta-learner.
  arXiv  Detail & Related papers  (2021-06-08T17:59:03Z)
• Meta-learning the Learning Trends Shared Across Tasks [123.10294801296926]
  Gradient-based meta-learning algorithms excel at quick adaptation to new tasks with limited data. Existing meta-learning approaches only depend on the current task information during the adaptation. We propose a 'Path-aware' model-agnostic meta-learning approach.
  arXiv  Detail & Related papers  (2020-10-19T08:06:47Z)
• Bilevel Continual Learning [76.50127663309604]
  We present a novel framework of continual learning named "Bilevel Continual Learning" (BCL) Our experiments on continual learning benchmarks demonstrate the efficacy of the proposed BCL compared to many state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-30T16:00:23Z)
• KoGuN: Accelerating Deep Reinforcement Learning via Integrating Human Suboptimal Knowledge [40.343858932413376]
  We propose knowledge guided policy network (KoGuN), a novel framework that combines human prior suboptimal knowledge with reinforcement learning. Our framework consists of a fuzzy rule controller to represent human knowledge and a refine module to fine-tune suboptimal prior knowledge.
  arXiv  Detail & Related papers  (2020-02-18T07:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.