Relational Experience Replay: Continual Learning by Adaptively Tuning Task-wise Relationship

• URL: http://arxiv.org/abs/2112.15402v3
• Date: Thu, 3 Aug 2023 14:00:42 GMT
• Title: Relational Experience Replay: Continual Learning by Adaptively Tuning Task-wise Relationship
• Authors: Quanziang Wang, Renzhen Wang, Yuexiang Li, Dong Wei, Kai Ma, Yefeng Zheng, Deyu Meng
• Abstract summary: We propose Experience Continual Replay (ERR), a bi-level learning framework to adaptively tune task-wise to achieve a better stability plasticity' tradeoff. ERR can consistently improve the performance of all baselines and surpass current state-of-the-art methods.
• Score: 54.73817402934303
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual learning is a promising machine learning paradigm to learn new tasks while retaining previously learned knowledge over streaming training data. Till now, rehearsal-based methods, keeping a small part of data from old tasks as a memory buffer, have shown good performance in mitigating catastrophic forgetting for previously learned knowledge. However, most of these methods typically treat each new task equally, which may not adequately consider the relationship or similarity between old and new tasks. Furthermore, these methods commonly neglect sample importance in the continual training process and result in sub-optimal performance on certain tasks. To address this challenging problem, we propose Relational Experience Replay (RER), a bi-level learning framework, to adaptively tune task-wise relationships and sample importance within each task to achieve a better `stability' and `plasticity' trade-off. As such, the proposed method is capable of accumulating new knowledge while consolidating previously learned old knowledge during continual learning. Extensive experiments conducted on three publicly available datasets (i.e., CIFAR-10, CIFAR-100, and Tiny ImageNet) show that the proposed method can consistently improve the performance of all baselines and surpass current state-of-the-art methods.

Related papers

• Replay-enhanced Continual Reinforcement Learning [37.34722105058351]
  We introduce RECALL, a replay-enhanced method that greatly improves the plasticity of existing replay-based methods on new tasks. Experiments on the Continual World benchmark show that RECALL performs significantly better than purely perfect memory replay.
  arXiv  Detail & Related papers  (2023-11-20T06:21:52Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Online Continual Learning via the Knowledge Invariant and Spread-out Properties [4.109784267309124]
  Key challenge in continual learning is catastrophic forgetting. We propose a new method, named Online Continual Learning via the Knowledge Invariant and Spread-out Properties (OCLKISP) We empirically evaluate our proposed method on four popular benchmarks for continual learning: Split CIFAR 100, Split SVHN, Split CUB200 and Split Tiny-Image-Net.
  arXiv  Detail & Related papers  (2023-02-02T04:03:38Z)
• Continual Prompt Tuning for Dialog State Tracking [58.66412648276873]
  A desirable dialog system should be able to continually learn new skills without forgetting old ones. We present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks.
  arXiv  Detail & Related papers  (2022-03-13T13:22:41Z)
• Consistent Representation Learning for Continual Relation Extraction [18.694012937149495]
  A consistent representation learning method is proposed, which maintains the stability of the relation embedding. Our method significantly outperforms state-of-the-art baselines and yield strong robustness on the imbalanced dataset.
  arXiv  Detail & Related papers  (2022-03-05T12:16:34Z)
• Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation [4.111899441919165]
  It is necessary for the model to learn novel relational patterns with very few labeled data while avoiding catastrophic forgetting of previous task knowledge. We propose a novel method based on embedding space regularization and data augmentation. Our method generalizes to new few-shot tasks and avoids catastrophic forgetting of previous tasks by enforcing extra constraints on the relational embeddings and by adding extra relevant data in a self-supervised manner.
  arXiv  Detail & Related papers  (2022-03-04T05:19:09Z)
• Learning Invariant Representation for Continual Learning [5.979373021392084]
  A key challenge in Continual learning is catastrophically forgetting previously learned tasks when the agent faces a new one. We propose a new pseudo-rehearsal-based method, named learning Invariant Representation for Continual Learning (IRCL) Disentangling the shared invariant representation helps to learn continually a sequence of tasks, while being more robust to forgetting and having better knowledge transfer.
  arXiv  Detail & Related papers  (2021-01-15T15:12:51Z)
• Parrot: Data-Driven Behavioral Priors for Reinforcement Learning [79.32403825036792]
  We propose a method for pre-training behavioral priors that can capture complex input-output relationships observed in successful trials. We show how this learned prior can be used for rapidly learning new tasks without impeding the RL agent's ability to try out novel behaviors.
  arXiv  Detail & Related papers  (2020-11-19T18:47:40Z)
• Bilevel Continual Learning [76.50127663309604]
  We present a novel framework of continual learning named "Bilevel Continual Learning" (BCL) Our experiments on continual learning benchmarks demonstrate the efficacy of the proposed BCL compared to many state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-30T16:00:23Z)
• Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling [126.69933134648541]
  We present a meta-reinforcement learning algorithm that is both efficient and extrapolates well when faced with out-of-distribution tasks at test time. Our method is based on a simple insight: we recognize that dynamics models can be adapted efficiently and consistently with off-policy data.
  arXiv  Detail & Related papers  (2020-06-12T13:34:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.