AFEC: Active Forgetting of Negative Transfer in Continual Learning
- URL: http://arxiv.org/abs/2110.12187v1
- Date: Sat, 23 Oct 2021 10:03:19 GMT
- Title: AFEC: Active Forgetting of Negative Transfer in Continual Learning
- Authors: Liyuan Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao,
Kaisheng Ma, Jun Zhu, Yi Zhong
- Abstract summary: We show that biological neural networks can actively forget the old knowledge that conflicts with the learning of a new experience.
Inspired by the biological active forgetting, we propose to actively forget the old knowledge that limits the learning of new tasks to benefit continual learning.
- Score: 37.03139674884091
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Continual learning aims to learn a sequence of tasks from dynamic data
distributions. Without accessing to the old training samples, knowledge
transfer from the old tasks to each new task is difficult to determine, which
might be either positive or negative. If the old knowledge interferes with the
learning of a new task, i.e., the forward knowledge transfer is negative, then
precisely remembering the old tasks will further aggravate the interference,
thus decreasing the performance of continual learning. By contrast, biological
neural networks can actively forget the old knowledge that conflicts with the
learning of a new experience, through regulating the learning-triggered
synaptic expansion and synaptic convergence. Inspired by the biological active
forgetting, we propose to actively forget the old knowledge that limits the
learning of new tasks to benefit continual learning. Under the framework of
Bayesian continual learning, we develop a novel approach named Active
Forgetting with synaptic Expansion-Convergence (AFEC). Our method dynamically
expands parameters to learn each new task and then selectively combines them,
which is formally consistent with the underlying mechanism of biological active
forgetting. We extensively evaluate AFEC on a variety of continual learning
benchmarks, including CIFAR-10 regression tasks, visual classification tasks
and Atari reinforcement tasks, where AFEC effectively improves the learning of
new tasks and achieves the state-of-the-art performance in a plug-and-play way.
Related papers
- Lifelong Sequence Generation with Dynamic Module Expansion and
Adaptation [39.886149621730915]
Lifelong sequence generation (LSG) aims to continually train a model on a sequence of generation tasks to learn constantly emerging new generation patterns.
Inspired by the learning paradigm of humans, we propose Dynamic Module Expansion and Adaptation (DMEA)
DMEA enables the model to dynamically determine the architecture for acquiring new knowledge based on task correlation and select the most similar previous tasks to facilitate adaptation to new tasks.
arXiv Detail & Related papers (2023-10-15T16:51:11Z) - Online Continual Learning via the Knowledge Invariant and Spread-out
Properties [4.109784267309124]
Key challenge in continual learning is catastrophic forgetting.
We propose a new method, named Online Continual Learning via the Knowledge Invariant and Spread-out Properties (OCLKISP)
We empirically evaluate our proposed method on four popular benchmarks for continual learning: Split CIFAR 100, Split SVHN, Split CUB200 and Split Tiny-Image-Net.
arXiv Detail & Related papers (2023-02-02T04:03:38Z) - Progressive Learning without Forgetting [8.563323015260709]
We focus on two challenging problems in the paradigm of Continual Learning (CL)
PLwF introduces functions from previous tasks to construct a knowledge space that contains the most reliable knowledge on each task.
Credit assignment controls the tug-of-war dynamics by removing gradient conflict through projection.
In comparison with other CL methods, we report notably better results even without relying on any raw data.
arXiv Detail & Related papers (2022-11-28T10:53:14Z) - Beyond Not-Forgetting: Continual Learning with Backward Knowledge
Transfer [39.99577526417276]
In continual learning (CL) an agent can improve the learning performance of both a new task and old' tasks.
Most existing CL methods focus on addressing catastrophic forgetting in neural networks by minimizing the modification of the learnt model for old tasks.
We propose a new CL method with Backward knowlEdge tRansfer (CUBER) for a fixed capacity neural network without data replay.
arXiv Detail & Related papers (2022-11-01T23:55:51Z) - Continual Prompt Tuning for Dialog State Tracking [58.66412648276873]
A desirable dialog system should be able to continually learn new skills without forgetting old ones.
We present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks.
arXiv Detail & Related papers (2022-03-13T13:22:41Z) - Learning Bayesian Sparse Networks with Full Experience Replay for
Continual Learning [54.7584721943286]
Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered.
Existing CL approaches often keep a buffer of previously-seen samples, perform knowledge distillation, or use regularization techniques towards this goal.
We propose to only activate and select sparse neurons for learning current and past tasks at any stage.
arXiv Detail & Related papers (2022-02-21T13:25:03Z) - Relational Experience Replay: Continual Learning by Adaptively Tuning
Task-wise Relationship [54.73817402934303]
We propose Experience Continual Replay (ERR), a bi-level learning framework to adaptively tune task-wise to achieve a better stability plasticity' tradeoff.
ERR can consistently improve the performance of all baselines and surpass current state-of-the-art methods.
arXiv Detail & Related papers (2021-12-31T12:05:22Z) - Continual Learning with Neuron Activation Importance [1.7513645771137178]
Continual learning is a concept of online learning with multiple sequential tasks.
One of the critical barriers of continual learning is that a network should learn a new task keeping the knowledge of old tasks without access to any data of the old tasks.
We propose a neuron activation importance-based regularization method for stable continual learning regardless of the order of tasks.
arXiv Detail & Related papers (2021-07-27T08:09:32Z) - Bilevel Continual Learning [76.50127663309604]
We present a novel framework of continual learning named "Bilevel Continual Learning" (BCL)
Our experiments on continual learning benchmarks demonstrate the efficacy of the proposed BCL compared to many state-of-the-art methods.
arXiv Detail & Related papers (2020-07-30T16:00:23Z) - Bridging the Imitation Gap by Adaptive Insubordination [88.35564081175642]
We show that when the teaching agent makes decisions with access to privileged information, this information is marginalized during imitation learning.
We propose 'Adaptive Insubordination' (ADVISOR) to address this gap.
ADVISOR dynamically weights imitation and reward-based reinforcement learning losses during training, enabling on-the-fly switching between imitation and exploration.
arXiv Detail & Related papers (2020-07-23T17:59:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.