Continual Learning with Knowledge Transfer for Sentiment Classification

• URL: http://arxiv.org/abs/2112.10021v1
• Date: Sat, 18 Dec 2021 22:58:21 GMT
• Title: Continual Learning with Knowledge Transfer for Sentiment Classification
• Authors: Zixuan Ke, Bing Liu, Hao Wang, Lei Shu
• Abstract summary: KAN can markedly improve the accuracy of both the new task and the old tasks via forward and backward knowledge transfer. The effectiveness of KAN is demonstrated through extensive experiments.
• Score: 20.5365406439092
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: This paper studies continual learning (CL) for sentiment classification (SC). In this setting, the CL system learns a sequence of SC tasks incrementally in a neural network, where each task builds a classifier to classify the sentiment of reviews of a particular product category or domain. Two natural questions are: Can the system transfer the knowledge learned in the past from the previous tasks to the new task to help it learn a better model for the new task? And, can old models for previous tasks be improved in the process as well? This paper proposes a novel technique called KAN to achieve these objectives. KAN can markedly improve the SC accuracy of both the new task and the old tasks via forward and backward knowledge transfer. The effectiveness of KAN is demonstrated through extensive experiments.

Related papers

• Fine-Grained Knowledge Selection and Restoration for Non-Exemplar Class Incremental Learning [64.14254712331116]
  Non-exemplar class incremental learning aims to learn both the new and old tasks without accessing any training data from the past. We propose a novel framework of fine-grained knowledge selection and restoration.
  arXiv  Detail & Related papers  (2023-12-20T02:34:11Z)
• LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning [64.55001982176226]
  LIBERO is a novel benchmark of lifelong learning for robot manipulation. We focus on how to efficiently transfer declarative knowledge, procedural knowledge, or the mixture of both. We develop an extendible procedural generation pipeline that can in principle generate infinitely many tasks.
  arXiv  Detail & Related papers  (2023-06-05T23:32:26Z)
• Dense Network Expansion for Class Incremental Learning [61.00081795200547]
  State-of-the-art approaches use a dynamic architecture based on network expansion (NE), in which a task expert is added per task. A new NE method, dense network expansion (DNE), is proposed to achieve a better trade-off between accuracy and model complexity. It outperforms the previous SOTA methods by a margin of 4% in terms of accuracy, with similar or even smaller model scale.
  arXiv  Detail & Related papers  (2023-03-22T16:42:26Z)
• Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer [39.99577526417276]
  In continual learning (CL) an agent can improve the learning performance of both a new task and old' tasks. Most existing CL methods focus on addressing catastrophic forgetting in neural networks by minimizing the modification of the learnt model for old tasks. We propose a new CL method with Backward knowlEdge tRansfer (CUBER) for a fixed capacity neural network without data replay.
  arXiv  Detail & Related papers  (2022-11-01T23:55:51Z)
• Effects of Auxiliary Knowledge on Continual Learning [16.84113206569365]
  In Continual Learning (CL), a neural network is trained on a stream of data whose distribution changes over time. Most existing CL approaches focus on finding solutions to preserve acquired knowledge, so working on the past of the model. We argue that as the model has to continually learn new tasks, it is also important to put focus on the present knowledge that could improve following tasks learning.
  arXiv  Detail & Related papers  (2022-06-03T14:31:59Z)
• ConTinTin: Continual Learning from Task Instructions [101.36836925135091]
  This work defines a new learning paradigm ConTinTin, in which a system should learn a sequence of new tasks one by one, each task is explained by a piece of textual instruction. To our knowledge, this is the first time to study ConTinTin in NLP.
  arXiv  Detail & Related papers  (2022-03-16T10:27:18Z)
• Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning [54.7584721943286]
  Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered. Existing CL approaches often keep a buffer of previously-seen samples, perform knowledge distillation, or use regularization techniques towards this goal. We propose to only activate and select sparse neurons for learning current and past tasks at any stage.
  arXiv  Detail & Related papers  (2022-02-21T13:25:03Z)
• Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks [22.28374603976649]
  This paper studies continual learning of a sequence of aspect sentiment classification (ASC) tasks. A CL system that incrementally learns a sequence of ASC tasks should address the following two issues. A novel capsule network based model called B-CL is proposed to address these issues.
  arXiv  Detail & Related papers  (2021-12-06T02:46:06Z)
• CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks [23.515930312505954]
  This paper studies continual learning of a sequence of aspect sentiment classification(ASC) tasks in a particular CL setting called domain incremental learning (DIL) The DIL setting is particularly suited to ASC because in testing the system needs not know the task/domain to which the test data belongs. The key novelty is a contrastive continual learning method that enables both knowledge transfer across tasks and knowledge distillation from old tasks to the new task.
  arXiv  Detail & Related papers  (2021-12-05T23:55:53Z)
• Incremental Embedding Learning via Zero-Shot Translation [65.94349068508863]
  Current state-of-the-art incremental learning methods tackle catastrophic forgetting problem in traditional classification networks. We propose a novel class-incremental method for embedding network, named as zero-shot translation class-incremental method (ZSTCI) In addition, ZSTCI can easily be combined with existing regularization-based incremental learning methods to further improve performance of embedding networks.
  arXiv  Detail & Related papers  (2020-12-31T08:21:37Z)
• Efficient Architecture Search for Continual Learning [36.998565674813285]
  Continual learning with neural networks aims to learn a sequence of tasks well. It is often confronted with three challenges: (1) overcome the catastrophic forgetting problem, (2) adapt the current network to new tasks, and (3) control its model complexity. We propose a novel approach named as Continual Learning with Efficient Architecture Search, or CLEAS in short.
  arXiv  Detail & Related papers  (2020-06-07T02:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.