Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection

• URL: http://arxiv.org/abs/2204.07275v1
• Date: Fri, 15 Apr 2022 00:21:31 GMT
• Title: Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection
• Authors: Minqian Liu, Shiyu Chang, Lifu Huang
• Abstract summary: Lifelong event detection aims to incrementally update a model with new event types and data. One critical challenge is that the model would catastrophically forget old types when continually trained on new data. We introduce Episodic Memory Prompts (EMP) to explicitly preserve the learned task-specific knowledge.
• Score: 41.74511506187945
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lifelong event detection aims to incrementally update a model with new event types and data while retaining the capability on previously learned old types. One critical challenge is that the model would catastrophically forget old types when continually trained on new data. In this paper, we introduce Episodic Memory Prompts (EMP) to explicitly preserve the learned task-specific knowledge. Our method adopts continuous prompt for each task and they are optimized to instruct the model prediction and learn event-specific representation. The EMPs learned in previous tasks are carried along with the model in subsequent tasks, and can serve as a memory module that keeps the old knowledge and transferring to new tasks. Experiment results demonstrate the effectiveness of our method. Furthermore, we also conduct a comprehensive analysis of the new and old event types in lifelong learning.

Related papers

• Lifelong Event Detection with Embedding Space Separation and Compaction [30.05158209938146]
  Existing lifelong event detection methods typically maintain a memory module and replay the stored memory data during the learning of a new task. The simple combination of memory data and new-task samples can still result in substantial forgetting of previously acquired knowledge. We propose a novel method based on embedding space separation and compaction.
  arXiv  Detail & Related papers  (2024-04-03T06:51:49Z)
• Class-Incremental Few-Shot Event Detection [68.66116956283575]
  This paper proposes a new task, called class-incremental few-shot event detection. This task faces two problems, i.e., old knowledge forgetting and new class overfitting. To solve these problems, this paper presents a novel knowledge distillation and prompt learning based method, called Prompt-KD.
  arXiv  Detail & Related papers  (2024-04-02T09:31:14Z)
• Continual Few-shot Event Detection via Hierarchical Augmentation Networks [21.574099641753055]
  We introduce continual few-shot event detection (CFED), a more commonly encountered scenario when a substantial number of labeled samples are not accessible. The CFED task is challenging as it involves memorizing previous event types and learning new event types with few-shot samples. Our method significantly outperforms all of these methods in multiple continual few-shot event detection tasks.
  arXiv  Detail & Related papers  (2024-03-26T14:20:42Z)
• Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Study [68.75670223005716]
  We find that pre-trained language models like BERT have a potential ability to learn sequentially, even without any sparse memory replay. Our experiments reveal that BERT can actually generate high quality representations for previously learned tasks in a long term, under extremely sparse replay or even no replay.
  arXiv  Detail & Related papers  (2023-03-02T09:03:43Z)
• A Memory Transformer Network for Incremental Learning [64.0410375349852]
  We study class-incremental learning, a training setup in which new classes of data are observed over time for the model to learn from. Despite the straightforward problem formulation, the naive application of classification models to class-incremental learning results in the "catastrophic forgetting" of previously seen classes. One of the most successful existing methods has been the use of a memory of exemplars, which overcomes the issue of catastrophic forgetting by saving a subset of past data into a memory bank and utilizing it to prevent forgetting when training future tasks.
  arXiv  Detail & Related papers  (2022-10-10T08:27:28Z)
• Learning to Prompt for Continual Learning [34.609384246149325]
  This work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity.
  arXiv  Detail & Related papers  (2021-12-16T06:17:07Z)
• Lifelong Learning of Few-shot Learners across NLP Tasks [45.273018249235705]
  We study the challenge of lifelong learning to few-shot learn over a sequence of diverse NLP tasks. We propose a continual meta-learning approach which learns to generate adapter weights from a few examples. We demonstrate our approach preserves model performance over training tasks and leads to positive knowledge transfer when the future tasks are learned.
  arXiv  Detail & Related papers  (2021-04-18T10:41:56Z)
• Learning to Learn Variational Semantic Memory [132.39737669936125]
  We introduce variational semantic memory into meta-learning to acquire long-term knowledge for few-shot learning. The semantic memory is grown from scratch and gradually consolidated by absorbing information from tasks it experiences. We formulate memory recall as the variational inference of a latent memory variable from addressed contents.
  arXiv  Detail & Related papers  (2020-10-20T15:05:26Z)
• Extensively Matching for Few-shot Learning Event Detection [66.31312496170139]
  Event detection models under super-vised learning settings fail to transfer to new event types. Few-shot learning has not beenexplored in event detection. We propose two novelloss factors that matching examples in the sup-port set to provide more training signals to themodel.
  arXiv  Detail & Related papers  (2020-06-17T18:30:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.