Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue

• URL: http://arxiv.org/abs/2210.07783v1
• Date: Fri, 14 Oct 2022 13:12:14 GMT
• Title: Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue
• Authors: Yingxiu Zhao, Yinhe Zheng, Zhiliang Tian, Chang Gao, Bowen Yu, Haiyang Yu, Yongbin Li, Jian Sun, Nevin L. Zhang
• Abstract summary: generative replay methods are widely employed to consolidate past knowledge with generated pseudo samples. Most existing generative replay methods use only a single task-specific token to control their models. We propose a novel method, prompt conditioned VAE for lifelong learning, to enhance generative replay by incorporating tasks' statistics.
• Score: 80.05509768165135
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lifelong learning (LL) is vital for advanced task-oriented dialogue (ToD) systems. To address the catastrophic forgetting issue of LL, generative replay methods are widely employed to consolidate past knowledge with generated pseudo samples. However, most existing generative replay methods use only a single task-specific token to control their models. This scheme is usually not strong enough to constrain the generative model due to insufficient information involved. In this paper, we propose a novel method, prompt conditioned VAE for lifelong learning (PCLL), to enhance generative replay by incorporating tasks' statistics. PCLL captures task-specific distributions with a conditional variational autoencoder, conditioned on natural language prompts to guide the pseudo-sample generation. Moreover, it leverages a distillation process to further consolidate past knowledge by alleviating the noise in pseudo samples. Experiments on natural language understanding tasks of ToD systems demonstrate that PCLL significantly outperforms competitive baselines in building LL models.

Related papers

• P-RAG: Progressive Retrieval Augmented Generation For Planning on Embodied Everyday Task [94.08478298711789]
  Embodied Everyday Task is a popular task in the embodied AI community. Natural language instructions often lack explicit task planning. Extensive training is required to equip models with knowledge of the task environment.
  arXiv  Detail & Related papers  (2024-09-17T15:29:34Z)
• Pre-trained Language Models Improve the Few-shot Prompt Ability of Decision Transformer [10.338170161831496]
  Decision Transformer (DT) has emerged as a promising class of algorithms in offline reinforcement learning (RL) tasks. We introduce the Language model-d Prompt Transformer (LPDT), which leverages pre-trained language models for meta-RL tasks and fine-tunes the model using Low-rank Adaptation (LoRA) Our approach integrates pre-trained language model and RL tasks seamlessly.
  arXiv  Detail & Related papers  (2024-08-02T17:25:34Z)
• Domain Incremental Lifelong Learning in an Open World [45.704746275089555]
  We propose textbfDiana: a underlinedynamunderlineic underlinearchitecture-based lifelounderlineng leunderlinearning model. Four types of hierarchically organized prompts are used in Diana to capture knowledge from different granularities.
  arXiv  Detail & Related papers  (2023-05-11T04:19:08Z)
• Can BERT Refrain from Forgetting on Sequential Tasks? A Probing Study [68.75670223005716]
  We find that pre-trained language models like BERT have a potential ability to learn sequentially, even without any sparse memory replay. Our experiments reveal that BERT can actually generate high quality representations for previously learned tasks in a long term, under extremely sparse replay or even no replay.
  arXiv  Detail & Related papers  (2023-03-02T09:03:43Z)
• Recitation-Augmented Language Models [85.30591349383849]
  We show that RECITE is a powerful paradigm for knowledge-intensive NLP tasks. Specifically, we show that by utilizing recitation as the intermediate step, a recite-and-answer scheme can achieve new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-10-04T00:49:20Z)
• Few-shot Prompting Towards Controllable Response Generation [49.479958672988566]
  We first explored the combination of prompting and reinforcement learning (RL) to steer models' generation without accessing any of the models' parameters. We apply multi-task learning to make the model learn to generalize to new tasks better. Experiment results show that our proposed method can successfully control several state-of-the-art (SOTA) dialogue models without accessing their parameters.
  arXiv  Detail & Related papers  (2022-06-08T14:48:06Z)
• RVAE-LAMOL: Residual Variational Autoencoder to Enhance Lifelong Language Learning [13.946029695618018]
  Lifelong Language Learning (LLL) aims to train a neural network to learn a stream of NLP tasks while retaining knowledge from previous tasks. Previous works which followed data-free constraint still suffer from catastrophic forgetting issue. We propose the residual variational autoencoder (RVAE) to enhance LAMOL, a recent LLL model, by mapping different tasks into a limited unified semantic space.
  arXiv  Detail & Related papers  (2022-05-22T15:52:35Z)
• Learning to Prompt for Continual Learning [34.609384246149325]
  This work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity.
  arXiv  Detail & Related papers  (2021-12-16T06:17:07Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)
• Generative Feature Replay with Orthogonal Weight Modification for Continual Learning [20.8966035274874]
  generative replay is a promising strategy which generates and replays pseudo data for previous tasks to alleviate catastrophic forgetting. We propose to replay penultimate layer feature with a generative model; 2) leverage a self-supervised auxiliary task to further enhance the stability of feature. Empirical results on several datasets show our method always achieves substantial improvement over powerful OWM.
  arXiv  Detail & Related papers  (2020-05-07T13:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.