Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training

• URL: http://arxiv.org/abs/2105.15171v1
• Date: Mon, 31 May 2021 17:28:37 GMT
• Title: Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
• Authors: Wangchunshu Zhou, Qifei Li, Chenle Li
• Abstract summary: We propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems. IAT encourages the model to be sensitive to the perturbation in the dialogue history and therefore learning from perturbations. We show that our approach can better model dialogue history and generate more diverse and consistent responses.
• Score: 10.17868476063421
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. In contrast to standard adversarial training algorithms, IAT encourages the model to be sensitive to the perturbation in the dialogue history and therefore learning from perturbations. By giving higher rewards for responses whose output probability reduces more significantly when dialogue history is perturbed, the model is encouraged to generate more diverse and consistent responses. By penalizing the model when generating the same response given perturbed dialogue history, the model is forced to better capture dialogue history and generate more informative responses. Experimental results on two benchmark datasets show that our approach can better model dialogue history and generate more diverse and consistent responses. In addition, we point out a problem of the widely used maximum mutual information (MMI) based methods for improving the diversity of dialogue response generation models and demonstrate it empirically.

Related papers

• PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems [59.1250765143521]
  Current knowledge-grounded dialogue systems often fail to align the generated responses with human-preferred qualities. We propose Polished & Informed Candidate Scoring (PICK), a generation re-scoring framework. We demonstrate the effectiveness of PICK in generating responses that are more faithful while keeping them relevant to the dialogue history.
  arXiv  Detail & Related papers  (2023-09-19T08:27:09Z)
• Promoting Open-domain Dialogue Generation through Learning Pattern Information between Contexts and Responses [5.936682548344234]
  This paper improves the quality of generated responses by learning the implicit pattern information between contexts and responses in the training samples. We also design a response-aware mechanism for mining the implicit pattern information between contexts and responses so that the generated replies are more diverse and approximate to human replies.
  arXiv  Detail & Related papers  (2023-09-06T08:11:39Z)
• EM Pre-training for Multi-party Dialogue Response Generation [86.25289241604199]
  In multi-party dialogues, the addressee of a response utterance should be specified before it is generated. We propose an Expectation-Maximization (EM) approach that iteratively performs the expectation steps to generate addressee labels.
  arXiv  Detail & Related papers  (2023-05-21T09:22:41Z)
• Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation [57.73547958927826]
  We propose to refine the user dialogue history on a large scale, based on which we can handle more dialogue history and obtain more accurate persona information. Specifically, we design an MSP model which consists of three personal information refiners and a personalized response generator.
  arXiv  Detail & Related papers  (2022-04-18T02:02:56Z)
• DialAug: Mixing up Dialogue Contexts in Contrastive Learning for Robust Conversational Modeling [3.3578533367912025]
  We propose a framework that incorporates augmented versions of a dialogue context into the learning objective. We show that our proposed augmentation method outperforms previous data augmentation approaches.
  arXiv  Detail & Related papers  (2022-04-15T23:39:41Z)
• Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration [45.14559188965439]
  We propose to apply reinforcement learning to refine an MLE-based language model without user simulators. We distill sentence-level information about repetition, inconsistency and task relevance through rewards. Experiments show that our model outperforms previous state-of-the-art dialogue models on both automatic metrics and human evaluation results.
  arXiv  Detail & Related papers  (2020-12-31T00:02:51Z)
• DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances [18.199473005335093]
  This paper presents DialogBERT, a novel conversational response generation model that enhances previous PLM-based dialogue models. To efficiently capture the discourse-level coherence among utterances, we propose two training objectives, including masked utterance regression. Experiments on three multi-turn conversation datasets show that our approach remarkably outperforms the baselines.
  arXiv  Detail & Related papers  (2020-12-03T09:06:23Z)
• Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data [61.71319905364992]
  We propose a novel data augmentation method for training open-domain dialogue models by utilizing unpaired data. A data-level distillation process is first proposed to construct augmented dialogues where both post and response are retrieved from the unpaired data. A ranking module is employed to filter out low-quality dialogues. A model-level distillation process is employed to distill a teacher model trained on high-quality paired data to augmented dialogue pairs.
  arXiv  Detail & Related papers  (2020-09-20T13:06:38Z)
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
  We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination. We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
  arXiv  Detail & Related papers  (2020-09-14T08:44:46Z)
• Ranking Enhanced Dialogue Generation [77.8321855074999]
  How to effectively utilize the dialogue history is a crucial problem in multi-turn dialogue generation. Previous works usually employ various neural network architectures to model the history. This paper proposes a Ranking Enhanced Dialogue generation framework.
  arXiv  Detail & Related papers  (2020-08-13T01:49:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.