I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling

• URL: http://arxiv.org/abs/2012.13391v2
• Date: Mon, 28 Dec 2020 18:32:21 GMT
• Title: I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling
• Authors: Yixin Nie, Mary Williamson, Mohit Bansal, Douwe Kiela, Jason Weston
• Abstract summary: We introduce the DialoguE COntradiction DEtection task (DECODE) and a new conversational dataset containing both human-human and human-bot contradictory dialogues. We then compare a structured utterance-based approach of using pre-trained Transformer models for contradiction detection with the typical unstructured approach.
• Score: 104.09033240889106
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To quantify how well natural language understanding models can capture consistency in a general conversation, we introduce the DialoguE COntradiction DEtection task (DECODE) and a new conversational dataset containing both human-human and human-bot contradictory dialogues. We then compare a structured utterance-based approach of using pre-trained Transformer models for contradiction detection with the typical unstructured approach. Results reveal that: (i) our newly collected dataset is notably more effective at providing supervision for the dialogue contradiction detection task than existing NLI data including those aimed to cover the dialogue domain; (ii) the structured utterance-based approach is more robust and transferable on both analysis and out-of-distribution dialogues than its unstructured counterpart. We also show that our best contradiction detection model correlates well with human judgments and further provide evidence for its usage in both automatically evaluating and improving the consistency of state-of-the-art generative chatbots.

Related papers

• Evaluating Robustness of Dialogue Summarization Models in the Presence of Naturally Occurring Variations [13.749495524988774]
  We systematically investigate the impact of real-life variations on state-of-the-art dialogue summarization models. We introduce two types of perturbations: utterance-level perturbations that modify individual utterances with errors and language variations, and dialogue-level perturbations that add non-informative exchanges. We find that both fine-tuned and instruction-tuned models are affected by input variations, with the latter being more susceptible.
  arXiv  Detail & Related papers  (2023-11-15T05:11:43Z)
• Pre-training Multi-party Dialogue Models with Latent Discourse Inference [85.9683181507206]
  We pre-train a model that understands the discourse structure of multi-party dialogues, namely, to whom each utterance is replying. To fully utilize the unlabeled data, we propose to treat the discourse structures as latent variables, then jointly infer them and pre-train the discourse-aware model.
  arXiv  Detail & Related papers  (2023-05-24T14:06:27Z)
• Controllable Mixed-Initiative Dialogue Generation through Prompting [50.03458333265885]
  Mixed-initiative dialogue tasks involve repeated exchanges of information and conversational control. Agents gain control by generating responses that follow particular dialogue intents or strategies, prescribed by a policy planner. Standard approach has been fine-tuning pre-trained language models to perform generation conditioned on these intents. We instead prompt large language models as a drop-in replacement to fine-tuning on conditional generation.
  arXiv  Detail & Related papers  (2023-05-06T23:11:25Z)
• STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension [42.57581945778631]
  Abstractive dialogue summarization has long been viewed as an important standalone task in natural language processing. We propose a novel type of dialogue summarization task - STRUctured DiaLoguE Summarization. We show that our STRUDEL dialogue comprehension model can significantly improve the dialogue comprehension performance of transformer encoder language models.
  arXiv  Detail & Related papers  (2022-12-24T04:39:54Z)
• Enhancing Semantic Understanding with Self-supervised Methods for Abstractive Dialogue Summarization [4.226093500082746]
  We introduce self-supervised methods to compensate shortcomings to train a dialogue summarization model. Our principle is to detect incoherent information flows using pretext dialogue text to enhance BERT's ability to contextualize the dialogue text representations.
  arXiv  Detail & Related papers  (2022-09-01T07:51:46Z)
• Learning Locality and Isotropy in Dialogue Modeling [28.743212772593335]
  We propose a simple method for dialogue representation calibration, namely SimDRC, to build isotropic and conversational feature spaces. Experimental results show that our approach significantly outperforms the current state-of-the-art models on three dialogue tasks.
  arXiv  Detail & Related papers  (2022-05-29T06:48:53Z)
• DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations [46.942369532632604]
  We propose a Dialogue Evaluation metric that relies on AMR-based semantic manipulations for incoherent data generation. Our experiments show that DEAM achieves higher correlations with human judgments compared to baseline methods.
  arXiv  Detail & Related papers  (2022-03-18T03:11:35Z)
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
  We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination. We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
  arXiv  Detail & Related papers  (2020-09-14T08:44:46Z)
• Modeling Long Context for Task-Oriented Dialogue State Generation [51.044300192906995]
  We propose a multi-task learning model with a simple yet effective utterance tagging technique and a bidirectional language model. Our approaches attempt to solve the problem that the performance of the baseline significantly drops when the input dialogue context sequence is long. In our experiments, our proposed model achieves a 7.03% relative improvement over the baseline, establishing a new state-of-the-art joint goal accuracy of 52.04% on the MultiWOZ 2.0 dataset.
  arXiv  Detail & Related papers  (2020-04-29T11:02:25Z)
• Variational Hierarchical Dialog Autoencoder for Dialog State Tracking Data Augmentation [59.174903564894954]
  In this work, we extend this approach to the task of dialog state tracking for goal-oriented dialogs. We propose the Variational Hierarchical Dialog Autoencoder (VHDA) for modeling the complete aspects of goal-oriented dialogs. Experiments on various dialog datasets show that our model improves the downstream dialog trackers' robustness via generative data augmentation.
  arXiv  Detail & Related papers  (2020-01-23T15:34:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.