Precognition in Task-oriented Dialogue Understanding: Posterior Regularization by Future Context

• URL: http://arxiv.org/abs/2203.03244v1
• Date: Mon, 7 Mar 2022 09:58:50 GMT
• Title: Precognition in Task-oriented Dialogue Understanding: Posterior Regularization by Future Context
• Authors: Nan Su, Yuchi Zhang, Chao Liu, Bingzhu Du, Yongliang Wang
• Abstract summary: We propose to jointly model historical and future information through the posterior regularization method. We optimize the KL distance between these to regularize our model during training. Experiments on two dialogue datasets validate the effectiveness of our proposed method.
• Score: 8.59600111891194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Task-oriented dialogue systems have become overwhelmingly popular in recent researches. Dialogue understanding is widely used to comprehend users' intent, emotion and dialogue state in task-oriented dialogue systems. Most previous works on such discriminative tasks only models current query or historical conversations. Even if in some work the entire dialogue flow was modeled, it is not suitable for the real-world task-oriented conversations as the future contexts are not visible in such cases. In this paper, we propose to jointly model historical and future information through the posterior regularization method. More specifically, by modeling the current utterance and past contexts as prior, and the entire dialogue flow as posterior, we optimize the KL distance between these distributions to regularize our model during training. And only historical information is used for inference. Extensive experiments on two dialogue datasets validate the effectiveness of our proposed method, achieving superior results compared with all baseline models.

Related papers

• Unsupervised Extraction of Dialogue Policies from Conversations [3.102576158218633]
  We show how Large Language Models can be instrumental in extracting dialogue policies from datasets. We then propose a novel method for generating dialogue policies utilizing a controllable and interpretable graph-based methodology.
  arXiv  Detail & Related papers  (2024-06-21T14:57:25Z)
• FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue [20.79359173822053]
  We propose a novel dialogue pre-training model, FutureTOD, which distills future knowledge to the representation of the previous dialogue context. Our intuition is that a good dialogue representation both learns local context information and predicts future information.
  arXiv  Detail & Related papers  (2023-06-17T10:40:07Z)
• Pre-training Multi-party Dialogue Models with Latent Discourse Inference [85.9683181507206]
  We pre-train a model that understands the discourse structure of multi-party dialogues, namely, to whom each utterance is replying. To fully utilize the unlabeled data, we propose to treat the discourse structures as latent variables, then jointly infer them and pre-train the discourse-aware model.
  arXiv  Detail & Related papers  (2023-05-24T14:06:27Z)
• DialoGen: Generalized Long-Range Context Representation for Dialogue Systems [36.23733762476647]
  We propose DialoGen, a novel framework for dialogue generation with a generalized context representation. We study the effectiveness of our proposed method on both dialogue generation (open-domain) and understanding (DST)
  arXiv  Detail & Related papers  (2022-10-12T15:05:28Z)
• GODEL: Large-Scale Pre-Training for Goal-Directed Dialog [119.1397031992088]
  We introduce GODEL, a large pre-trained language model for dialog. We show that GODEL outperforms state-of-the-art pre-trained dialog models in few-shot fine-tuning setups. A novel feature of our evaluation methodology is the introduction of a notion of utility that assesses the usefulness of responses.
  arXiv  Detail & Related papers  (2022-06-22T18:19:32Z)
• Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling [80.51094098799736]
  We propose Bidirectional Information Decoupling Network (BiDeN) as a universal dialogue encoder. BiDeN explicitly incorporates both the past and future contexts and can be generalized to a wide range of dialogue-related tasks. Experimental results on datasets of different downstream tasks demonstrate the universality and effectiveness of our BiDeN.
  arXiv  Detail & Related papers  (2022-04-18T03:51:46Z)
• "How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations [87.95711406978157]
  This work presents a new benchmark on spoken task-oriented conversations. We study multi-domain dialogue state tracking and knowledge-grounded dialogue modeling. Our data set enables speech-based benchmarking of task-oriented dialogue systems.
  arXiv  Detail & Related papers  (2021-09-28T04:51:04Z)
• Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues [73.04906599884868]
  We propose a novel framework of Reasoning Paths in Dialogue Context (PDC) PDC model discovers information flows among dialogue turns through a semantic graph constructed based on lexical components in each question and answer. Our model sequentially processes both visual and textual information through this reasoning path and the propagated features are used to generate the answer.
  arXiv  Detail & Related papers  (2021-03-01T07:39:26Z)
• Ranking Enhanced Dialogue Generation [77.8321855074999]
  How to effectively utilize the dialogue history is a crucial problem in multi-turn dialogue generation. Previous works usually employ various neural network architectures to model the history. This paper proposes a Ranking Enhanced Dialogue generation framework.
  arXiv  Detail & Related papers  (2020-08-13T01:49:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.