TopicRefine: Joint Topic Prediction and Dialogue Response Generation for Multi-turn End-to-End Dialogue System

• URL: http://arxiv.org/abs/2109.05187v1
• Date: Sat, 11 Sep 2021 04:43:07 GMT
• Title: TopicRefine: Joint Topic Prediction and Dialogue Response Generation for Multi-turn End-to-End Dialogue System
• Authors: Hongru Wang, Mingyu Cui, Zimo Zhou, Gabriel Pui Cheong Fung, Kam-Fai Wong
• Abstract summary: A multi-turn dialogue always follows a specific topic thread, and topic shift at the discourse level occurs naturally. Previous research has either predicted the topic first and then generated the relevant response, or simply applied the attention mechanism to all topics. We propose a joint framework with a topic refinement mechanism to learn these two tasks simultaneously.
• Score: 12.135300607779753
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: A multi-turn dialogue always follows a specific topic thread, and topic shift at the discourse level occurs naturally as the conversation progresses, necessitating the model's ability to capture different topics and generate topic-aware responses. Previous research has either predicted the topic first and then generated the relevant response, or simply applied the attention mechanism to all topics, ignoring the joint distribution of the topic prediction and response generation models and resulting in uncontrollable and unrelated responses. In this paper, we propose a joint framework with a topic refinement mechanism to learn these two tasks simultaneously. Specifically, we design a three-pass iteration mechanism to generate coarse response first, then predict corresponding topics, and finally generate refined response conditioned on predicted topics. Moreover, we utilize GPT2DoubleHeads and BERT for the topic prediction task respectively, aiming to investigate the effects of joint learning and the understanding ability of GPT model. Experimental results demonstrate that our proposed framework achieves new state-of-the-art performance at response generation task and the great potential understanding capability of GPT model.

Related papers

• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• Sequential Topic Selection Model with Latent Variable for Topic-Grounded Dialogue [21.1427816176227]
  We propose a novel approach, named Sequential Global Topic Attention (SGTA) to exploit topic transition over all conversations. Our model outperforms competitive baselines on prediction and generation tasks.
  arXiv  Detail & Related papers  (2022-10-17T07:34:14Z)
• OTTers: One-turn Topic Transitions for Open-Domain Dialogue [11.305029351461306]
  Mixed initiative in open-domain dialogue requires a system to pro-actively introduce new topics. One-turn topic transition task explores how a system connects two topics in a cooperative and coherent manner.
  arXiv  Detail & Related papers  (2021-05-28T10:16:59Z)
• Who Responded to Whom: The Joint Effects of Latent Topics and Discourse in Conversation Structure [53.77234444565652]
  We identify the responding relations in the conversation discourse, which link response utterances to their initiations. We propose a model to learn latent topics and discourse in word distributions, and predict pairwise initiation-response links. Experimental results on both English and Chinese conversations show that our model significantly outperforms the previous state of the arts.
  arXiv  Detail & Related papers  (2021-04-17T17:46:00Z)
• Response Selection for Multi-Party Conversations with Dynamic Topic Tracking [63.15158355071206]
  We frame response selection as a dynamic topic tracking task to match the topic between the response and relevant conversation context. We propose a novel multi-task learning framework that supports efficient encoding through large pretrained models. Experimental results on the DSTC-8 Ubuntu IRC dataset show state-of-the-art results in response selection and topic disentanglement tasks.
  arXiv  Detail & Related papers  (2020-10-15T14:21:38Z)
• Modeling Topical Relevance for Multi-Turn Dialogue Generation [61.87165077442267]
  We propose a new model, named STAR-BTM, to tackle the problem of topic drift in multi-turn dialogue. The Biterm Topic Model is pre-trained on the whole training dataset. Then, the topic level attention weights are computed based on the topic representation of each context. Experimental results on both Chinese customer services data and English Ubuntu dialogue data show that STAR-BTM significantly outperforms several state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-27T03:33:22Z)
• Topic-Aware Multi-turn Dialogue Modeling [91.52820664879432]
  This paper presents a novel solution for multi-turn dialogue modeling, which segments and extracts topic-aware utterances in an unsupervised way. Our topic-aware modeling is implemented by a newly proposed unsupervised topic-aware segmentation algorithm and Topic-Aware Dual-attention Matching (TADAM) Network.
  arXiv  Detail & Related papers  (2020-09-26T08:43:06Z)
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
  We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination. We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
  arXiv  Detail & Related papers  (2020-09-14T08:44:46Z)
• Gated Convolutional Bidirectional Attention-based Model for Off-topic Spoken Response Detection [10.321357718530473]
  We propose a novel approach for off-topic spoken response detection with high off-topic recall on both seen and unseen prompts. We introduce a new model, Gated Convolutional Bidirectional Attention-based Model (GCBiA), which applies bi-attention mechanism and convolutions to extract topic words of prompts and key-phrases of responses.
  arXiv  Detail & Related papers  (2020-04-20T03:16:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.