Modeling Topical Relevance for Multi-Turn Dialogue Generation

• URL: http://arxiv.org/abs/2009.12735v1
• Date: Sun, 27 Sep 2020 03:33:22 GMT
• Title: Modeling Topical Relevance for Multi-Turn Dialogue Generation
• Authors: Hainan Zhang, Yanyan Lan, Liang Pang, Hongshen Chen, Zhuoye Ding and Dawei Yin
• Abstract summary: We propose a new model, named STAR-BTM, to tackle the problem of topic drift in multi-turn dialogue. The Biterm Topic Model is pre-trained on the whole training dataset. Then, the topic level attention weights are computed based on the topic representation of each context. Experimental results on both Chinese customer services data and English Ubuntu dialogue data show that STAR-BTM significantly outperforms several state-of-the-art methods.
• Score: 61.87165077442267
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Topic drift is a common phenomenon in multi-turn dialogue. Therefore, an ideal dialogue generation models should be able to capture the topic information of each context, detect the relevant context, and produce appropriate responses accordingly. However, existing models usually use word or sentence level similarities to detect the relevant contexts, which fail to well capture the topical level relevance. In this paper, we propose a new model, named STAR-BTM, to tackle this problem. Firstly, the Biterm Topic Model is pre-trained on the whole training dataset. Then, the topic level attention weights are computed based on the topic representation of each context. Finally, the attention weights and the topic distribution are utilized in the decoding process to generate the corresponding responses. Experimental results on both Chinese customer services data and English Ubuntu dialogue data show that STAR-BTM significantly outperforms several state-of-the-art methods, in terms of both metric-based and human evaluations.

Related papers

• Interactive Topic Models with Optimal Transport [75.26555710661908]
  We present EdTM, as an approach for label name supervised topic modeling. EdTM models topic modeling as an assignment problem while leveraging LM/LLM based document-topic affinities.
  arXiv  Detail & Related papers  (2024-06-28T13:57:27Z)
• Multi-turn Dialogue Comprehension from a Topic-aware Perspective [70.37126956655985]
  This paper proposes to model multi-turn dialogues from a topic-aware perspective. We use a dialogue segmentation algorithm to split a dialogue passage into topic-concentrated fragments in an unsupervised way. We also present a novel model, Topic-Aware Dual-Attention Matching (TADAM) Network, which takes topic segments as processing elements.
  arXiv  Detail & Related papers  (2023-09-18T11:03:55Z)
• Multi-Granularity Prompts for Topic Shift Detection in Dialogue [13.739991183173494]
  The goal of dialogue topic shift detection is to identify whether the current topic in a conversation has changed or needs to change. Previous work focused on detecting topic shifts using pre-trained models to encode the utterance. We take a prompt-based approach to fully extract topic information from dialogues at multiple-granularity, i.e., label, turn, and topic.
  arXiv  Detail & Related papers  (2023-05-23T12:35:49Z)
• DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization [127.714919036388]
  DIONYSUS is a pre-trained encoder-decoder model for summarizing dialogues in any new domain. Our experiments show that DIONYSUS outperforms existing methods on six datasets.
  arXiv  Detail & Related papers  (2022-12-20T06:21:21Z)
• Context Matters in Semantically Controlled Language Generation for Task-oriented Dialogue Systems [6.1478669848771546]
  This work combines information about the dialogue history encoded by pre-trained model with a meaning representation of the current system utterance to realize contextual language generation in task-oriented dialogues. We utilize the pre-trained multi-context ConveRT model for context representation in a model trained from scratch; and leverage the immediate preceding user utterance for context generation in a model adapted from the pre-trained GPT-2.
  arXiv  Detail & Related papers  (2021-11-28T11:48:02Z)
• Topic-Aware Multi-turn Dialogue Modeling [91.52820664879432]
  This paper presents a novel solution for multi-turn dialogue modeling, which segments and extracts topic-aware utterances in an unsupervised way. Our topic-aware modeling is implemented by a newly proposed unsupervised topic-aware segmentation algorithm and Topic-Aware Dual-attention Matching (TADAM) Network.
  arXiv  Detail & Related papers  (2020-09-26T08:43:06Z)
• An Empirical Investigation of Pre-Trained Transformer Language Models for Open-Domain Dialogue Generation [23.343006562849126]
  We present an empirical investigation of pre-trained Transformer-based auto-regressive language models for the task of open-domain dialogue generation. Training paradigm of pre-training and fine-tuning is employed to conduct learning. Experiments are conducted on the typical single-turn and multi-turn dialogue corpora such as Weibo, Douban, Reddit, DailyDialog, and Persona-Chat.
  arXiv  Detail & Related papers  (2020-03-09T15:20:21Z)
• Modality-Balanced Models for Visual Dialogue [102.35406085738325]
  The Visual Dialog task requires a model to exploit both image and conversational context information to generate the next response to the dialogue. We show that previous joint-modality (history and image) models over-rely on and are more prone to memorizing the dialogue history. We present methods for this integration of the two models, via ensemble and consensus dropout fusion with shared parameters.
  arXiv  Detail & Related papers  (2020-01-17T14:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.