EM Pre-training for Multi-party Dialogue Response Generation
- URL: http://arxiv.org/abs/2305.12412v1
- Date: Sun, 21 May 2023 09:22:41 GMT
- Title: EM Pre-training for Multi-party Dialogue Response Generation
- Authors: Yiyang Li, Hai Zhao
- Abstract summary: In multi-party dialogues, the addressee of a response utterance should be specified before it is generated.
We propose an Expectation-Maximization (EM) approach that iteratively performs the expectation steps to generate addressee labels.
- Score: 86.25289241604199
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Dialogue response generation requires an agent to generate a response
according to the current dialogue history, in terms of which two-party
dialogues have been well studied, but leaving a great gap for multi-party
dialogues at the same time. Different from two-party dialogues where each
response is a direct reply to its previous utterance, the addressee of a
response utterance should be specified before it is generated in the
multi-party scenario. Thanks to the huge amount of two-party conversational
data, various pre-trained language models for two-party dialogue response
generation have been proposed. However, due to the lack of annotated addressee
labels in multi-party dialogue datasets, it is hard to use them to pre-train a
response generation model for multi-party dialogues. To tackle this obstacle,
we propose an Expectation-Maximization (EM) approach that iteratively performs
the expectation steps to generate addressee labels, and the maximization steps
to optimize a response generation model. Theoretical analyses and extensive
experiments have justified the feasibility and effectiveness of our proposed
method.
Related papers
- Pre-training Multi-party Dialogue Models with Latent Discourse Inference [85.9683181507206]
We pre-train a model that understands the discourse structure of multi-party dialogues, namely, to whom each utterance is replying.
To fully utilize the unlabeled data, we propose to treat the discourse structures as latent variables, then jointly infer them and pre-train the discourse-aware model.
arXiv Detail & Related papers (2023-05-24T14:06:27Z) - Counterfactual Data Augmentation via Perspective Transition for
Open-Domain Dialogues [34.78482218571574]
We propose a data augmentation method to automatically augment high-quality responses with different semantics by counterfactual inference.
Experimental results show that our data augmentation method can augment high-quality responses with different semantics for a given dialogue history, and can outperform competitive baselines on multiple downstream tasks.
arXiv Detail & Related papers (2022-10-30T13:26:49Z) - DialogUSR: Complex Dialogue Utterance Splitting and Reformulation for
Multiple Intent Detection [27.787807111516706]
Instead of training a dedicated multi-intent detection model, we propose DialogUSR.
DialogUSR splits multi-intent user query into several single-intent sub-queries.
It then recovers all the coreferred and omitted information in the sub-queries.
arXiv Detail & Related papers (2022-10-20T13:56:35Z) - Dialogue History Matters! Personalized Response Selectionin Multi-turn
Retrieval-based Chatbots [62.295373408415365]
We propose a personalized hybrid matching network (PHMN) for context-response matching.
Our contributions are two-fold: 1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information.
We evaluate our model on two large datasets with user identification, i.e., personalized dialogue Corpus Ubuntu (P- Ubuntu) and personalized Weibo dataset (P-Weibo)
arXiv Detail & Related papers (2021-03-17T09:42:11Z) - Learning an Effective Context-Response Matching Model with
Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination.
We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner.
Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
arXiv Detail & Related papers (2020-09-14T08:44:46Z) - Controlling Dialogue Generation with Semantic Exemplars [55.460082747572734]
We present an Exemplar-based Dialogue Generation model, EDGE, that uses the semantic frames present in exemplar responses to guide generation.
We show that controlling dialogue generation based on the semantic frames of exemplars, rather than words in the exemplar itself, improves the coherence of generated responses.
arXiv Detail & Related papers (2020-08-20T17:02:37Z) - Multi-Domain Dialogue Acts and Response Co-Generation [34.27525685962274]
We propose a neural co-generation model that generates dialogue acts and responses concurrently.
Our model achieves very favorable improvement over several state-of-the-art models in both automatic and human evaluations.
arXiv Detail & Related papers (2020-04-26T12:21:17Z) - Dialogue-Based Relation Extraction [53.2896545819799]
We present the first human-annotated dialogue-based relation extraction (RE) dataset DialogRE.
We argue that speaker-related information plays a critical role in the proposed task, based on an analysis of similarities and differences between dialogue-based and traditional RE tasks.
Experimental results demonstrate that a speaker-aware extension on the best-performing model leads to gains in both the standard and conversational evaluation settings.
arXiv Detail & Related papers (2020-04-17T03:51:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.