A Unified Pre-training Framework for Conversational AI
- URL: http://arxiv.org/abs/2105.02482v1
- Date: Thu, 6 May 2021 07:27:11 GMT
- Title: A Unified Pre-training Framework for Conversational AI
- Authors: Siqi Bao, Bingjin Chen, Huang He, Xin Tian, Han Zhou, Fan Wang, Hua
Wu, Haifeng Wang, Wenquan Wu, Yingzhan Lin
- Abstract summary: PLATO-2 is trained via two-stage curriculum learning to fit the simplified one-to-one mapping relationship.
PLATO-2 has obtained the 1st place in all three tasks, verifying its effectiveness as a unified framework for various dialogue systems.
- Score: 25.514505462661763
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this work, we explore the application of PLATO-2 on various dialogue
systems, including open-domain conversation, knowledge grounded dialogue, and
task-oriented conversation. PLATO-2 is initially designed as an open-domain
chatbot, trained via two-stage curriculum learning. In the first stage, a
coarse-grained response generation model is learned to fit the simplified
one-to-one mapping relationship. This model is applied to the task-oriented
conversation, given that the semantic mappings tend to be deterministic in task
completion. In the second stage, another fine-grained generation model and an
evaluation model are further learned for diverse response generation and
coherence estimation, respectively. With superior capability on capturing
one-to-many mapping, such models are suitable for the open-domain conversation
and knowledge grounded dialogue. For the comprehensive evaluation of PLATO-2,
we have participated in multiple tasks of DSTC9, including interactive
evaluation of open-domain conversation (Track3-task2), static evaluation of
knowledge grounded dialogue (Track3-task1), and end-to-end task-oriented
conversation (Track2-task1). PLATO-2 has obtained the 1st place in all three
tasks, verifying its effectiveness as a unified framework for various dialogue
systems.
Related papers
- Unified Conversational Models with System-Initiated Transitions between
Chit-Chat and Task-Oriented Dialogues [4.714297769572548]
We investigate the potential initiative'' that occurs when there is a change between dialogue modes in one dialogue.
We contribute two efficient prompt models which can proactively generate a transition sentence to trigger system-initiated transitions.
arXiv Detail & Related papers (2023-07-04T11:53:23Z) - SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for
Task-Oriented Dialog Understanding [68.94808536012371]
We propose a tree-structured pre-trained conversation model, which learns dialog representations from limited labeled dialogs and large-scale unlabeled dialog corpora.
Our method can achieve new state-of-the-art results on the DialoGLUE benchmark consisting of seven datasets and four popular dialog understanding tasks.
arXiv Detail & Related papers (2022-09-14T13:42:50Z) - Interactive Evaluation of Dialog Track at DSTC9 [8.2208199207543]
The Interactive Evaluation of Dialog Track was introduced at the 9th Dialog System Technology Challenge.
This paper provides an overview of the track, including the methodology and results.
arXiv Detail & Related papers (2022-07-28T22:54:04Z) - KETOD: Knowledge-Enriched Task-Oriented Dialogue [77.59814785157877]
Existing studies in dialogue system research mostly treat task-oriented dialogue and chit-chat as separate domains.
We investigate how task-oriented dialogue and knowledge-grounded chit-chat can be effectively integrated into a single model.
arXiv Detail & Related papers (2022-05-11T16:01:03Z) - TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue
Modeling on Spoken Conversations [24.245354500835465]
We propose a novel model-agnostic data augmentation paradigm to boost the robustness of task-oriented dialogue modeling on spoken conversations.
Our approach ranked first in both tasks of DSTC10 Track2, a benchmark for task-oriented dialogue modeling on spoken conversations.
arXiv Detail & Related papers (2021-12-23T10:04:25Z) - Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System [26.837972034630003]
PPTOD is a unified plug-and-play model for task-oriented dialogue.
We extensively test our model on three benchmark TOD tasks, including end-to-end dialogue modelling, dialogue state tracking, and intent classification.
arXiv Detail & Related papers (2021-09-29T22:02:18Z) - Overview of the Ninth Dialog System Technology Challenge: DSTC9 [111.35889309106359]
The Ninth Dialog System Technology Challenge (DSTC-9) focuses on applying end-to-end dialog technologies for four distinct tasks in dialog systems.
This paper describes the task definition, provided datasets, baselines and evaluation set-up for each track.
arXiv Detail & Related papers (2020-11-12T16:43:10Z) - DCR-Net: A Deep Co-Interactive Relation Network for Joint Dialog Act
Recognition and Sentiment Classification [77.59549450705384]
In dialog system, dialog act recognition and sentiment classification are two correlative tasks.
Most of the existing systems either treat them as separate tasks or just jointly model the two tasks.
We propose a Deep Co-Interactive Relation Network (DCR-Net) to explicitly consider the cross-impact and model the interaction between the two tasks.
arXiv Detail & Related papers (2020-08-16T14:13:32Z) - Point or Generate Dialogue State Tracker [0.0]
We propose the Point-Or-Generate Dialogue State Tracker (POGD)
POGD points out explicitly expressed slot values from the user's utterance, and generates implicitly expressed ones based on slot-specific contexts.
Experiments show that POGD not only obtains state-of-the-art results on both WoZ 2.0 and MultiWoZ 2.0 datasets but also has good generalization on unseen values and new slots.
arXiv Detail & Related papers (2020-08-08T02:15:25Z) - UniConv: A Unified Conversational Neural Architecture for Multi-domain
Task-oriented Dialogues [101.96097419995556]
"UniConv" is a novel unified neural architecture for end-to-end conversational systems in task-oriented dialogues.
We conduct comprehensive experiments in dialogue state tracking, context-to-text, and end-to-end settings on the MultiWOZ2.1 benchmark.
arXiv Detail & Related papers (2020-04-29T16:28:22Z) - Recent Advances and Challenges in Task-oriented Dialog System [63.82055978899631]
Task-oriented dialog systems are attracting more and more attention in academic and industrial communities.
We discuss three critical topics for task-oriented dialog systems: (1) improving data efficiency to facilitate dialog modeling in low-resource settings, (2) modeling multi-turn dynamics for dialog policy learning, and (3) integrating domain knowledge into the dialog model.
arXiv Detail & Related papers (2020-03-17T01:34:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.