Controllable Dialogue Simulation with In-Context Learning
- URL: http://arxiv.org/abs/2210.04185v4
- Date: Tue, 6 Jun 2023 02:19:08 GMT
- Title: Controllable Dialogue Simulation with In-Context Learning
- Authors: Zekun Li, Wenhu Chen, Shiyang Li, Hong Wang, Jing Qian, Xifeng Yan
- Abstract summary: textscDialogic is a dialogue simulation method based on large language model in-context learning.
Our method can rapidly expand a small set of dialogue data with minimum or zero human involvement.
Our simulated dialogues have near-human fluency and annotation accuracy.
- Score: 39.04491297557292
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Building dialogue systems requires a large corpus of annotated dialogues.
Such datasets are usually created via crowdsourcing, which is expensive and
time-consuming. In this paper, we propose \textsc{Dialogic}, a novel dialogue
simulation method based on large language model in-context learning to automate
dataset creation. Seeded with a few annotated dialogues, \textsc{Dialogic}
automatically selects in-context examples for demonstration and prompts GPT-3
to generate new dialogues and annotations in a controllable way. Our method can
rapidly expand a small set of dialogue data with minimum or zero \textit{human
involvement} and \textit{parameter update} and is thus much more cost-efficient
and time-saving than crowdsourcing. Experimental results on the MultiWOZ
dataset demonstrate that training a model on the simulated dialogues leads to
even better performance than using the same amount of human-generated dialogues
under the challenging low-resource settings, with as few as 85 dialogues as a
seed. When enough data is available, our method can still serve as an effective
data augmentation method. Human evaluation results also show that our simulated
dialogues have near-human fluency and annotation accuracy. The code and data
are available at \textbf{\url{https://github.com/Leezekun/dialogic}}.
Related papers
- Stabilized In-Context Learning with Pre-trained Language Models for Few
Shot Dialogue State Tracking [57.92608483099916]
Large pre-trained language models (PLMs) have shown impressive unaided performance across many NLP tasks.
For more complex tasks such as dialogue state tracking (DST), designing prompts that reliably convey the desired intent is nontrivial.
We introduce a saliency model to limit dialogue text length, allowing us to include more exemplars per query.
arXiv Detail & Related papers (2023-02-12T15:05:10Z) - Prompting for a conversation: How to control a dialog model? [9.268682116424518]
Dialog models are trained on a large amount of text, yet their responses need to be limited to a desired scope and style of a dialog agent.
Because the datasets used to achieve the former contain language that is not compatible with the latter, pre-trained dialog models are fine-tuned on smaller curated datasets.
In this paper we investigate if prompting can mitigate the above trade-off.
arXiv Detail & Related papers (2022-09-22T14:59:55Z) - SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for
Task-Oriented Dialog Understanding [68.94808536012371]
We propose a tree-structured pre-trained conversation model, which learns dialog representations from limited labeled dialogs and large-scale unlabeled dialog corpora.
Our method can achieve new state-of-the-art results on the DialoGLUE benchmark consisting of seven datasets and four popular dialog understanding tasks.
arXiv Detail & Related papers (2022-09-14T13:42:50Z) - Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue
Comprehension [48.483910831143724]
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances.
We develop a novel narrative-guided pre-training strategy that learns by narrating the key information from a dialogue input.
arXiv Detail & Related papers (2022-03-19T05:20:25Z) - In-Context Learning for Few-Shot Dialogue State Tracking [55.91832381893181]
We propose an in-context (IC) learning framework for few-shot dialogue state tracking (DST)
A large pre-trained language model (LM) takes a test instance and a few annotated examples as input, and directly decodes the dialogue states without any parameter updates.
This makes the LM more flexible and scalable compared to prior few-shot DST work when adapting to new domains and scenarios.
arXiv Detail & Related papers (2022-03-16T11:58:24Z) - Structure Extraction in Task-Oriented Dialogues with Slot Clustering [94.27806592467537]
In task-oriented dialogues, dialogue structure has often been considered as transition graphs among dialogue states.
We propose a simple yet effective approach for structure extraction in task-oriented dialogues.
arXiv Detail & Related papers (2022-02-28T20:18:12Z) - DG2: Data Augmentation Through Document Grounded Dialogue Generation [41.81030088619399]
We propose an automatic data augmentation technique grounded on documents through a generative dialogue model.
When supplementing the original dataset, our method achieves significant improvement over traditional data augmentation methods.
arXiv Detail & Related papers (2021-12-15T18:50:14Z) - Response Generation with Context-Aware Prompt Learning [19.340498579331555]
We present a novel approach for pre-trained dialogue modeling that casts the dialogue generation problem as a prompt-learning task.
Instead of fine-tuning on limited dialogue data, our approach, DialogPrompt, learns continuous prompt embeddings optimized for dialogue contexts.
Our approach significantly outperforms the fine-tuning baseline and the generic prompt-learning methods.
arXiv Detail & Related papers (2021-11-04T05:40:13Z) - Dialog Simulation with Realistic Variations for Training Goal-Oriented
Conversational Systems [14.206866126142002]
Goal-oriented dialog systems enable users to complete specific goals like requesting information about a movie or booking a ticket.
We propose an approach for automatically creating a large corpus of annotated dialogs from a few thoroughly annotated sample dialogs and the dialog schema.
We achieve 18? 50% relative accuracy on a held-out test set compared to a baseline dialog generation approach.
arXiv Detail & Related papers (2020-11-16T19:39:15Z) - A Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise
ThingTalk Representation [5.56536459714557]
Previous attempts to build effective semantics for Wizard-of-Oz (WOZ) conversations suffer from the difficulty in acquiring a high-quality, manually annotated training set.
This paper proposes a new dialogue representation and a sample-efficient methodology that can predict precise dialogue states in WOZ conversations.
arXiv Detail & Related papers (2020-09-16T22:52:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.