LUCID: LLM-Generated Utterances for Complex and Interesting Dialogues

• URL: http://arxiv.org/abs/2403.00462v2
• Date: Fri, 3 May 2024 16:57:07 GMT
• Title: LUCID: LLM-Generated Utterances for Complex and Interesting Dialogues
• Authors: Joe Stacey, Jianpeng Cheng, John Torr, Tristan Guigue, Joris Driesen, Alexandru Coca, Mark Gaynor, Anders Johannsen,
• Abstract summary: Virtual assistants are poised to take a leap forward in terms of their dialogue capabilities. Yet a major bottleneck to achieving genuinely transformative task-oriented dialogue capabilities remains the scarcity of high quality data. We use LUCID to generate a seed dataset of 4,277 conversations across 100 intents to demonstrate its capabilities.
• Score: 38.6183579217801
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spurred by recent advances in Large Language Models (LLMs), virtual assistants are poised to take a leap forward in terms of their dialogue capabilities. Yet a major bottleneck to achieving genuinely transformative task-oriented dialogue capabilities remains the scarcity of high quality data. Existing datasets, while impressive in scale, have limited domain coverage and contain few genuinely challenging conversational phenomena; those which are present are typically unlabelled, making it difficult to assess the strengths and weaknesses of models without time-consuming and costly human evaluation. Moreover, creating high quality dialogue data has until now required considerable human input, limiting both the scale of these datasets and the ability to rapidly bootstrap data for a new target domain. We aim to overcome these issues with LUCID, a modularised and highly automated LLM-driven data generation system that produces realistic, diverse and challenging dialogues. We use LUCID to generate a seed dataset of 4,277 conversations across 100 intents to demonstrate its capabilities, with a human review finding consistently high quality labels in the generated data.

Related papers

• CoDi: Conversational Distillation for Grounded Question Answering [10.265241619616676]
  We introduce a novel data distillation framework named CoDi. CoDi allows us to synthesize large-scale, assistant-style datasets in a steerable and diverse manner. We show that SLMs trained with CoDi-synthesized data achieve performance comparable to models trained on human-annotated data in standard metrics.
  arXiv  Detail & Related papers  (2024-08-20T22:35:47Z)
• Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcripts [10.829227084902428]
  We investigate the feasibility and effectiveness of Large Language Models (LLMs)-based data generation in source-grounded information-seeking dialogs. We create MISeD -- Meeting Information Seeking Dialogs dataset -- a dataset of information-seeking dialogs focused on meeting transcripts. Finetuning on MISeD gives comparable response generation quality to finetuning on fully manual data, while improving attribution quality and reducing time and effort.
  arXiv  Detail & Related papers  (2024-05-02T09:35:06Z)
• MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning [63.80739044622555]
  We introduce MuSR, a dataset for evaluating language models on soft reasoning tasks specified in a natural language narrative. This dataset has two crucial features. First, it is created through a novel neurosymbolic synthetic-to-natural generation algorithm. Second, our dataset instances are free text narratives corresponding to real-world domains of reasoning.
  arXiv  Detail & Related papers  (2023-10-24T17:59:20Z)
• Does Collaborative Human-LM Dialogue Generation Help Information Extraction from Human Dialogues? [55.28340832822234]
  Problem-solving human dialogues in real applications can be much more complex than existing Wizard-of-Oz collections. We introduce a human-in-the-loop dialogue generation framework capable of synthesizing realistic dialogues.
  arXiv  Detail & Related papers  (2023-07-13T20:02:50Z)
• STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models [56.27786433792638]
  STAR is a data generation method that leverages Large Language Models (LLMs) to synthesize data instances. We design fine-grained step-by-step instructions to obtain the initial data instances. Our experiments show that the data generated by STAR significantly improve the performance of low-resource event extraction and relation extraction tasks.
  arXiv  Detail & Related papers  (2023-05-24T12:15:19Z)
• Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization [3.6393183544320236]
  Speech recognition has become an important challenge when using deep learning (DL) It requires large-scale training datasets and high computational and storage resources. Deep transfer learning (DTL) has been introduced to overcome these issues.
  arXiv  Detail & Related papers  (2023-04-27T21:08:05Z)
• PRESTO: A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs [39.58414649004708]
  PRESTO is a dataset of over 550K contextual multilingual conversations between humans and virtual assistants. It contains challenges that occur in real-world NLU tasks such as disfluencies, code-switching, and revisions. Our mT5 model based baselines demonstrate that the conversational phenomenon present in PRESTO are challenging to model.
  arXiv  Detail & Related papers  (2023-03-15T21:51:13Z)
• Weakly Supervised Data Augmentation Through Prompting for Dialogue Understanding [103.94325597273316]
  We present a novel approach that iterates on augmentation quality by applying weakly-supervised filters. We evaluate our methods on the emotion and act classification tasks in DailyDialog and the intent classification task in Facebook Multilingual Task-Oriented Dialogue. For DailyDialog specifically, using 10% of the ground truth data we outperform the current state-of-the-art model which uses 100% of the data.
  arXiv  Detail & Related papers  (2022-10-25T17:01:30Z)
• Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation [70.81596088969378]
  Cross-lingual Outline-based Dialogue dataset (termed COD) enables natural language understanding. COD enables dialogue state tracking, and end-to-end dialogue modelling and evaluation in 4 diverse languages.
  arXiv  Detail & Related papers  (2022-01-31T18:11:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.