Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue

• URL: http://arxiv.org/abs/2204.04327v1
• Date: Fri, 8 Apr 2022 23:27:18 GMT
• Title: Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue
• Authors: Raghav Gupta, Harrison Lee, Jeffrey Zhao, Abhinav Rastogi, Yuan Cao, Yonghui Wu
• Abstract summary: Show, Don't Tell is a prompt format for seq2seq modeling which uses a short labeled example dialogue to show the semantics of schema elements. While requiring similar effort from service developers, we show that using short examples as schema representations with large language models results in stronger performance and better generalization.
• Score: 27.43338545216015
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Building universal dialogue systems that can seamlessly operate across multiple domains/APIs and generalize to new ones with minimal supervision and maintenance is a critical challenge. Recent works have leveraged natural language descriptions for schema elements to enable such systems; however, descriptions can only indirectly convey schema semantics. In this work, we propose Show, Don't Tell, a prompt format for seq2seq modeling which uses a short labeled example dialogue to show the semantics of schema elements rather than tell the model via descriptions. While requiring similar effort from service developers, we show that using short examples as schema representations with large language models results in stronger performance and better generalization on two popular dialogue state tracking benchmarks: the Schema-Guided Dialogue dataset and the MultiWoZ leave-one-out benchmark.

Related papers

• Drafting Event Schemas using Language Models [48.81285141287434]
  We look at the process of creating such schemas to describe complex events. Our focus is on whether we can achieve sufficient diversity and recall of key events. We show that large language models are able to achieve moderate recall against schemas taken from two different datasets.
  arXiv  Detail & Related papers  (2023-05-24T07:57:04Z)
• Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking [57.92608483099916]
  Large pre-trained language models (PLMs) have shown impressive unaided performance across many NLP tasks. For more complex tasks such as dialogue state tracking (DST), designing prompts that reliably convey the desired intent is nontrivial. We introduce a saliency model to limit dialogue text length, allowing us to include more exemplars per query.
  arXiv  Detail & Related papers  (2023-02-12T15:05:10Z)
• Manual-Guided Dialogue for Flexible Conversational Agents [84.46598430403886]
  How to build and use dialogue data efficiently, and how to deploy models in different domains at scale can be critical issues in building a task-oriented dialogue system. We propose a novel manual-guided dialogue scheme, where the agent learns the tasks from both dialogue and manuals. Our proposed scheme reduces the dependence of dialogue models on fine-grained domain ontology, and makes them more flexible to adapt to various domains.
  arXiv  Detail & Related papers  (2022-08-16T08:21:12Z)
• Unsupervised Slot Schema Induction for Task-oriented Dialog [12.585986197627477]
  We propose an unsupervised approach for slot schema induction from unlabeled dialog corpora. We show significant performance improvement in slot schema induction on MultiWoz and SGD datasets. We also demonstrate the effectiveness of induced schemas on downstream applications including dialog state tracking and response generation.
  arXiv  Detail & Related papers  (2022-05-09T18:36:25Z)
• Dialogue Meaning Representation for Task-Oriented Dialogue Systems [51.91615150842267]
  We propose Dialogue Meaning Representation (DMR), a flexible and easily extendable representation for task-oriented dialogue. Our representation contains a set of nodes and edges with inheritance hierarchy to represent rich semantics for compositional semantics and task-specific concepts. We propose two evaluation tasks to evaluate different machine learning based dialogue models, and further propose a novel coreference resolution model GNNCoref for the graph-based coreference resolution task.
  arXiv  Detail & Related papers  (2022-04-23T04:17:55Z)
• Mining Logical Event Schemas From Pre-Trained Language Models [0.0]
  We present NESL (the Neuro-Episodic Learner), an event schema learning system that combines large language models, FrameNet parsing, and a set of simple behavioral schemas. We show that careful sampling from the language model can help emphasize stereotypical properties of situations and de-emphasize irrelevant details.
  arXiv  Detail & Related papers  (2022-04-12T16:41:18Z)
• In-Context Learning for Few-Shot Dialogue State Tracking [55.91832381893181]
  We propose an in-context (IC) learning framework for few-shot dialogue state tracking (DST) A large pre-trained language model (LM) takes a test instance and a few annotated examples as input, and directly decodes the dialogue states without any parameter updates. This makes the LM more flexible and scalable compared to prior few-shot DST work when adapting to new domains and scenarios.
  arXiv  Detail & Related papers  (2022-03-16T11:58:24Z)
• Structure Extraction in Task-Oriented Dialogues with Slot Clustering [94.27806592467537]
  In task-oriented dialogues, dialogue structure has often been considered as transition graphs among dialogue states. We propose a simple yet effective approach for structure extraction in task-oriented dialogues.
  arXiv  Detail & Related papers  (2022-02-28T20:18:12Z)
• SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems [26.14268488547028]
  We release SGD-X, a benchmark for measuring robustness of dialogue systems to linguistic variations in schemas. We evaluate two dialogue state tracking models on SGD-X and observe that neither generalizes well across schema variations. We present a simple model-agnostic data augmentation method to improve schema robustness and zero-shot generalization to unseen services.
  arXiv  Detail & Related papers  (2021-10-13T15:38:29Z)
• Dialogue State Tracking with a Language Model using Schema-Driven Prompting [18.83983018421701]
  We introduce a new variation of the language modeling approach that uses schema-driven prompting to provide task-aware history encoding. Our purely generative system achieves state-of-the-art performance on MultiWOZ 2.2 and achieves competitive performance on two other benchmarks: MultiWOZ 2.1 and M2M.
  arXiv  Detail & Related papers  (2021-09-15T18:11:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.