Generative Induction of Dialogue Task Schemas with Streaming Refinement and Simulated Interactions

• URL: http://arxiv.org/abs/2504.18474v1
• Date: Fri, 25 Apr 2025 16:29:45 GMT
• Title: Generative Induction of Dialogue Task Schemas with Streaming Refinement and Simulated Interactions
• Authors: James D. Finch, Yasasvi Josyula, Jinho D. Choi,
• Abstract summary: Slot Induction (SSI) is essential for automatically identifying key information slots from dialogue data without manual intervention.<n>This paper presents a novel state-of-the-art (SoTA) approach that formulates SSI as a text generation task.
• Score: 10.781063445675423
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In task-oriented dialogue (TOD) systems, Slot Schema Induction (SSI) is essential for automatically identifying key information slots from dialogue data without manual intervention. This paper presents a novel state-of-the-art (SoTA) approach that formulates SSI as a text generation task, where a language model incrementally constructs and refines a slot schema over a stream of dialogue data. To develop this approach, we present a fully automatic LLM-based TOD simulation method that creates data with high-quality state labels for novel task domains. Furthermore, we identify issues in SSI evaluation due to data leakage and poor metric alignment with human judgment. We resolve these by creating new evaluation data using our simulation method with human guidance and correction, as well as designing improved evaluation metrics. These contributions establish a foundation for future SSI research and advance the SoTA in dialogue understanding and system development.

Related papers

• Transforming Slot Schema Induction with Generative Dialogue State Inference [14.06505399101404]
  Slot Induction (SSI) aims to automatically induce slots from unlabeled dialogue data. Our SSI method discovers high-quality candidate information for representing dialogue state. Experimental comparisons on the MultiWOZ and SGD datasets demonstrate that Generative Dialogue State Inference (GenDSI) outperforms the previous state-of-the-art.
  arXiv  Detail & Related papers  (2024-08-03T02:41:10Z)
• Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcripts [10.829227084902428]
  We investigate the feasibility and effectiveness of Large Language Models (LLMs)-based data generation in source-grounded information-seeking dialogs. We create MISeD -- Meeting Information Seeking Dialogs dataset -- a dataset of information-seeking dialogs focused on meeting transcripts. Finetuning on MISeD gives comparable response generation quality to finetuning on fully manual data, while improving attribution quality and reducing time and effort.
  arXiv  Detail & Related papers  (2024-05-02T09:35:06Z)
• Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Models [16.94819621353007]
  SynTOD is a new synthetic data generation approach for developing end-to-end Task-Oriented Dialogue (TOD) systems. It generates diverse, structured conversations through random walks and response simulation using large language models. In our experiments, using graph-guided response simulations leads to significant improvements in intent classification, slot filling and response relevance.
  arXiv  Detail & Related papers  (2024-04-23T06:23:34Z)
• Injecting linguistic knowledge into BERT for Dialogue State Tracking [60.42231674887294]
  This paper proposes a method that extracts linguistic knowledge via an unsupervised framework. We then utilize this knowledge to augment BERT's performance and interpretability in Dialogue State Tracking (DST) tasks. We benchmark this framework on various DST tasks and observe a notable improvement in accuracy.
  arXiv  Detail & Related papers  (2023-11-27T08:38:42Z)
• InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems [60.53276524369498]
  Large language models (LLMs) have been used for diverse tasks in natural language processing (NLP) We present InstructTODS, a novel framework for zero-shot end-to-end task-oriented dialogue systems. InstructTODS generates a proxy belief state that seamlessly translates user intentions into dynamic queries.
  arXiv  Detail & Related papers  (2023-10-13T06:36:26Z)
• Using Textual Interface to Align External Knowledge for End-to-End Task-Oriented Dialogue Systems [53.38517204698343]
  We propose a novel paradigm that uses a textual interface to align external knowledge and eliminate redundant processes. We demonstrate our paradigm in practice through MultiWOZ-Remake, including an interactive textual interface built for the MultiWOZ database.
  arXiv  Detail & Related papers  (2023-05-23T05:48:21Z)
• Turning Flowchart into Dialog: Augmenting Flowchart-grounded Troubleshooting Dialogs via Synthetic Data Generation [50.06143883455979]
  Flowchart-grounded troubleshooting dialogue (FTD) systems follow the instructions of a flowchart to diagnose users' problems in specific domains. We propose a plan-based synthetic data generation approach that generates diverse synthetic dialog data at scale.
  arXiv  Detail & Related papers  (2023-05-02T11:08:27Z)
• GODEL: Large-Scale Pre-Training for Goal-Directed Dialog [119.1397031992088]
  We introduce GODEL, a large pre-trained language model for dialog. We show that GODEL outperforms state-of-the-art pre-trained dialog models in few-shot fine-tuning setups. A novel feature of our evaluation methodology is the introduction of a notion of utility that assesses the usefulness of responses.
  arXiv  Detail & Related papers  (2022-06-22T18:19:32Z)
• Quick Starting Dialog Systems with Paraphrase Generation [0.0]
  We propose a method to reduce the cost and effort of creating new conversational agents by artificially generating more data from existing examples. Our proposed approach can kick-start a dialog system with little human effort, and brings its performance to a level satisfactory enough for allowing actual interactions with real end-users.
  arXiv  Detail & Related papers  (2022-04-06T02:35:59Z)
• A Simple But Effective Approach to n-shot Task-Oriented Dialogue Augmentation [32.43362825854633]
  We introduce a framework that creates synthetic task-oriented dialogues in a fully automatic manner. Our framework uses the simple idea that each turn-pair in a task-oriented dialogue has a certain function. We observe significant improvements in the fine-tuning scenarios in several domains.
  arXiv  Detail & Related papers  (2021-02-27T18:55:12Z)
• Modeling Long Context for Task-Oriented Dialogue State Generation [51.044300192906995]
  We propose a multi-task learning model with a simple yet effective utterance tagging technique and a bidirectional language model. Our approaches attempt to solve the problem that the performance of the baseline significantly drops when the input dialogue context sequence is long. In our experiments, our proposed model achieves a 7.03% relative improvement over the baseline, establishing a new state-of-the-art joint goal accuracy of 52.04% on the MultiWOZ 2.0 dataset.
  arXiv  Detail & Related papers  (2020-04-29T11:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.