S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs

• URL: http://arxiv.org/abs/2309.08827v1
• Date: Sat, 16 Sep 2023 00:59:23 GMT
• Title: S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs
• Authors: Sarkar Snigdha Sarathi Das, Chirag Shah, Mengting Wan, Jennifer Neville, Longqi Yang, Reid Andersen, Georg Buscher, Tara Safavi
• Abstract summary: The advent of Large Language Model (LLM)-based chat systems has introduced many real-world intricacies in open-domain dialogues. We propose joint dialogue segmentation and state tracking per segment in open-domain dialogue systems. We evaluate S3-DST on a proprietary anonymized open-domain dialogue dataset, as well as publicly available DST and segmentation datasets.
• Score: 22.319211779438934
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The traditional Dialogue State Tracking (DST) problem aims to track user preferences and intents in user-agent conversations. While sufficient for task-oriented dialogue systems supporting narrow domain applications, the advent of Large Language Model (LLM)-based chat systems has introduced many real-world intricacies in open-domain dialogues. These intricacies manifest in the form of increased complexity in contextual interactions, extended dialogue sessions encompassing a diverse array of topics, and more frequent contextual shifts. To handle these intricacies arising from evolving LLM-based chat systems, we propose joint dialogue segmentation and state tracking per segment in open-domain dialogue systems. Assuming a zero-shot setting appropriate to a true open-domain dialogue system, we propose S3-DST, a structured prompting technique that harnesses Pre-Analytical Recollection, a novel grounding mechanism we designed for improving long context tracking. To demonstrate the efficacy of our proposed approach in joint segmentation and state tracking, we evaluate S3-DST on a proprietary anonymized open-domain dialogue dataset, as well as publicly available DST and segmentation datasets. Across all datasets and settings, S3-DST consistently outperforms the state-of-the-art, demonstrating its potency and robustness the next generation of LLM-based chat systems.

Related papers

• Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbot [3.2288892242158984]
  We propose a novel approach to enhance Dialogue State Tracking (DST) performance. Our method enables Large Language Model (LLM) to infer dialogue states through carefully designed prompts. Our approach achieved state-of-the-art with a JGA of 42.57%, and performed well in open-domain real-world conversations.
  arXiv  Detail & Related papers  (2024-10-30T07:36:23Z)
• Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs [59.74002011562726]
  We propose a novel linguistic cue-based chain-of-thoughts (textitCue-CoT) to provide a more personalized and engaging response. We build a benchmark with in-depth dialogue questions, consisting of 6 datasets in both Chinese and English. Empirical results demonstrate our proposed textitCue-CoT method outperforms standard prompting methods in terms of both textithelpfulness and textitacceptability on all datasets.
  arXiv  Detail & Related papers  (2023-05-19T16:27:43Z)
• Act-Aware Slot-Value Predicting in Multi-Domain Dialogue State Tracking [5.816391291790977]
  Dialogue state tracking (DST) aims to track human-machine interactions and generate state representations for managing the dialogue. Recent advances in machine reading comprehension predict both categorical and non-categorical types of slots for dialogue state tracking. We formulate and incorporate dialogue acts, and leverage recent advances in machine reading comprehension to predict both categorical and non-categorical types of slots for dialogue state tracking.
  arXiv  Detail & Related papers  (2022-08-04T05:18:30Z)
• Multimodal Dialogue State Tracking [97.25466640240619]
  Video-Dialogue Transformer Network (VDTN) learns contextual dependencies between videos and dialogues to generate multimodal dialogue states. VDTN combines both object-level features and segment-level features and learns contextual dependencies between videos and dialogues to generate multimodal dialogue states.
  arXiv  Detail & Related papers  (2022-06-16T03:18:42Z)
• In-Context Learning for Few-Shot Dialogue State Tracking [55.91832381893181]
  We propose an in-context (IC) learning framework for few-shot dialogue state tracking (DST) A large pre-trained language model (LM) takes a test instance and a few annotated examples as input, and directly decodes the dialogue states without any parameter updates. This makes the LM more flexible and scalable compared to prior few-shot DST work when adapting to new domains and scenarios.
  arXiv  Detail & Related papers  (2022-03-16T11:58:24Z)
• Structure Extraction in Task-Oriented Dialogues with Slot Clustering [94.27806592467537]
  In task-oriented dialogues, dialogue structure has often been considered as transition graphs among dialogue states. We propose a simple yet effective approach for structure extraction in task-oriented dialogues.
  arXiv  Detail & Related papers  (2022-02-28T20:18:12Z)
• The SPPD System for Schema Guided Dialogue State Tracking Challenge [2.181419218084711]
  This paper introduces one of our group's work on the Dialog System Technology Challenges 8 (DSTC8) The challenge, named as Track 4 in DSTC8, provides a brand new and challenging dataset for developing scalable multi-domain dialogue state tracking algorithms. We propose a zero-shot dialogue state tracking system for this task.
  arXiv  Detail & Related papers  (2020-06-16T09:57:40Z)
• UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues [101.96097419995556]
  "UniConv" is a novel unified neural architecture for end-to-end conversational systems in task-oriented dialogues. We conduct comprehensive experiments in dialogue state tracking, context-to-text, and end-to-end settings on the MultiWOZ2.1 benchmark.
  arXiv  Detail & Related papers  (2020-04-29T16:28:22Z)
• Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking [32.36259992245]
  Dialogue state tracking (DST) aims at estimating the current dialogue state given all the preceding conversation. For multi-domain DST, the data sparsity problem is a major obstacle due to increased numbers of state candidates and dialogue lengths. We utilize the previous dialogue state (predicted) and the current dialogue utterance as the input for DST.
  arXiv  Detail & Related papers  (2020-04-07T13:46:39Z)
• Non-Autoregressive Dialog State Tracking [122.2328875457225]
  We propose a novel framework of Non-Autoregressive Dialog State Tracking (NADST) NADST can factor in potential dependencies among domains and slots to optimize the models towards better prediction of dialogue states as a complete set rather than separate slots. Our results show that our model achieves the state-of-the-art joint accuracy across all domains on the MultiWOZ 2.1 corpus.
  arXiv  Detail & Related papers  (2020-02-19T06:39:26Z)
• Domain-Aware Dialogue State Tracker for Multi-Domain Dialogue Systems [2.3859169601259347]
  In task-oriented dialogue systems the dialogue state tracker (DST) component is responsible for predicting the state of the dialogue based on the dialogue history. We propose a domain-aware dialogue state tracker that is completely data-driven and it is modeled to predict for dynamic service schemas.
  arXiv  Detail & Related papers  (2020-01-21T13:41:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.