Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems

• URL: http://arxiv.org/abs/2204.05158v1
• Date: Mon, 11 Apr 2022 14:45:55 GMT
• Title: Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems
• Authors: Ella Rabinovich, Matan Vetzler, David Boaz, Vineet Kumar, Gaurav Pandey, Ateret Anaby-Tavor
• Abstract summary: We present an end-to-end pipeline for processing unrecognized user utterances, including a specifically-tailored clustering algorithm. We evaluate the proposed clustering algorithm and compared its performance to out-of-the-box SOTA solutions.
• Score: 6.09284941695878
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapidly growing market demand for dialogue agents capable of goal-oriented behavior has caused many tech-industry leaders to invest considerable efforts into task-oriented dialog systems. The performance and success of these systems is highly dependent on the accuracy of their intent identification -- the process of deducing the goal or meaning of the user's request and mapping it to one of the known intents for further processing. Gaining insights into unrecognized utterances -- user requests the systems fails to attribute to a known intent -- is therefore a key process in continuous improvement of goal-oriented dialog systems. We present an end-to-end pipeline for processing unrecognized user utterances, including a specifically-tailored clustering algorithm, a novel approach to cluster representative extraction, and cluster naming. We evaluated the proposed clustering algorithm and compared its performance to out-of-the-box SOTA solutions, demonstrating its benefits in the analysis of unrecognized user requests.

Related papers

• Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents [110.25679611755962]
  Current language model-driven agents often lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions. We introduce Intention-in-Interaction (IN3), a novel benchmark designed to inspect users' implicit intentions through explicit queries. We empirically train Mistral-Interact, a powerful model that proactively assesses task vagueness, inquires user intentions, and refines them into actionable goals.
  arXiv  Detail & Related papers  (2024-02-14T14:36:30Z)
• IntentDial: An Intent Graph based Multi-Turn Dialogue System with Reasoning Path Visualization [24.888848712778664]
  We present a novel graph-based multi-turn dialogue system called. It identifies a user's intent by identifying intent elements and a standard query from a graph using reinforcement learning.
  arXiv  Detail & Related papers  (2023-10-18T09:21:37Z)
• PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems [59.1250765143521]
  Current knowledge-grounded dialogue systems often fail to align the generated responses with human-preferred qualities. We propose Polished & Informed Candidate Scoring (PICK), a generation re-scoring framework. We demonstrate the effectiveness of PICK in generating responses that are more faithful while keeping them relevant to the dialogue history.
  arXiv  Detail & Related papers  (2023-09-19T08:27:09Z)
• Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals [76.69419538047813]
  ProToD approach anticipates the future dialogue actions and incorporates the goal-oriented reward signal to enhance ToD systems. We present a novel evaluation method that assesses ToD systems based on goal-driven dialogue simulations. Empirical experiments conducted on the MultiWoZ 2.1 dataset demonstrate that our model can achieve superior performance using only 10% of the data.
  arXiv  Detail & Related papers  (2023-09-16T10:56:00Z)
• Using Textual Interface to Align External Knowledge for End-to-End Task-Oriented Dialogue Systems [53.38517204698343]
  We propose a novel paradigm that uses a textual interface to align external knowledge and eliminate redundant processes. We demonstrate our paradigm in practice through MultiWOZ-Remake, including an interactive textual interface built for the MultiWOZ database.
  arXiv  Detail & Related papers  (2023-05-23T05:48:21Z)
• Investigating the Combination of Planning-Based and Data-Driven Methods for Goal Recognition [7.620967781722714]
  We investigate the application of two state-of-the-art, planning-based plan recognition approaches in a real-world setting. We show that such approaches have difficulties when used to recognize the goals of human subjects, because human behaviour is typically not perfectly rational. We propose an extension to the existing approaches through a classification-based method trained on observed behaviour data.
  arXiv  Detail & Related papers  (2023-01-13T15:24:02Z)
• Discovering Customer-Service Dialog System with Semi-Supervised Learning and Coarse-to-Fine Intent Detection [6.869753194843482]
  Task-oriented dialog aims to assist users in achieving specific goals through multi-turn conversation. We constructed a weakly supervised dataset based on a teacher/student paradigm. We also built a modular dialogue system and integrated coarse-to-fine grained classification for user intent detection.
  arXiv  Detail & Related papers  (2022-12-23T14:36:43Z)
• Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentation [32.764356638437214]
  We introduce a new technique for target-guided response generation. We also propose techniques to re-purpose existing dialogue datasets for target-guided generation. Our work generally enables dialogue system designers to exercise more control over the conversations that their systems produce.
  arXiv  Detail & Related papers  (2022-05-19T04:01:40Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• Optimizing Interactive Systems via Data-Driven Objectives [70.3578528542663]
  We propose an approach that infers the objective directly from observed user interactions. These inferences can be made regardless of prior knowledge and across different types of user behavior. We introduce Interactive System (ISO), a novel algorithm that uses these inferred objectives for optimization.
  arXiv  Detail & Related papers  (2020-06-19T20:49:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.