User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems

• URL: http://arxiv.org/abs/2202.02912v1
• Date: Mon, 7 Feb 2022 02:50:07 GMT
• Title: User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems
• Authors: Yang Deng, Wenxuan Zhang, Wai Lam, Hong Cheng, Helen Meng
• Abstract summary: We propose a novel framework, namely USDA, to incorporate the sequential dynamics of dialogue acts for predicting user satisfaction. USDA incorporates the sequential transitions of both content and act features in the dialogue to predict the user satisfaction. Experimental results on four benchmark goal-oriented dialogue datasets show that the proposed method substantially and consistently outperforms existing methods on USE.
• Score: 65.88679683468143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: User Satisfaction Estimation (USE) is an important yet challenging task in goal-oriented conversational systems. Whether the user is satisfied with the system largely depends on the fulfillment of the user's needs, which can be implicitly reflected by users' dialogue acts. However, existing studies often neglect the sequential transitions of dialogue act or rely heavily on annotated dialogue act labels when utilizing dialogue acts to facilitate USE. In this paper, we propose a novel framework, namely USDA, to incorporate the sequential dynamics of dialogue acts for predicting user satisfaction, by jointly learning User Satisfaction Estimation and Dialogue Act Recognition tasks. In specific, we first employ a Hierarchical Transformer to encode the whole dialogue context, with two task-adaptive pre-training strategies to be a second-phase in-domain pre-training for enhancing the dialogue modeling ability. In terms of the availability of dialogue act labels, we further develop two variants of USDA to capture the dialogue act information in either supervised or unsupervised manners. Finally, USDA leverages the sequential transitions of both content and act features in the dialogue to predict the user satisfaction. Experimental results on four benchmark goal-oriented dialogue datasets across different applications show that the proposed method substantially and consistently outperforms existing methods on USE, and validate the important role of dialogue act sequences in USE.

Related papers

• 'No' Matters: Out-of-Distribution Detection in Multimodality Long Dialogue [3.971267935825097]
  This paper aims to improve the user experience that involves multi-round long dialogues by efficiently detecting OOD dialogues and images. We introduce a novel scoring framework named Dialogue Image Aligning and Enhancing Framework (DIAEF) that integrates the visual language models with the novel proposed scores.
  arXiv  Detail & Related papers  (2024-10-31T12:45:54Z)
• Multi-User MultiWOZ: Task-Oriented Dialogues among Multiple Users [51.34484827552774]
  We release the Multi-User MultiWOZ dataset: task-oriented dialogues among two users and one agent. These dialogues reflect interesting dynamics of collaborative decision-making in task-oriented scenarios. We propose a novel task of multi-user contextual query rewriting: to rewrite a task-oriented chat between two users as a concise task-oriented query.
  arXiv  Detail & Related papers  (2023-10-31T14:12:07Z)
• Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals [76.69419538047813]
  ProToD approach anticipates the future dialogue actions and incorporates the goal-oriented reward signal to enhance ToD systems. We present a novel evaluation method that assesses ToD systems based on goal-driven dialogue simulations. Empirical experiments conducted on the MultiWoZ 2.1 dataset demonstrate that our model can achieve superior performance using only 10% of the data.
  arXiv  Detail & Related papers  (2023-09-16T10:56:00Z)
• Dialog-to-Actions: Building Task-Oriented Dialogue System via Action-Level Generation [7.110201160927713]
  We propose a task-oriented dialogue system via action-level generation. Specifically, we first construct dialogue actions from large-scale dialogues and represent each natural language (NL) response as a sequence of dialogue actions. We train a Sequence-to-Sequence model which takes the dialogue history as input and outputs sequence of dialogue actions.
  arXiv  Detail & Related papers  (2023-04-03T11:09:20Z)
• Dialog Acts for Task-Driven Embodied Agents [10.275619475149433]
  Embodied agents need to be able to interact in natural language understanding task descriptions and asking appropriate follow up questions. We propose a set of dialog acts for modelling such dialogs and annotate the TEACh dataset that includes over 3,000 situated, task oriented conversations. We demonstrate the use of this annotated dataset in training models for tagging the dialog acts of a given utterance, predicting the dialog act of the next response given a dialog history, and use the dialog acts to guide agent's non-dialog behaviour.
  arXiv  Detail & Related papers  (2022-09-26T18:41:28Z)
• Act-Aware Slot-Value Predicting in Multi-Domain Dialogue State Tracking [5.816391291790977]
  Dialogue state tracking (DST) aims to track human-machine interactions and generate state representations for managing the dialogue. Recent advances in machine reading comprehension predict both categorical and non-categorical types of slots for dialogue state tracking. We formulate and incorporate dialogue acts, and leverage recent advances in machine reading comprehension to predict both categorical and non-categorical types of slots for dialogue state tracking.
  arXiv  Detail & Related papers  (2022-08-04T05:18:30Z)
• UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues [59.499965460525694]
  We propose a unified dialogue system (UniDS) with the two aforementioned skills. We design a unified dialogue data schema, compatible for both chit-chat and task-oriented dialogues. We train UniDS with mixed dialogue data from a pretrained chit-chat dialogue model.
  arXiv  Detail & Related papers  (2021-10-15T11:56:47Z)
• Rethinking Dialogue State Tracking with Reasoning [76.0991910623001]
  This paper proposes to track dialogue states gradually with reasoning over dialogue turns with the help of the back-end data. Empirical results demonstrate that our method significantly outperforms the state-of-the-art methods by 38.6% in terms of joint belief accuracy for MultiWOZ 2.1.
  arXiv  Detail & Related papers  (2020-05-27T02:05:33Z)
• Dialogue-Based Relation Extraction [53.2896545819799]
  We present the first human-annotated dialogue-based relation extraction (RE) dataset DialogRE. We argue that speaker-related information plays a critical role in the proposed task, based on an analysis of similarities and differences between dialogue-based and traditional RE tasks. Experimental results demonstrate that a speaker-aware extension on the best-performing model leads to gains in both the standard and conversational evaluation settings.
  arXiv  Detail & Related papers  (2020-04-17T03:51:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.