Are Current Task-oriented Dialogue Systems Able to Satisfy Impolite Users?

• URL: http://arxiv.org/abs/2210.12942v1
• Date: Mon, 24 Oct 2022 04:11:52 GMT
• Title: Are Current Task-oriented Dialogue Systems Able to Satisfy Impolite Users?
• Authors: Zhiqiang Hu, Roy Kaa-Wei Lee, Nancy F. Chen
• Abstract summary: We constructed an impolite dialogue corpus and conducted experiments to evaluate the state-of-the-art TOD systems. Our experimental results show that existing TOD systems are unable to handle impolite user utterances. We also present a data augmentation method to improve TOD performance in impolite dialogues.
• Score: 26.066439234012275
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Task-oriented dialogue (TOD) systems have assisted users on many tasks, including ticket booking and service inquiries. While existing TOD systems have shown promising performance in serving customer needs, these systems mostly assume that users would interact with the dialogue agent politely. This assumption is unrealistic as impatient or frustrated customers may also interact with TOD systems impolitely. This paper aims to address this research gap by investigating impolite users' effects on TOD systems. Specifically, we constructed an impolite dialogue corpus and conducted extensive experiments to evaluate the state-of-the-art TOD systems on our impolite dialogue corpus. Our experimental results show that existing TOD systems are unable to handle impolite user utterances. We also present a data augmentation method to improve TOD performance in impolite dialogues. Nevertheless, handling impolite dialogues remains a very challenging research task. We hope by releasing the impolite dialogue corpus and establishing the benchmark evaluations, more researchers are encouraged to investigate this new challenging research task.

Related papers

• Who Speaks Next? Multi-party AI Discussion Leveraging the Systematics of Turn-taking in Murder Mystery Games [7.946510318969307]
  This study focuses on conversational norms such as adjacency pairs and turn-taking found in conversation analysis. We propose a new framework called "Murder Mystery Agents" that applies these norms to AI agents' dialogue control. The proposed framework integrates next speaker selection based on adjacency pairs and a self-selection mechanism.
  arXiv  Detail & Related papers  (2024-12-06T10:45:54Z)
• Are cascade dialogue state tracking models speaking out of turn in spoken dialogues? [1.786898113631979]
  This paper proposes a comprehensive analysis of the errors of state of the art systems in complex settings such as Dialogue State Tracking. Based on spoken MultiWoz, we identify that errors on non-categorical slots' values are essential to address in order to bridge the gap between spoken and chat-based dialogue systems.
  arXiv  Detail & Related papers  (2023-11-03T08:45:22Z)
• Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals [76.69419538047813]
  ProToD approach anticipates the future dialogue actions and incorporates the goal-oriented reward signal to enhance ToD systems. We present a novel evaluation method that assesses ToD systems based on goal-driven dialogue simulations. Empirical experiments conducted on the MultiWoZ 2.1 dataset demonstrate that our model can achieve superior performance using only 10% of the data.
  arXiv  Detail & Related papers  (2023-09-16T10:56:00Z)
• A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects [100.75759050696355]
  We provide a comprehensive overview of the prominent problems and advanced designs for conversational agent's proactivity in different types of dialogues. We discuss challenges that meet the real-world application needs but require a greater research focus in the future.
  arXiv  Detail & Related papers  (2023-05-04T11:38:49Z)
• A Chit-Chats Enhanced Task-Oriented Dialogue Corpora for Fuse-Motive Conversation Systems [9.541995537438394]
  We release a multi-turn dialogues dataset called CCET (Chinese Chat-Enhanced-Task) We propose a line of fuse-motive dialogues formalization approach, along with several evaluation metrics for TOD sessions that are integrated by CC utterances.
  arXiv  Detail & Related papers  (2022-05-12T05:43:18Z)
• User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems [65.88679683468143]
  We propose a novel framework, namely USDA, to incorporate the sequential dynamics of dialogue acts for predicting user satisfaction. USDA incorporates the sequential transitions of both content and act features in the dialogue to predict the user satisfaction. Experimental results on four benchmark goal-oriented dialogue datasets show that the proposed method substantially and consistently outperforms existing methods on USE.
  arXiv  Detail & Related papers  (2022-02-07T02:50:07Z)
• Task-oriented Dialogue Systems: performance vs. quality-optima, a review [0.0]
  State-of-the-art task-oriented dialogue systems are not yet reaching their full potential. Other conversational quality attributes that may point to the success, or otherwise, of the dialogue, may be ignored. This paper explores the literature on evaluative frameworks of dialogue systems and the role of conversational quality attributes in dialogue systems.
  arXiv  Detail & Related papers  (2021-12-21T13:16:24Z)
• Automatic Evaluation and Moderation of Open-domain Dialogue Systems [59.305712262126264]
  A long standing challenge that bothers the researchers is the lack of effective automatic evaluation metrics. This paper describes the data, baselines and results obtained for the Track 5 at the Dialogue System Technology Challenge 10 (DSTC10)
  arXiv  Detail & Related papers  (2021-11-03T10:08:05Z)
• UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues [59.499965460525694]
  We propose a unified dialogue system (UniDS) with the two aforementioned skills. We design a unified dialogue data schema, compatible for both chit-chat and task-oriented dialogues. We train UniDS with mixed dialogue data from a pretrained chit-chat dialogue model.
  arXiv  Detail & Related papers  (2021-10-15T11:56:47Z)
• Recent Advances and Challenges in Task-oriented Dialog System [63.82055978899631]
  Task-oriented dialog systems are attracting more and more attention in academic and industrial communities. We discuss three critical topics for task-oriented dialog systems: (1) improving data efficiency to facilitate dialog modeling in low-resource settings, (2) modeling multi-turn dynamics for dialog policy learning, and (3) integrating domain knowledge into the dialog model.
  arXiv  Detail & Related papers  (2020-03-17T01:34:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.