Learning from Emotions, Demographic Information and Implicit User Feedback in Task-Oriented Document-Grounded Dialogues

• URL: http://arxiv.org/abs/2401.09248v1
• Date: Wed, 17 Jan 2024 14:52:26 GMT
• Title: Learning from Emotions, Demographic Information and Implicit User Feedback in Task-Oriented Document-Grounded Dialogues
• Authors: Dominic Petrak, Thy Thy Tran, Iryna Gurevych
• Abstract summary: We introduce FEDI, the first English dialogue dataset for task-oriented document-grounded dialogues annotated with demographic information, user emotions and implicit feedback. Our experiments with FLAN-T5, GPT-2 and LLaMA-2 show that these data have the potential to improve task completion and the factual consistency of the generated responses and user acceptance.
• Score: 59.516187851808375
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The success of task-oriented and document-grounded dialogue systems depends on users accepting and enjoying using them. To achieve this, recently published work in the field of Human-Computer Interaction suggests that the combination of considering demographic information, user emotions and learning from the implicit feedback in their utterances, is particularly important. However, these findings have not yet been transferred to the field of Natural Language Processing, where these data are primarily studied separately. Accordingly, no sufficiently annotated dataset is available. To address this gap, we introduce FEDI, the first English dialogue dataset for task-oriented document-grounded dialogues annotated with demographic information, user emotions and implicit feedback. Our experiments with FLAN-T5, GPT-2 and LLaMA-2 show that these data have the potential to improve task completion and the factual consistency of the generated responses and user acceptance.

Related papers

• Natural Language Task-Oriented Dialog System 2.0 [2.757798192967912]
  Task-oriented dialog (TOD) systems play a crucial role in facilitating efficient interactions between users and machines. These systems traditionally rely on manually annotated metadata, such as dialog states and policy annotations. We introduce Natural Language Task Oriented Dialog System (NL-ToD), a novel model that removes the dependency on manually annotated turn-wise data.
  arXiv  Detail & Related papers  (2024-07-21T04:52:38Z)
• Investigating Low-Cost LLM Annotation for~Spoken Dialogue Understanding Datasets [9.78470355087662]
  In spoken Task-Oriented Dialogue (TOD) systems, the choice of the semantic representation describing the users' requests is key to a smooth interaction. This paper provides insights into automatic enhancement of spoken dialogue datasets' semantic representations.
  arXiv  Detail & Related papers  (2024-06-19T06:59:57Z)
• Narrative Action Evaluation with Prompt-Guided Multimodal Interaction [60.281405999483]
  Narrative action evaluation (NAE) aims to generate professional commentary that evaluates the execution of an action. NAE is a more challenging task because it requires both narrative flexibility and evaluation rigor. We propose a prompt-guided multimodal interaction framework to facilitate the interaction between different modalities of information.
  arXiv  Detail & Related papers  (2024-04-22T17:55:07Z)
• "You tell me": A Dataset of GPT-4-Based Behaviour Change Support Conversations [1.104960878651584]
  We share a dataset containing text-based user interactions related to behaviour change with two GPT-4-based conversational agents. This dataset includes conversation data, user language analysis, perception measures, and user feedback for LLM-generated turns.
  arXiv  Detail & Related papers  (2024-01-29T13:54:48Z)
• Learning From Free-Text Human Feedback -- Collect New Datasets Or Extend Existing Ones? [57.16050211534735]
  We investigate the types and frequency of free-text human feedback in commonly used dialog datasets. Our findings provide new insights into the composition of the datasets examined, including error types, user response types, and the relations between them.
  arXiv  Detail & Related papers  (2023-10-24T12:01:11Z)
• Evaluating Large Language Models for Document-grounded Response Generation in Information-Seeking Dialogues [17.41334279810008]
  We investigate the use of large language models (LLMs) like ChatGPT for document-grounded response generation in the context of information-seeking dialogues. For evaluation, we use the MultiDoc2Dial corpus of task-oriented dialogues in four social service domains. While both ChatGPT variants are more likely to include information not present in the relevant segments, possibly including a presence of hallucinations, they are rated higher than both the shared task winning system and human responses.
  arXiv  Detail & Related papers  (2023-09-21T07:28:03Z)
• Information Extraction and Human-Robot Dialogue towards Real-life Tasks: A Baseline Study with the MobileCS Dataset [52.22314870976088]
  The SereTOD challenge is organized and releases the MobileCS dataset, which consists of real-world dialog transcripts between real users and customer-service staffs from China Mobile. Based on the MobileCS dataset, the SereTOD challenge has two tasks, not only evaluating the construction of the dialogue system itself, but also examining information extraction from dialog transcripts. This paper mainly presents a baseline study of the two tasks with the MobileCS dataset.
  arXiv  Detail & Related papers  (2022-09-27T15:30:43Z)
• OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue [40.62090743056549]
  This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD) Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: dialogue state tracker (DST) and response generator (RG)
  arXiv  Detail & Related papers  (2022-09-10T04:38:27Z)
• Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation [70.81596088969378]
  Cross-lingual Outline-based Dialogue dataset (termed COD) enables natural language understanding. COD enables dialogue state tracking, and end-to-end dialogue modelling and evaluation in 4 diverse languages.
  arXiv  Detail & Related papers  (2022-01-31T18:11:21Z)
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots [62.295373408415365]
  We propose a personalized hybrid matching network (PHMN) for context-response matching. Our contributions are two-fold: 1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information. We evaluate our model on two large datasets with user identification, i.e., personalized dialogue Corpus Ubuntu (P- Ubuntu) and personalized Weibo dataset (P-Weibo)
  arXiv  Detail & Related papers  (2021-03-17T09:42:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.