Learning from Emotions, Demographic Information and Implicit User Feedback in Task-Oriented Document-Grounded Dialogues

• URL: http://arxiv.org/abs/2401.09248v1
• Date: Wed, 17 Jan 2024 14:52:26 GMT
• Title: Learning from Emotions, Demographic Information and Implicit User Feedback in Task-Oriented Document-Grounded Dialogues
• Authors: Dominic Petrak, Thy Thy Tran, Iryna Gurevych
• Abstract summary: We introduce FEDI, the first English dialogue dataset for task-oriented document-grounded dialogues annotated with demographic information, user emotions and implicit feedback. Our experiments with FLAN-T5, GPT-2 and LLaMA-2 show that these data have the potential to improve task completion and the factual consistency of the generated responses and user acceptance.
• Score: 59.516187851808375
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The success of task-oriented and document-grounded dialogue systems depends on users accepting and enjoying using them. To achieve this, recently published work in the field of Human-Computer Interaction suggests that the combination of considering demographic information, user emotions and learning from the implicit feedback in their utterances, is particularly important. However, these findings have not yet been transferred to the field of Natural Language Processing, where these data are primarily studied separately. Accordingly, no sufficiently annotated dataset is available. To address this gap, we introduce FEDI, the first English dialogue dataset for task-oriented document-grounded dialogues annotated with demographic information, user emotions and implicit feedback. Our experiments with FLAN-T5, GPT-2 and LLaMA-2 show that these data have the potential to improve task completion and the factual consistency of the generated responses and user acceptance.

Related papers

• Teaching Language Models To Gather Information Proactively [53.85419549904644]
  Large language models (LLMs) are increasingly expected to function as collaborative partners.<n>In this work, we introduce a new task paradigm: proactive information gathering.<n>We design a scalable framework that generates partially specified, real-world tasks, masking key information.<n>Within this setup, our core innovation is a reinforcement finetuning strategy that rewards questions that elicit genuinely new, implicit user information.
  arXiv  Detail & Related papers  (2025-07-28T23:50:09Z)
• Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
  In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied. We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
  arXiv  Detail & Related papers  (2024-04-19T16:45:50Z)
• $\textit{Dial BeInfo for Faithfulness}$: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuning [55.96744451743273]
  We introduce BeInfo, a method that applies behavioural tuning to aid information-seeking dialogue systems. We show that models tuned with BeInfo become considerably more faithful to the knowledge source. We also show that the models with 3B parameters tuned with BeInfo demonstrate strong performance on data from real production' conversations.
  arXiv  Detail & Related papers  (2023-11-16T11:25:44Z)
• WHAT, WHEN, and HOW to Ground: Designing User Persona-Aware Conversational Agents for Engaging Dialogue [4.328280329592151]
  We present a method for building a personalized open-domain dialogue system to address the WWH problem for natural response generation in a commercial setting. The proposed approach involves weighted dataset blending, negative persona information augmentation methods, and the design of personalized conversation datasets. Our work effectively balances dialogue fluency and tendency to ground, while also introducing a response-type label to improve the controllability and explainability of the grounded responses.
  arXiv  Detail & Related papers  (2023-06-06T02:28:38Z)
• FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systems [53.89014188309486]
  We present a flexible neural framework that can integrate contextual information from multiple channels. We evaluate our model on the MSDialog dataset widely used for evaluating conversational response ranking tasks.
  arXiv  Detail & Related papers  (2023-03-31T23:58:28Z)
• EVI: Multilingual Spoken Dialogue Tasks and Dataset for Knowledge-Based Enrolment, Verification, and Identification [49.77911492230467]
  We formalise the three authentication tasks and their evaluation protocols. We present EVI, a challenging spoken multilingual dataset with 5,506 dialogues in English, Polish, and French.
  arXiv  Detail & Related papers  (2022-04-28T13:39:24Z)
• Dual Task Framework for Debiasing Persona-grounded Dialogue Dataset [17.403065663306567]
  We introduce a data-centric approach for the task of improving persona-conditioned dialogue agents. Specifically, we augment relevant personas to improve dialogue dataset/agent, by leveraging the primal-dual structure of the two tasks. Experiments on Persona-Chat show that our approach outperforms pre-trained LMs by an 11.7 point gain in terms of accuracy.
  arXiv  Detail & Related papers  (2022-02-11T04:08:46Z)
• Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledge [25.378474996192438]
  We introduce a call For Customized conversation dataset where the customized answers are built with the user's persona and Wikipedia knowledge. We evaluate the abilities to make informative and customized utterances of pre-trained language models.
  arXiv  Detail & Related papers  (2021-12-16T04:44:27Z)
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots [62.295373408415365]
  We propose a personalized hybrid matching network (PHMN) for context-response matching. Our contributions are two-fold: 1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information. We evaluate our model on two large datasets with user identification, i.e., personalized dialogue Corpus Ubuntu (P- Ubuntu) and personalized Weibo dataset (P-Weibo)
  arXiv  Detail & Related papers  (2021-03-17T09:42:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.