Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues

• URL: http://arxiv.org/abs/2401.09248v2
• Date: Sat, 09 Nov 2024 13:07:39 GMT
• Title: Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues
• Authors: Dominic Petrak, Thy Thy Tran, Iryna Gurevych,
• Abstract summary: We introduce FEDI, the first English task-oriented and document-grounded dialogue dataset annotated with this information. Experiments with Flan-T5, GPT-2 and Llama 2 show a particularly positive impact on task completion and factual consistency.
• Score: 52.95506649193427
• License:
• Abstract: Implicit user feedback, user emotions and demographic information have shown to be promising sources for improving the accuracy and user engagement of responses generated by dialogue systems. However, the influence of such information on task completion and factual consistency, which are important criteria for task-oriented and document-grounded dialogues, is not yet known. To address this, we introduce FEDI, the first English task-oriented and document-grounded dialogue dataset annotated with this information. Our experiments with Flan-T5, GPT-2 and Llama 2 show a particularly positive impact on task completion and factual consistency. Participants in our human evaluation reported that the responses generated by the feedback-trained models were more informative (Flan-T5 and GPT-2), relevant and factual consistent (Llama 2).

Related papers

• Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
  In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied. We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
  arXiv  Detail & Related papers  (2024-04-19T16:45:50Z)
• $\textit{Dial BeInfo for Faithfulness}$: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuning [55.96744451743273]
  We introduce BeInfo, a method that applies behavioural tuning to aid information-seeking dialogue systems. We show that models tuned with BeInfo become considerably more faithful to the knowledge source. We also show that the models with 3B parameters tuned with BeInfo demonstrate strong performance on data from real production' conversations.
  arXiv  Detail & Related papers  (2023-11-16T11:25:44Z)
• WHAT, WHEN, and HOW to Ground: Designing User Persona-Aware Conversational Agents for Engaging Dialogue [4.328280329592151]
  We present a method for building a personalized open-domain dialogue system to address the WWH problem for natural response generation in a commercial setting. The proposed approach involves weighted dataset blending, negative persona information augmentation methods, and the design of personalized conversation datasets. Our work effectively balances dialogue fluency and tendency to ground, while also introducing a response-type label to improve the controllability and explainability of the grounded responses.
  arXiv  Detail & Related papers  (2023-06-06T02:28:38Z)
• FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systems [53.89014188309486]
  We present a flexible neural framework that can integrate contextual information from multiple channels. We evaluate our model on the MSDialog dataset widely used for evaluating conversational response ranking tasks.
  arXiv  Detail & Related papers  (2023-03-31T23:58:28Z)
• EVI: Multilingual Spoken Dialogue Tasks and Dataset for Knowledge-Based Enrolment, Verification, and Identification [49.77911492230467]
  We formalise the three authentication tasks and their evaluation protocols. We present EVI, a challenging spoken multilingual dataset with 5,506 dialogues in English, Polish, and French.
  arXiv  Detail & Related papers  (2022-04-28T13:39:24Z)
• Dual Task Framework for Debiasing Persona-grounded Dialogue Dataset [17.403065663306567]
  We introduce a data-centric approach for the task of improving persona-conditioned dialogue agents. Specifically, we augment relevant personas to improve dialogue dataset/agent, by leveraging the primal-dual structure of the two tasks. Experiments on Persona-Chat show that our approach outperforms pre-trained LMs by an 11.7 point gain in terms of accuracy.
  arXiv  Detail & Related papers  (2022-02-11T04:08:46Z)
• Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledge [25.378474996192438]
  We introduce a call For Customized conversation dataset where the customized answers are built with the user's persona and Wikipedia knowledge. We evaluate the abilities to make informative and customized utterances of pre-trained language models.
  arXiv  Detail & Related papers  (2021-12-16T04:44:27Z)
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots [62.295373408415365]
  We propose a personalized hybrid matching network (PHMN) for context-response matching. Our contributions are two-fold: 1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information. We evaluate our model on two large datasets with user identification, i.e., personalized dialogue Corpus Ubuntu (P- Ubuntu) and personalized Weibo dataset (P-Weibo)
  arXiv  Detail & Related papers  (2021-03-17T09:42:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.