NUANCED: Natural Utterance Annotation for Nuanced Conversation with Estimated Distributions

• URL: http://arxiv.org/abs/2010.12758v2
• Date: Thu, 9 Sep 2021 17:58:10 GMT
• Title: NUANCED: Natural Utterance Annotation for Nuanced Conversation with Estimated Distributions
• Authors: Zhiyu Chen, Honglei Liu, Hu Xu, Seungwhan Moon, Hao Zhou, Bing Liu
• Abstract summary: In this work, we attempt to build a user-centric dialogue system. We first model the user preferences as estimated distributions over the system ontology and map the users' utterances to such distributions. We build a new dataset named NUANCED that focuses on such realistic settings for conversational recommendation.
• Score: 36.00476428803116
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing conversational systems are mostly agent-centric, which assumes the user utterances would closely follow the system ontology (for NLU or dialogue state tracking). However, in real-world scenarios, it is highly desirable that the users can speak freely in their own way. It is extremely hard, if not impossible, for the users to adapt to the unknown system ontology. In this work, we attempt to build a user-centric dialogue system. As there is no clean mapping for a user's free form utterance to an ontology, we first model the user preferences as estimated distributions over the system ontology and map the users' utterances to such distributions. Learning such a mapping poses new challenges on reasoning over existing knowledge, ranging from factoid knowledge, commonsense knowledge to the users' own situations. To this end, we build a new dataset named NUANCED that focuses on such realistic settings for conversational recommendation. Collected via dialogue simulation and paraphrasing, NUANCED contains 5.1k dialogues, 26k turns of high-quality user responses. We conduct experiments, showing both the usefulness and challenges of our problem setting. We believe NUANCED can serve as a valuable resource to push existing research from the agent-centric system to the user-centric system. The code and data is publicly available at \url{https://github.com/facebookresearch/nuanced}.

Related papers

• In-Context Learning User Simulators for Task-Oriented Dialog Systems [1.7086737326992172]
  This paper presents a novel application of large language models in user simulation for task-oriented dialog systems. By harnessing the power of these models, the proposed approach generates diverse utterances based on user goals and limited dialog examples.
  arXiv  Detail & Related papers  (2023-06-01T15:06:11Z)
• Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation [17.41434948048325]
  We conduct an interactive user study to unveil how vulnerable TOD systems are against realistic scenarios. Our study reveals that conversations in open-goal settings lead to catastrophic failures of the system. We discover a novel "pretending" behavior, in which the system pretends to handle the user requests even though they are beyond the system's capabilities.
  arXiv  Detail & Related papers  (2023-05-23T09:24:53Z)
• "What do others think?": Task-Oriented Conversational Modeling with Subjective Knowledge [69.46702968979885]
  Task-oriented Dialogue (TOD) Systems aim to build dialogue systems that assist users in accomplishing specific goals. Traditional TODs rely on domain-specific APIs/DBs or external factual knowledge to generate responses. We propose a novel task of subjective-knowledge-based TOD (SK-TOD)
  arXiv  Detail & Related papers  (2023-05-20T04:43:26Z)
• Grounding in social media: An approach to building a chit-chat dialogue model [9.247397520986999]
  Building open-domain dialogue systems capable of rich human-like conversational ability is one of the fundamental challenges in language generation. Current work on knowledge-grounded dialogue generation primarily focuses on persona incorporation or searching a fact-based structured knowledge source such as Wikipedia. Our method takes a broader and simpler approach, which aims to improve the raw conversation ability of the system by mimicking the human response behavior on social media.
  arXiv  Detail & Related papers  (2022-06-12T09:01:57Z)
• What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation [73.03318027164605]
  We propose to use information that can be automatically extracted from the next user utterance as a proxy to measure the quality of the previous system response. Our model generalizes across both spoken and written open-domain dialog corpora collected from real and paid users.
  arXiv  Detail & Related papers  (2022-03-25T22:09:52Z)
• Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems [109.16553492049441]
  We propose a novel method to incorporate the knowledge reasoning capability into dialogue systems in a more scalable and generalizable manner. To the best of our knowledge, this is the first work to have transformer models generate responses by reasoning over differentiable knowledge graphs.
  arXiv  Detail & Related papers  (2022-03-20T17:51:49Z)
• Advances and Challenges in Conversational Recommender Systems: A Survey [133.93908165922804]
  We provide a systematic review of the techniques used in current conversational recommender systems (CRSs) We summarize the key challenges of developing CRSs into five directions. These research directions involve multiple research fields like information retrieval (IR), natural language processing (NLP), and human-computer interaction (HCI)
  arXiv  Detail & Related papers  (2021-01-23T08:53:15Z)
• Saying No is An Art: Contextualized Fallback Responses for Unanswerable Dialogue Queries [3.593955557310285]
  Most dialogue systems rely on hybrid approaches for generating a set of ranked responses. We design a neural approach which generates responses which are contextually aware with the user query. Our simple approach makes use of rules over dependency parses and a text-to-text transformer fine-tuned on synthetic data of question-response pairs.
  arXiv  Detail & Related papers  (2020-12-03T12:34:22Z)
• Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning [69.42679922160684]
  We propose feedback-weighted learning based on importance sampling to improve upon an initial supervised system using binary user feedback. Our work opens the prospect to exploit interactions with real users and improve conversational systems after deployment.
  arXiv  Detail & Related papers  (2020-11-01T19:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.