"What do others think?": Task-Oriented Conversational Modeling with Subjective Knowledge

• URL: http://arxiv.org/abs/2305.12091v2
• Date: Tue, 3 Oct 2023 03:33:20 GMT
• Title: "What do others think?": Task-Oriented Conversational Modeling with Subjective Knowledge
• Authors: Chao Zhao, Spandana Gella, Seokhwan Kim, Di Jin, Devamanyu Hazarika, Alexandros Papangelis, Behnam Hedayatnia, Mahdi Namazifar, Yang Liu, Dilek Hakkani-Tur
• Abstract summary: Task-oriented Dialogue (TOD) Systems aim to build dialogue systems that assist users in accomplishing specific goals. Traditional TODs rely on domain-specific APIs/DBs or external factual knowledge to generate responses. We propose a novel task of subjective-knowledge-based TOD (SK-TOD)
• Score: 69.46702968979885
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Task-oriented Dialogue (TOD) Systems aim to build dialogue systems that assist users in accomplishing specific goals, such as booking a hotel or a restaurant. Traditional TODs rely on domain-specific APIs/DBs or external factual knowledge to generate responses, which cannot accommodate subjective user requests (e.g., "Is the WIFI reliable?" or "Does the restaurant have a good atmosphere?"). To address this issue, we propose a novel task of subjective-knowledge-based TOD (SK-TOD). We also propose the first corresponding dataset, which contains subjective knowledge-seeking dialogue contexts and manually annotated responses grounded in subjective knowledge sources. When evaluated with existing TOD approaches, we find that this task poses new challenges such as aggregating diverse opinions from multiple knowledge snippets. We hope this task and dataset can promote further research on TOD and subjective content understanding. The code and the dataset are available at https://github.com/alexa/dstc11-track5.

Related papers

• Reasoning in Conversation: Solving Subjective Tasks through Dialogue Simulation for Large Language Models [56.93074140619464]
  We propose RiC (Reasoning in Conversation), a method that focuses on solving subjective tasks through dialogue simulation. The motivation of RiC is to mine useful contextual information by simulating dialogues instead of supplying chain-of-thought style rationales. We evaluate both API-based and open-source LLMs including GPT-4, ChatGPT, and OpenChat across twelve tasks.
  arXiv  Detail & Related papers  (2024-02-27T05:37:10Z)
• CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation [75.60156479374416]
  CGoDial is a new challenging and comprehensive Chinese benchmark for Goal-oriented Dialog evaluation. It contains 96,763 dialog sessions and 574,949 dialog turns totally, covering three datasets with different knowledge sources. To bridge the gap between academic benchmarks and spoken dialog scenarios, we either collect data from real conversations or add spoken features to existing datasets via crowd-sourcing.
  arXiv  Detail & Related papers  (2022-11-21T16:21:41Z)
• Q-TOD: A Query-driven Task-oriented Dialogue System [33.18698942938547]
  We introduce a novel query-driven task-oriented dialogue system, namely Q-TOD. The essential information from the dialogue context is extracted into a query, which is further employed to retrieve relevant knowledge records for response generation. To evaluate the effectiveness of the proposed Q-TOD, we collect query annotations for three publicly available task-oriented dialogue datasets.
  arXiv  Detail & Related papers  (2022-10-14T06:38:19Z)
• OPERA: Harmonizing Task-Oriented Dialogs and Information Seeking Experience [87.0233567695073]
  Existing studies in conversational AI mostly treat task-oriented dialog (TOD) and question answering (QA) as separate tasks. We propose a new task, Open-Book TOD (OB-TOD), which combines TOD with QA task and expand external knowledge sources. We propose a unified model OPERA which can appropriately access explicit and implicit external knowledge to tackle the defined task.
  arXiv  Detail & Related papers  (2022-06-24T18:21:26Z)
• KETOD: Knowledge-Enriched Task-Oriented Dialogue [77.59814785157877]
  Existing studies in dialogue system research mostly treat task-oriented dialogue and chit-chat as separate domains. We investigate how task-oriented dialogue and knowledge-grounded chit-chat can be effectively integrated into a single model.
  arXiv  Detail & Related papers  (2022-05-11T16:01:03Z)
• Database Search Results Disambiguation for Task-Oriented Dialog Systems [37.36255492341847]
  We propose Database Search Result (DSR) Disambiguation, a novel task that focuses on disambiguating database search results. To study this task, we augment the popular task-oriented dialog datasets (MultiWOZ and SGD) with turns that resolve ambiguities by (a) synthetically generating turns through a pre-defined grammar, and (b) collecting human paraphrases for a subset. We find that training on our augmented dialog data improves the model's ability to deal with ambiguous scenarios, without sacrificing performance on unmodified turns.
  arXiv  Detail & Related papers  (2021-12-15T18:56:18Z)
• Can I Be of Further Assistance? Using Unstructured Knowledge Access to Improve Task-oriented Conversational Modeling [39.60614611655266]
  This work focuses on responding to these beyond-API-coverage user turns by incorporating external, unstructured knowledge sources. We introduce novel data augmentation methods for the first two steps and demonstrate that the use of information extracted from dialogue context improves the knowledge selection and end-to-end performances.
  arXiv  Detail & Related papers  (2021-06-16T23:31:42Z)
• Reasoning in Dialog: Improving Response Generation by Context Reading Comprehension [49.92173751203827]
  In multi-turn dialog, utterances do not always take the full form of sentences. We propose to improve the response generation performance by examining the model's ability to answer a reading comprehension question.
  arXiv  Detail & Related papers  (2020-12-14T10:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.