Can Current Task-oriented Dialogue Models Automate Real-world Scenarios in the Wild?

• URL: http://arxiv.org/abs/2212.10504v2
• Date: Wed, 24 May 2023 11:46:33 GMT
• Title: Can Current Task-oriented Dialogue Models Automate Real-world Scenarios in the Wild?
• Authors: Sang-Woo Lee, Sungdong Kim, Donghyeon Ko, Donghoon Ham, Youngki Hong, Shin Ah Oh, Hyunhoon Jung, Wangkyo Jung, Kyunghyun Cho, Donghyun Kwak, Hyungsuk Noh, Woomyoung Park
• Abstract summary: Task-oriented dialogue (TOD) systems are mainly based on the slot-filling-based TOD (SF-TOD) framework. We argue that the current TOD benchmarks are limited to surrogate real-world scenarios and that the current TOD models are still a long way to cover the scenarios. In WebTOD, the dialogue system learns how to understand the web/mobile interface that the human agent interacts with, powered by a large-scale language model.
• Score: 48.79943762731801
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Task-oriented dialogue (TOD) systems are mainly based on the slot-filling-based TOD (SF-TOD) framework, in which dialogues are broken down into smaller, controllable units (i.e., slots) to fulfill a specific task. A series of approaches based on this framework achieved remarkable success on various TOD benchmarks. However, we argue that the current TOD benchmarks are limited to surrogate real-world scenarios and that the current TOD models are still a long way to cover the scenarios. In this position paper, we first identify current status and limitations of SF-TOD systems. After that, we explore the WebTOD framework, the alternative direction for building a scalable TOD system when a web/mobile interface is available. In WebTOD, the dialogue system learns how to understand the web/mobile interface that the human agent interacts with, powered by a large-scale language model.

Related papers

• Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbot [3.2288892242158984]
  We propose a novel approach to enhance Dialogue State Tracking (DST) performance. Our method enables Large Language Model (LLM) to infer dialogue states through carefully designed prompts. Our approach achieved state-of-the-art with a JGA of 42.57%, and performed well in open-domain real-world conversations.
  arXiv  Detail & Related papers  (2024-10-30T07:36:23Z)
• Training Zero-Shot Generalizable End-to-End Task-Oriented Dialog System Without Turn-level Dialog Annotations [2.757798192967912]
  This work employs multi-task instruction fine-tuning to create more efficient and scalable task-oriented dialogue systems. Our approach outperforms both state-of-the-art models trained on annotated data and billion-scale parameter off-the-shelf ChatGPT models.
  arXiv  Detail & Related papers  (2024-07-21T04:52:38Z)
• VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks [93.85005277463802]
  VisualWebArena is a benchmark designed to assess the performance of multimodal web agents on realistic tasks. To perform on this benchmark, agents need to accurately process image-text inputs, interpret natural language instructions, and execute actions on websites to accomplish user-defined objectives.
  arXiv  Detail & Related papers  (2024-01-24T18:35:21Z)
• InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems [60.53276524369498]
  Large language models (LLMs) have been used for diverse tasks in natural language processing (NLP) We present InstructTODS, a novel framework for zero-shot end-to-end task-oriented dialogue systems. InstructTODS generates a proxy belief state that seamlessly translates user intentions into dynamic queries.
  arXiv  Detail & Related papers  (2023-10-13T06:36:26Z)
• Enhancing Performance on Seen and Unseen Dialogue Scenarios using Retrieval-Augmented End-to-End Task-Oriented System [89.40590076430297]
  This work enables the TOD systems with more flexibility through a simple cache. We train end-to-end TOD models that can refer to and ground on both dialogue history and retrieved information during TOD generation. Experiments demonstrate the superior performance of our framework, with a notable improvement in non-empty joint goal accuracy by 6.7% compared to strong baselines.
  arXiv  Detail & Related papers  (2023-08-16T06:52:10Z)
• Enhancing Task Bot Engagement with Synthesized Open-Domain Dialog [89.35658776144638]
  It is essential to build a system that can handle both TOD and ODD and access different knowledge sources. We propose a framework for automatically generating dialogues that combine knowledge-grounded ODDs and TODs in various settings. We introduce a unified model PivotBot that is capable of appropriately adopting TOD and ODD modes and accessing different knowledge sources.
  arXiv  Detail & Related papers  (2022-12-20T05:51:47Z)
• A Chit-Chats Enhanced Task-Oriented Dialogue Corpora for Fuse-Motive Conversation Systems [9.541995537438394]
  We release a multi-turn dialogues dataset called CCET (Chinese Chat-Enhanced-Task) We propose a line of fuse-motive dialogues formalization approach, along with several evaluation metrics for TOD sessions that are integrated by CC utterances.
  arXiv  Detail & Related papers  (2022-05-12T05:43:18Z)
• Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation [70.81596088969378]
  Cross-lingual Outline-based Dialogue dataset (termed COD) enables natural language understanding. COD enables dialogue state tracking, and end-to-end dialogue modelling and evaluation in 4 diverse languages.
  arXiv  Detail & Related papers  (2022-01-31T18:11:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.