Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecture

• URL: http://arxiv.org/abs/2210.06706v1
• Date: Thu, 13 Oct 2022 03:57:17 GMT
• Title: Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecture
• Authors: Hong Liu, Zhijian Ou, Yi Huang and Junlan Feng
• Abstract summary: Online reinforcement learning of a GPT-2 based dialog system (DS) and a end-to-end user simulator (US) has not ever been explored. In this paper, we first propose Simplified Generative Architectures (SGA) for DS and US respectively, both based on GPT-2 but using shortened history. Our DS with the proposed SGA, when only supervised trained, achieves state-of-the-art performance on MultiWOZ2.1 and is more compute-efficient in both training and generation.
• Score: 24.305558215176752
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, there has been progress in supervised funetuning pretrained GPT-2 to build end-to-end task-oriented dialog (TOD) systems. However, online reinforcement learning of a GPT-2 based dialog system (DS), together with a end-to-end user simulator (US), has not ever been explored. Moreover, a drawback with existing GPT-2 based TOD systems is that they mostly employ the whole dialog history as input, which brings inefficiencies in memory and compute. In this paper, we first propose Simplified Generative Architectures (SGA) for DS and US respectively, both based on GPT-2 but using shortened history. Then, we successfully develop Jointly Reinforced US and DS, called SGA-JRUD. Our DS with the proposed SGA, when only supervised trained, achieves state-of-the-art performance on MultiWOZ2.1 and is more compute-efficient in both training and generation. Extensive experiments on MultiWOZ2.1 further show the superiority of SGA-JRUD in both offline and online evaluations.

Related papers

• RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation [80.20970723577818]
  We introduce RoboCerebra, a benchmark for evaluating high-level reasoning in long-horizon robotic manipulation.<n>The dataset is constructed via a top-down pipeline, where GPT generates task instructions and decomposes them into subtask sequences.<n>Compared to prior benchmarks, RoboCerebra features significantly longer action sequences and denser annotations.
  arXiv  Detail & Related papers  (2025-06-07T06:15:49Z)
• The 2nd FutureDial Challenge: Dialog Systems with Retrieval Augmented Generation (FutureDial-RAG) [23.849336345191556]
  The challenge builds upon the MobileCS2 dataset, a real-life customer service datasets with nearly 3000 high-quality dialogs. We define two tasks, track 1 for knowledge retrieval and track 2 for response generation, which are core research questions in dialog systems with RAG. We build baseline systems for the two tracks and design metrics to measure whether the systems can perform accurate retrieval and generate informative and coherent response.
  arXiv  Detail & Related papers  (2024-05-21T07:35:21Z)
• InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems [60.53276524369498]
  Large language models (LLMs) have been used for diverse tasks in natural language processing (NLP) We present InstructTODS, a novel framework for zero-shot end-to-end task-oriented dialogue systems. InstructTODS generates a proxy belief state that seamlessly translates user intentions into dynamic queries.
  arXiv  Detail & Related papers  (2023-10-13T06:36:26Z)
• Enhancing Performance on Seen and Unseen Dialogue Scenarios using Retrieval-Augmented End-to-End Task-Oriented System [89.40590076430297]
  This work enables the TOD systems with more flexibility through a simple cache. We train end-to-end TOD models that can refer to and ground on both dialogue history and retrieved information during TOD generation. Experiments demonstrate the superior performance of our framework, with a notable improvement in non-empty joint goal accuracy by 6.7% compared to strong baselines.
  arXiv  Detail & Related papers  (2023-08-16T06:52:10Z)
• Semi-Supervised Knowledge-Grounded Pre-training for Task-Oriented Dialog Systems [25.164042288343683]
  We present our models for Track 2 of the SereTOD 2022 challenge, which is the first challenge of building semi-supervised and reinforced TOD systems. We build a knowledge-grounded dialog model to formulate dialog history and local KB as input and predict the system response. And we perform semi-supervised pre-training both on the labeled and unlabeled data.
  arXiv  Detail & Related papers  (2022-10-17T09:10:03Z)
• A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systems [22.249113574918034]
  We propose a generative user simulator (GUS) with GPT-2 based architecture and goal state tracking. The GUS achieves superior results in all three evaluation tasks.
  arXiv  Detail & Related papers  (2022-10-17T01:57:50Z)
• A Multi-Task BERT Model for Schema-Guided Dialogue State Tracking [78.2700757742992]
  Task-oriented dialogue systems often employ a Dialogue State Tracker (DST) to successfully complete conversations. Recent state-of-the-art DST implementations rely on schemata of diverse services to improve model robustness. We propose a single multi-task BERT-based model that jointly solves the three DST tasks of intent prediction, requested slot prediction and slot filling.
  arXiv  Detail & Related papers  (2022-07-02T13:27:59Z)
• Revisiting Markovian Generative Architectures for Efficient Task-Oriented Dialog Systems [22.249113574918034]
  We propose to revisit Markovian Generative Architectures (MGA), which have been used in previous LSTM-based TOD systems. Experiments on MultiWOZ2.1 show the efficiency advantages of the proposed Markovian PLM-based systems over their non-Markovian counterparts.
  arXiv  Detail & Related papers  (2022-04-13T15:21:34Z)
• HyKnow: End-to-End Task-Oriented Dialog Modeling with Hybrid Knowledge Management [58.82499963373537]
  We propose a TOD system with hybrid knowledge management, HyKnow. It extends the belief state to manage both structured and unstructured knowledge. It is the first end-to-end model that jointly optimize modeling grounded on these two kinds of knowledge.
  arXiv  Detail & Related papers  (2021-05-13T01:58:39Z)
• A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretraining [25.674966922466467]
  This paper describes our submission for the End-to-end Multi-domain Task Completion Dialog shared task at the 9th Dialog System Technology Challenge (DSTC-9) Participants in the shared task build an end-to-end task completion dialog system which is evaluated by human evaluation and a user simulator based automatic evaluation.
  arXiv  Detail & Related papers  (2021-02-08T20:02:30Z)
• MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems [75.43457658815943]
  We propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ.
  arXiv  Detail & Related papers  (2020-09-25T02:19:13Z)
• Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System [49.39150449455407]
  HDNO is an option framework for designing latent dialogue acts to avoid designing specific dialogue act representations. We test HDNO on MultiWoz 2.0 and MultiWoz 2.1, the datasets on multi-domain dialogues, in comparison with word-level E2E model trained with RL, LaRL and HDSA.
  arXiv  Detail & Related papers  (2020-06-11T20:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.