Pre-Trained and Attention-Based Neural Networks for Building Noetic Task-Oriented Dialogue Systems

• URL: http://arxiv.org/abs/2004.01940v1
• Date: Sat, 4 Apr 2020 14:14:43 GMT
• Title: Pre-Trained and Attention-Based Neural Networks for Building Noetic Task-Oriented Dialogue Systems
• Authors: Jia-Chen Gu, Tianda Li, Quan Liu, Xiaodan Zhu, Zhen-Hua Ling, Yu-Ping Ruan
• Abstract summary: This paper describes our systems that are evaluated on all subtasks under NOESIS II challenge. Several adaptation methods are proposed to adapt the pre-trained language models for multi-turn dialogue systems. In the released evaluation results of Track 2 of DSTC 8, our proposed models ranked fourth in subtask 1, third in subtask 2, and first in subtask 3 and subtask 4 respectively.
• Score: 47.230754691257836
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The NOESIS II challenge, as the Track 2 of the 8th Dialogue System Technology Challenges (DSTC 8), is the extension of DSTC 7. This track incorporates new elements that are vital for the creation of a deployed task-oriented dialogue system. This paper describes our systems that are evaluated on all subtasks under this challenge. We study the problem of employing pre-trained attention-based network for multi-turn dialogue systems. Meanwhile, several adaptation methods are proposed to adapt the pre-trained language models for multi-turn dialogue systems, in order to keep the intrinsic property of dialogue systems. In the released evaluation results of Track 2 of DSTC 8, our proposed models ranked fourth in subtask 1, third in subtask 2, and first in subtask 3 and subtask 4 respectively.

Related papers

• UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues [59.499965460525694]
  We propose a unified dialogue system (UniDS) with the two aforementioned skills. We design a unified dialogue data schema, compatible for both chit-chat and task-oriented dialogues. We train UniDS with mixed dialogue data from a pretrained chit-chat dialogue model.
  arXiv  Detail & Related papers  (2021-10-15T11:56:47Z)
• A Knowledge-Grounded Dialog System Based on Pre-Trained Language Models [0.7699714865575189]
  We present a knowledge-grounded dialog system developed for the ninth Dialog System Technology Challenge (DSTC9) We leverage transfer learning with existing language models to accomplish the tasks in this challenge track.
  arXiv  Detail & Related papers  (2021-06-28T07:56:10Z)
• A Unified Pre-training Framework for Conversational AI [25.514505462661763]
  PLATO-2 is trained via two-stage curriculum learning to fit the simplified one-to-one mapping relationship. PLATO-2 has obtained the 1st place in all three tasks, verifying its effectiveness as a unified framework for various dialogue systems.
  arXiv  Detail & Related papers  (2021-05-06T07:27:11Z)
• Overview of the Ninth Dialog System Technology Challenge: DSTC9 [111.35889309106359]
  The Ninth Dialog System Technology Challenge (DSTC-9) focuses on applying end-to-end dialog technologies for four distinct tasks in dialog systems. This paper describes the task definition, provided datasets, baselines and evaluation set-up for each track.
  arXiv  Detail & Related papers  (2020-11-12T16:43:10Z)
• The SPPD System for Schema Guided Dialogue State Tracking Challenge [2.181419218084711]
  This paper introduces one of our group's work on the Dialog System Technology Challenges 8 (DSTC8) The challenge, named as Track 4 in DSTC8, provides a brand new and challenging dataset for developing scalable multi-domain dialogue state tracking algorithms. We propose a zero-shot dialogue state tracking system for this task.
  arXiv  Detail & Related papers  (2020-06-16T09:57:40Z)
• Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System [49.39150449455407]
  HDNO is an option framework for designing latent dialogue acts to avoid designing specific dialogue act representations. We test HDNO on MultiWoz 2.0 and MultiWoz 2.1, the datasets on multi-domain dialogues, in comparison with word-level E2E model trained with RL, LaRL and HDSA.
  arXiv  Detail & Related papers  (2020-06-11T20:55:28Z)
• TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue [113.45485470103762]
  In this work, we unify nine human-human and multi-turn task-oriented dialogue datasets for language modeling. To better model dialogue behavior during pre-training, we incorporate user and system tokens into the masked language modeling.
  arXiv  Detail & Related papers  (2020-04-15T04:09:05Z)
• Recent Advances and Challenges in Task-oriented Dialog System [63.82055978899631]
  Task-oriented dialog systems are attracting more and more attention in academic and industrial communities. We discuss three critical topics for task-oriented dialog systems: (1) improving data efficiency to facilitate dialog modeling in low-resource settings, (2) modeling multi-turn dynamics for dialog policy learning, and (3) integrating domain knowledge into the dialog model.
  arXiv  Detail & Related papers  (2020-03-17T01:34:56Z)
• Multi-step Joint-Modality Attention Network for Scene-Aware Dialogue System [13.687071779732285]
  We propose a multi-step joint-modality attention network (JMAN) based on recurrent neural network (RNN) to reason on videos. Our model achieves a relative 12.1% and 22.4% improvement over the baseline on ROUGE-L score and CIDEr score.
  arXiv  Detail & Related papers  (2020-01-17T09:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.