JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset

• URL: http://arxiv.org/abs/2403.17319v1
• Date: Tue, 26 Mar 2024 02:01:18 GMT
• Title: JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset
• Authors: Atsumoto Ohashi, Ryu Hirai, Shinya Iizuka, Ryuichiro Higashinaka,
• Abstract summary: JMultiWOZ is the first Japanese language large-scale multi-domain task-oriented dialogue dataset. We evaluated the dialogue state tracking and response generation capabilities of the state-of-the-art methods.
• Score: 3.1311340484197814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dialogue datasets are crucial for deep learning-based task-oriented dialogue system research. While numerous English language multi-domain task-oriented dialogue datasets have been developed and contributed to significant advancements in task-oriented dialogue systems, such a dataset does not exist in Japanese, and research in this area is limited compared to that in English. In this study, towards the advancement of research and development of task-oriented dialogue systems in Japanese, we constructed JMultiWOZ, the first Japanese language large-scale multi-domain task-oriented dialogue dataset. Using JMultiWOZ, we evaluated the dialogue state tracking and response generation capabilities of the state-of-the-art methods on the existing major English benchmark dataset MultiWOZ2.2 and the latest large language model (LLM)-based methods. Our evaluation results demonstrated that JMultiWOZ provides a benchmark that is on par with MultiWOZ2.2. In addition, through evaluation experiments of interactive dialogues with the models and human participants, we identified limitations in the task completion capabilities of LLMs in Japanese.

Related papers

• Multi3WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systems [64.40789703661987]
  Multi3WOZ is a novel multilingual, multi-domain, multi-parallel ToD dataset. It is large-scale and offers culturally adapted dialogs in 4 languages. We describe a complex bottom-up data collection process that yielded the final dataset.
  arXiv  Detail & Related papers  (2023-07-26T08:29:42Z)
• DialogZoo: Large-Scale Dialog-Oriented Task Learning [52.18193690394549]
  We aim to build a unified foundation model which can solve massive diverse dialogue tasks. To achieve this goal, we first collect a large-scale well-labeled dialogue dataset from 73 publicly available datasets.
  arXiv  Detail & Related papers  (2022-05-25T11:17:16Z)
• Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog [67.20796950016735]
  Multi2WOZ dataset spans four typologically diverse languages: Chinese, German, Arabic, and Russian. We introduce a new framework for multilingual conversational specialization of pretrained language models (PrLMs) that aims to facilitate cross-lingual transfer for arbitrary downstream TOD tasks. Our experiments show that, in most setups, the best performance entails the combination of (I) conversational specialization in the target language and (ii) few-shot transfer for the concrete TOD task.
  arXiv  Detail & Related papers  (2022-05-20T18:35:38Z)
• ViWOZ: A Multi-Domain Task-Oriented Dialogue Systems Dataset For Low-resource Language [0.0]
  ViWOZ is the first multi-turn, multi-domain tasked oriented dataset in Vietnamese. The dataset consists of a total of 5,000 dialogues, including 60,946 fully annotated utterances. With those characteristics, the ViWOZ dataset enables future studies on creating a multilingual task-oriented dialogue system.
  arXiv  Detail & Related papers  (2022-03-15T09:22:04Z)
• Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation [70.81596088969378]
  Cross-lingual Outline-based Dialogue dataset (termed COD) enables natural language understanding. COD enables dialogue state tracking, and end-to-end dialogue modelling and evaluation in 4 diverse languages.
  arXiv  Detail & Related papers  (2022-01-31T18:11:21Z)
• GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systems [66.92182084456809]
  We introduce a novel data curation method that generates GlobalWoZ -- a large-scale multilingual ToD dataset from an English ToD dataset. Our method is based on translating dialogue templates and filling them with local entities in the target-language countries. We release our dataset as well as a set of strong baselines to encourage research on learning multilingual ToD systems for real use cases.
  arXiv  Detail & Related papers  (2021-10-14T19:33:04Z)
• BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling [52.99188200886738]
  BiToD is the first bilingual multi-domain dataset for end-to-end task-oriented dialogue modeling. BiToD contains over 7k multi-domain dialogues (144k utterances) with a large and realistic bilingual knowledge base.
  arXiv  Detail & Related papers  (2021-06-05T03:38:42Z)
• An Empirical Study of Cross-Lingual Transferability in Generative Dialogue State Tracker [33.2309643963072]
  We study the transferability of a cross-lingual generative dialogue state tracking system using a multilingual pre-trained seq2seq model. We also find out the low cross-lingual transferability of our approaches and provides investigation and discussion.
  arXiv  Detail & Related papers  (2021-01-27T12:45:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.