EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training

• URL: http://arxiv.org/abs/2108.01547v1
• Date: Tue, 3 Aug 2021 14:55:24 GMT
• Title: EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training
• Authors: Hao Zhou, Pei Ke, Zheng Zhang, Yuxian Gu, Yinhe Zheng, Chujie Zheng, Yida Wang, Chen Henry Wu, Hao Sun, Xiaocong Yang, Bosi Wen, Xiaoyan Zhu, Minlie Huang, Jie Tang
• Abstract summary: We propose EVA, a Chinese dialogue system that contains the largest Chinese pre-trained dialogue model with 2.8B parameters. To build this model, we collect the largest Chinese dialogue dataset named WDC-Dialogue from various public social media. Experiments on automatic and human evaluation show that EVA outperforms other Chinese pre-trained dialogue models.
• Score: 40.85554509137999
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although pre-trained language models have remarkably enhanced the generation ability of dialogue systems, open-domain Chinese dialogue systems are still limited by the dialogue data and the model size compared with English ones. In this paper, we propose EVA, a Chinese dialogue system that contains the largest Chinese pre-trained dialogue model with 2.8B parameters. To build this model, we collect the largest Chinese dialogue dataset named WDC-Dialogue from various public social media. This dataset contains 1.4B context-response pairs and is used as the pre-training corpus of EVA. Extensive experiments on automatic and human evaluation show that EVA outperforms other Chinese pre-trained dialogue models especially in the multi-turn interaction of human-bot conversations.

Related papers

• Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation [55.043492250775294]
  We introduce a novel Face-to-Face spoken dialogue model. It processes audio-visual speech from user input and generates audio-visual speech as the response. We also introduce MultiDialog, the first large-scale multimodal spoken dialogue corpus.
  arXiv  Detail & Related papers  (2024-06-12T04:48:36Z)
• Contextual Data Augmentation for Task-Oriented Dialog Systems [8.085645180329417]
  We develop a novel dialog augmentation model that generates a user turn, conditioning on full dialog context. With a new prompt design for language model, and output re-ranking, the dialogs generated from our model can be directly used to train downstream dialog systems.
  arXiv  Detail & Related papers  (2023-10-16T13:22:34Z)
• ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human [76.62897301298699]
  ChatPLUG is a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format. We show that modelname outperforms state-of-the-art Chinese dialogue systems on both automatic and human evaluation. We deploy modelname to real-world applications such as Smart Speaker and Instant Message applications with fast inference.
  arXiv  Detail & Related papers  (2023-04-16T18:16:35Z)
• GODEL: Large-Scale Pre-Training for Goal-Directed Dialog [119.1397031992088]
  We introduce GODEL, a large pre-trained language model for dialog. We show that GODEL outperforms state-of-the-art pre-trained dialog models in few-shot fine-tuning setups. A novel feature of our evaluation methodology is the introduction of a notion of utility that assesses the usefulness of responses.
  arXiv  Detail & Related papers  (2022-06-22T18:19:32Z)
• Building a Personalized Dialogue System with Prompt-Tuning [5.942602139622984]
  We build a dialogue system that responds based on a given character setting (persona) We propose an approach that uses prompt-tuning, which has low learning costs, on pre-trained large-scale language models.
  arXiv  Detail & Related papers  (2022-06-11T02:21:11Z)
• PANGUBOT: Efficient Generative Dialogue Pre-training from Pre-trained Language Model [47.858326419602115]
  We introduce PANGUBOT, a Chinese pre-trained open-domain dialogue generation model based on a large pre-trained language model (PLM) PANGU-alpha. We show that PANGUBOT outperforms state-of-the-art Chinese dialogue systems. We also demonstrate that PANGUBOT can be easily deployed to generate emotional responses without further training.
  arXiv  Detail & Related papers  (2022-03-31T15:09:12Z)
• EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training [73.98154158068134]
  EVA2.0 is a large-scale pre-trained open-domain Chinese dialogue model with 2.8 billion parameters. We propose EVA2.0, a large-scale pre-trained open-domain Chinese dialogue model with 2.8 billion parameters, and will make our models and codes publicly available.
  arXiv  Detail & Related papers  (2022-03-17T13:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.