EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems

• URL: http://arxiv.org/abs/2003.02958v1
• Date: Thu, 5 Mar 2020 23:09:24 GMT
• Title: EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems
• Authors: Rohola Zandie and Mohammad H. Mahoor
• Abstract summary: This paper presents EmpTransfo, a multi-head Transformer architecture for creating an empathetic dialog system. We show that utilizing the history of emotions and other metadata can improve the quality of generated conversations.
• Score: 4.41738804598711
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding emotions and responding accordingly is one of the biggest challenges of dialog systems. This paper presents EmpTransfo, a multi-head Transformer architecture for creating an empathetic dialog system. EmpTransfo utilizes state-of-the-art pre-trained models (e.g., OpenAI-GPT) for language generation, though models with different sizes can be used. We show that utilizing the history of emotions and other metadata can improve the quality of generated conversations by the dialog system. Our experimental results using a challenging language corpus show that the proposed approach outperforms other models in terms of Hit@1 and PPL (Perplexity).

Related papers

• Contextual Data Augmentation for Task-Oriented Dialog Systems [8.085645180329417]
  We develop a novel dialog augmentation model that generates a user turn, conditioning on full dialog context. With a new prompt design for language model, and output re-ranking, the dialogs generated from our model can be directly used to train downstream dialog systems.
  arXiv  Detail & Related papers  (2023-10-16T13:22:34Z)
• ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human [76.62897301298699]
  ChatPLUG is a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format. We show that modelname outperforms state-of-the-art Chinese dialogue systems on both automatic and human evaluation. We deploy modelname to real-world applications such as Smart Speaker and Instant Message applications with fast inference.
  arXiv  Detail & Related papers  (2023-04-16T18:16:35Z)
• An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation [27.445562543667357]
  We propose an Equal-size Hard Expectation--Maximization algorithm to train a multi-decoder model for diverse dialogue generation. Our algorithm assigns a sample to a decoder in a hard manner and additionally imposes an equal-assignment constraint to ensure that all decoders are well-trained.
  arXiv  Detail & Related papers  (2022-09-29T08:41:32Z)
• DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation [80.45816053153722]
  DialogVED introduces continuous latent variables into the enhanced encoder-decoder pre-training framework to increase the relevance and diversity of responses. We conduct experiments on PersonaChat, DailyDialog, and DSTC7-AVSD benchmarks for response generation.
  arXiv  Detail & Related papers  (2022-04-27T16:18:15Z)
• Modeling Coreference Relations in Visual Dialog [18.926582410644375]
  The occurrences of coreference relations in the dialog makes it a more challenging task than visual question-answering. We propose two soft constraints that can improve the model's ability of resolving coreferences in dialog in an unsupervised way.
  arXiv  Detail & Related papers  (2022-03-06T15:22:24Z)
• The Adapter-Bot: All-In-One Controllable Conversational Model [66.48164003532484]
  We propose a dialogue model that uses a fixed backbone model such as DialGPT and triggers on-demand dialogue skills via different adapters. Depending on the skills, the model is able to process multiple knowledge types, such as text, tables, and emphatic responses. We evaluate our model using automatic evaluation by comparing it with existing state-of-the-art conversational models.
  arXiv  Detail & Related papers  (2020-08-28T10:59:31Z)
• Video-Grounded Dialogues with Pretrained Generation Language Models [88.15419265622748]
  We leverage the power of pre-trained language models for improving video-grounded dialogue. We propose a framework by formulating sequence-to-grounded dialogue tasks as a sequence-to-grounded task. Our framework allows fine-tuning language models to capture dependencies across multiple modalities.
  arXiv  Detail & Related papers  (2020-06-27T08:24:26Z)
• VD-BERT: A Unified Vision and Dialog Transformer with BERT [161.0016161052714]
  We propose VD-BERT, a simple yet effective framework of unified vision-dialog Transformer. We adapt BERT for the effective fusion of vision and dialog contents via visually grounded training. Our model yields new state of the art, achieving the top position in both single-model and ensemble settings.
  arXiv  Detail & Related papers  (2020-04-28T04:08:46Z)
• Paraphrase Augmented Task-Oriented Dialog Generation [68.1790912977053]
  We propose a paraphrase augmented response generation (PARG) framework that jointly trains a paraphrase model and a response generation model. We also design a method to automatically construct paraphrase training data set based on dialog state and dialog act labels.
  arXiv  Detail & Related papers  (2020-04-16T05:12:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.