Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation

• URL: http://arxiv.org/abs/2105.09235v1
• Date: Wed, 19 May 2021 16:34:33 GMT
• Title: Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation
• Authors: Giovanni Bonetta, Rossella Cancelliere, Ding Liu, Paul Vozila
• Abstract summary: We present a transformer-based model for multi-turn dialog response generation. Our solution is based on a hybrid approach which augments a transformer-based generative model with a novel retrieval mechanism.
• Score: 16.90730526207747
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformer-based models have demonstrated excellent capabilities of capturing patterns and structures in natural language generation and achieved state-of-the-art results in many tasks. In this paper we present a transformer-based model for multi-turn dialog response generation. Our solution is based on a hybrid approach which augments a transformer-based generative model with a novel retrieval mechanism, which leverages the memorized information in the training data via k-Nearest Neighbor search. Our system is evaluated on two datasets made by customer/assistant dialogs: the Taskmaster-1, released by Google and holding high quality, goal-oriented conversational data and a proprietary dataset collected from a real customer service call center. Both achieve better BLEU scores over strong baselines.

Related papers

• Contrastive Transformer Learning with Proximity Data Generation for Text-Based Person Search [60.626459715780605]
  Given a descriptive text query, text-based person search aims to retrieve the best-matched target person from an image gallery. Such a cross-modal retrieval task is quite challenging due to significant modality gap, fine-grained differences and insufficiency of annotated data. In this paper, we propose a simple yet effective dual Transformer model for text-based person search.
  arXiv  Detail & Related papers  (2023-11-15T16:26:49Z)
• Stateful Memory-Augmented Transformers for Efficient Dialogue Modeling [69.31802246621963]
  We propose a novel memory-augmented transformer that is compatible with existing pre-trained encoder-decoder models. By incorporating a separate memory module alongside the pre-trained transformer, the model can effectively interchange information between the memory states and the current input context.
  arXiv  Detail & Related papers  (2022-09-15T22:37:22Z)
• AARGH! End-to-end Retrieval-Generation for Task-Oriented Dialog [3.42658286826597]
  AARGH is an end-to-end task-oriented dialog system combining retrieval and generative approaches in a single model. We show that our approach produces more diverse outputs while maintaining or improving state tracking and context-to-response generation performance.
  arXiv  Detail & Related papers  (2022-09-08T08:15:22Z)
• DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation [80.45816053153722]
  DialogVED introduces continuous latent variables into the enhanced encoder-decoder pre-training framework to increase the relevance and diversity of responses. We conduct experiments on PersonaChat, DailyDialog, and DSTC7-AVSD benchmarks for response generation.
  arXiv  Detail & Related papers  (2022-04-27T16:18:15Z)
• A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation [107.82729587882397]
  It is expensive to scale up current persona-based dialogue datasets. Each data sample in this task is more complex to learn with than conventional dialogue data. We propose a data manipulation method, which is model-agnostic to be packed with any persona-based dialogue generation model.
  arXiv  Detail & Related papers  (2022-04-21T03:49:54Z)
• Quick Starting Dialog Systems with Paraphrase Generation [0.0]
  We propose a method to reduce the cost and effort of creating new conversational agents by artificially generating more data from existing examples. Our proposed approach can kick-start a dialog system with little human effort, and brings its performance to a level satisfactory enough for allowing actual interactions with real end-users.
  arXiv  Detail & Related papers  (2022-04-06T02:35:59Z)
• Context Matters in Semantically Controlled Language Generation for Task-oriented Dialogue Systems [6.1478669848771546]
  This work combines information about the dialogue history encoded by pre-trained model with a meaning representation of the current system utterance to realize contextual language generation in task-oriented dialogues. We utilize the pre-trained multi-context ConveRT model for context representation in a model trained from scratch; and leverage the immediate preceding user utterance for context generation in a model adapted from the pre-trained GPT-2.
  arXiv  Detail & Related papers  (2021-11-28T11:48:02Z)
• Robust Conversational AI with Grounded Text Generation [77.56950706340767]
  GTG is a hybrid model which uses a large-scale Transformer neural network as its backbone. It generates responses grounded in dialog belief state and real-world knowledge for task completion.
  arXiv  Detail & Related papers  (2020-09-07T23:49:28Z)
• Hybrid Generative-Retrieval Transformers for Dialogue Domain Adaptation [77.62366712130196]
  We present the winning entry at the fast domain adaptation task of DSTC8, a hybrid generative-retrieval model based on GPT-2 fine-tuned to the multi-domain MetaLWOz dataset. Our model uses retrieval logic as a fallback, being SoTA on MetaLWOz in human evaluation (>4% improvement over the 2nd place system) and attaining competitive generalization performance in adaptation to the unseen MultiWOZ dataset.
  arXiv  Detail & Related papers  (2020-03-03T18:07:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.