Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems

• URL: http://arxiv.org/abs/2306.04357v5
• Date: Wed, 27 Mar 2024 03:06:13 GMT
• Title: Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems
• Authors: Zhenpeng Su, Xing Wu, Wei Zhou, Guangyuan Ma, Songlin Hu,
• Abstract summary: Dial-MAE is a straightforward yet effective post-training technique tailored for dense encoders in dialogue response selection. Our experiments have demonstrated that Dial-MAE is highly effective, achieving state-of-the-art performance on two commonly evaluated benchmarks.
• Score: 22.302137281411646
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dialogue response selection aims to select an appropriate response from several candidates based on a given user and system utterance history. Most existing works primarily focus on post-training and fine-tuning tailored for cross-encoders. However, there are no post-training methods tailored for dense encoders in dialogue response selection. We argue that when the current language model, based on dense dialogue systems (such as BERT), is employed as a dense encoder, it separately encodes dialogue context and response, leading to a struggle to achieve the alignment of both representations. Thus, we propose Dial-MAE (Dialogue Contextual Masking Auto-Encoder), a straightforward yet effective post-training technique tailored for dense encoders in dialogue response selection. Dial-MAE uses an asymmetric encoder-decoder architecture to compress the dialogue semantics into dense vectors, which achieves better alignment between the features of the dialogue context and response. Our experiments have demonstrated that Dial-MAE is highly effective, achieving state-of-the-art performance on two commonly evaluated benchmarks.

Related papers

• Fine-grained Conversational Decoding via Isotropic and Proximal Search [17.904421465456814]
  We present a fine-grained conversational decoding method, termed textitisotropic and proximal search (IPS). Our method is designed to generate the semantic-concentrated response, while still maintaining informativeness and discrimination against the context. Experiments show that our approach outperforms existing decoding strategies in the dialogue field across both automatic and human evaluation metrics.
  arXiv  Detail & Related papers  (2023-10-12T08:38:12Z)
• SuperDialseg: A Large-scale Dataset for Supervised Dialogue Segmentation [55.82577086422923]
  We provide a feasible definition of dialogue segmentation points with the help of document-grounded dialogues. We release a large-scale supervised dataset called SuperDialseg, containing 9,478 dialogues. We also provide a benchmark including 18 models across five categories for the dialogue segmentation task.
  arXiv  Detail & Related papers  (2023-05-15T06:08:01Z)
• Unsupervised Dialogue Topic Segmentation with Topic-aware Utterance Representation [51.22712675266523]
  Dialogue Topic (DTS) plays an essential role in a variety of dialogue modeling tasks. We propose a novel unsupervised DTS framework, which learns topic-aware utterance representations from unlabeled dialogue data.
  arXiv  Detail & Related papers  (2023-05-04T11:35:23Z)
• Dialog act guided contextual adapter for personalized speech recognition [9.672512327395435]
  Personalization in multi-turn dialogs has been a long standing challenge for end-to-end automatic speech recognition (E2E ASR) models. Recent work on contextual adapters has tackled rare word recognition using user catalogs. We propose a dialog act guided contextual adapter network.
  arXiv  Detail & Related papers  (2023-03-31T05:13:44Z)
• An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation [27.445562543667357]
  We propose an Equal-size Hard Expectation--Maximization algorithm to train a multi-decoder model for diverse dialogue generation. Our algorithm assigns a sample to a decoder in a hard manner and additionally imposes an equal-assignment constraint to ensure that all decoders are well-trained.
  arXiv  Detail & Related papers  (2022-09-29T08:41:32Z)
• SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation [123.37377363355363]
  SPACE-3 is a novel unified semi-supervised pre-trained conversation model learning from large-scale dialog corpora. It can be effectively fine-tuned on a wide range of downstream dialog tasks. Results show that SPACE-3 achieves state-of-the-art performance on eight downstream dialog benchmarks.
  arXiv  Detail & Related papers  (2022-09-14T14:17:57Z)
• Discovering Dialog Structure Graph for Open-Domain Dialog Generation [51.29286279366361]
  We conduct unsupervised discovery of dialog structure from chitchat corpora. We then leverage it to facilitate dialog generation in downstream systems. We present a Discrete Variational Auto-Encoder with Graph Neural Network (DVAE-GNN), to discover a unified human-readable dialog structure.
  arXiv  Detail & Related papers  (2020-12-31T10:58:37Z)
• Reasoning in Dialog: Improving Response Generation by Context Reading Comprehension [49.92173751203827]
  In multi-turn dialog, utterances do not always take the full form of sentences. We propose to improve the response generation performance by examining the model's ability to answer a reading comprehension question.
  arXiv  Detail & Related papers  (2020-12-14T10:58:01Z)
• Multi-turn Response Selection using Dialogue Dependency Relations [39.99448321736736]
  Multi-turn response selection is a task designed for developing dialogue agents. We propose a dialogue extraction algorithm to transform a dialogue history into threads based on their dependency relations. Our model outperforms the state-of-the-art baselines on both D7 and DSTC8*, with competitive results on Ubuntu.
  arXiv  Detail & Related papers  (2020-10-04T08:00:19Z)
• Hierarchical Pre-training for Sequence Labelling in Spoken Dialog [10.216901061363641]
  We propose a new approach to learn generic representations adapted to spoken dialog. We obtain our representations with a hierarchical encoder based on transformer architectures. Pre-training is performed on OpenSubtitles: a large corpus of spoken dialog containing over $2.3$ billion of tokens.
  arXiv  Detail & Related papers  (2020-09-23T13:54:57Z)
• Conversation Learner -- A Machine Teaching Tool for Building Dialog Managers for Task-Oriented Dialog Systems [57.082447660944965]
  Conversation Learner is a machine teaching tool for building dialog managers. It enables dialog authors to create a dialog flow using familiar tools, converting the dialog flow into a parametric model. It allows dialog authors to improve the dialog manager over time by leveraging user-system dialog logs as training data.
  arXiv  Detail & Related papers  (2020-04-09T00:10:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.