Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders

• URL: http://arxiv.org/abs/2106.03410v1
• Date: Mon, 7 Jun 2021 08:19:31 GMT
• Title: Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders
• Authors: Bin Sun, Shaoxiong Feng, Yiwei Li, Jiamou Liu, Kan Li
• Abstract summary: Conditional Variational AutoEncoder (CVAE) effectively increases the diversity and informativeness of responses in open-ended dialogue generation tasks. We propose Self-separated Conditional Variational AutoEncoder (abbreviated as SepaCVAE) that introduces group information to regularize the latent variables. SepaCVAE actively divides the input data into groups, and then widens the absolute difference between data pairs from distinct groups.
• Score: 10.910845951559388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conditional Variational AutoEncoder (CVAE) effectively increases the diversity and informativeness of responses in open-ended dialogue generation tasks through enriching the context vector with sampled latent variables. However, due to the inherent one-to-many and many-to-one phenomena in human dialogues, the sampled latent variables may not correctly reflect the contexts' semantics, leading to irrelevant and incoherent generated responses. To resolve this problem, we propose Self-separated Conditional Variational AutoEncoder (abbreviated as SepaCVAE) that introduces group information to regularize the latent variables, which enhances CVAE by improving the responses' relevance and coherence while maintaining their diversity and informativeness. SepaCVAE actively divides the input data into groups, and then widens the absolute difference between data pairs from distinct groups, while narrowing the relative distance between data pairs in the same group. Empirical results from automatic evaluation and detailed analysis demonstrate that SepaCVAE can significantly boost responses in well-established open-domain dialogue datasets.

Related papers

• Long-Span Question-Answering: Automatic Question Generation and QA-System Ranking via Side-by-Side Evaluation [65.16137964758612]
  We explore the use of long-context capabilities in large language models to create synthetic reading comprehension data from entire books. Our objective is to test the capabilities of LLMs to analyze, understand, and reason over problems that require a detailed comprehension of long spans of text.
  arXiv  Detail & Related papers  (2024-05-31T20:15:10Z)
• Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inference [82.28542500317445]
  We present an end-to-end learning framework, termed Sequential Posterior Inference (SPI), capable of selecting knowledge and generating dialogues. Unlike other methods, SPI does not require the inference network or assume a simple geometry of the posterior distribution.
  arXiv  Detail & Related papers  (2023-06-01T21:23:13Z)
• Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information [18.859159491548006]
  We propose a novel learning-based automatic evaluation metric (CMN) for open-domain dialogues. We employ Conditional Variational Autoencoders (CVAEs) with a Next Sentence Prediction (NSP) objective and employing Mutual Information (MI) to model the semantic similarity of text in the latent space. Experimental results on two open-domain dialogue datasets demonstrate the superiority of our method compared with a wide range of baselines.
  arXiv  Detail & Related papers  (2023-05-26T14:21:54Z)
• Towards Diverse, Relevant and Coherent Open-Domain Dialogue Generation via Hybrid Latent Variables [20.66743177460193]
  We combine the merits of both continuous and discrete latent variables and propose a Hybrid Latent Variable (HLV) method. HLV constrains the global semantics of responses through discrete latent variables and enriches responses with continuous latent variables. In addition, we propose Conditional Hybrid Variational Transformer (CHVT) to construct and to utilize HLV with transformers for dialogue generation.
  arXiv  Detail & Related papers  (2022-12-02T12:48:01Z)
• Counterfactual Data Augmentation via Perspective Transition for Open-Domain Dialogues [34.78482218571574]
  We propose a data augmentation method to automatically augment high-quality responses with different semantics by counterfactual inference. Experimental results show that our data augmentation method can augment high-quality responses with different semantics for a given dialogue history, and can outperform competitive baselines on multiple downstream tasks.
  arXiv  Detail & Related papers  (2022-10-30T13:26:49Z)
• Incorporating Casual Analysis into Diversified and Logical Response Generation [14.4586344491264]
  Conditional Variational AutoEncoder (CVAE) model can generate more diversified responses than the traditional Seq2Seq model. We propose to predict the mediators to preserve relevant information and auto-regressively incorporate the mediators into generating process.
  arXiv  Detail & Related papers  (2022-09-20T05:51:11Z)
• DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations [46.942369532632604]
  We propose a Dialogue Evaluation metric that relies on AMR-based semantic manipulations for incoherent data generation. Our experiments show that DEAM achieves higher correlations with human judgments compared to baseline methods.
  arXiv  Detail & Related papers  (2022-03-18T03:11:35Z)
• Learning an Unreferenced Metric for Online Dialogue Evaluation [53.38078951628143]
  We propose an unreferenced automated evaluation metric that uses large pre-trained language models to extract latent representations of utterances. We show that our model achieves higher correlation with human annotations in an online setting, while not requiring true responses for comparison during inference.
  arXiv  Detail & Related papers  (2020-05-01T20:01:39Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)
• On the Encoder-Decoder Incompatibility in Variational Text Modeling and Beyond [82.18770740564642]
  Variational autoencoders (VAEs) combine latent variables with amortized variational inference. We observe the encoder-decoder incompatibility that leads to poor parameterizations of the data manifold. We propose Coupled-VAE, which couples a VAE model with a deterministic autoencoder with the same structure.
  arXiv  Detail & Related papers  (2020-04-20T10:34:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.