Progressive Open-Domain Response Generation with Multiple Controllable Attributes

• URL: http://arxiv.org/abs/2106.14614v1
• Date: Mon, 7 Jun 2021 08:48:39 GMT
• Title: Progressive Open-Domain Response Generation with Multiple Controllable Attributes
• Authors: Haiqin Yang, Xiaoyuan Yao, Yiqun Duan, Jianping Shen, Jie Zhong, Kun Zhang
• Abstract summary: We propose a Progressively trained Hierarchical Vari-Decoder (PHED) to tackle this task. PHED deploys Conditional AutoEncoder (CVAE) on Transformer to include one aspect of attributes at one stage. PHED significantly outperforms the state-of-the-art neural generation models and produces more diverse responses as expected.
• Score: 13.599621571488033
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: It is desirable to include more controllable attributes to enhance the diversity of generated responses in open-domain dialogue systems. However, existing methods can generate responses with only one controllable attribute or lack a flexible way to generate them with multiple controllable attributes. In this paper, we propose a Progressively trained Hierarchical Encoder-Decoder (PHED) to tackle this task. More specifically, PHED deploys Conditional Variational AutoEncoder (CVAE) on Transformer to include one aspect of attributes at one stage. A vital characteristic of the CVAE is to separate the latent variables at each stage into two types: a global variable capturing the common semantic features and a specific variable absorbing the attribute information at that stage. PHED then couples the CVAE latent variables with the Transformer encoder and is trained by minimizing a newly derived ELBO and controlled losses to produce the next stage's input and produce responses as required. Finally, we conduct extensive evaluations to show that PHED significantly outperforms the state-of-the-art neural generation models and produces more diverse responses as expected.

Related papers

• Multi-Attribute Constraint Satisfaction via Language Model Rewriting [67.5778646504987]
  Multi-Attribute Constraint Satisfaction (MACS) is a method capable of finetuning language models to satisfy user-specified constraints on multiple external real-value attributes. Our work opens new avenues for generalized and real-value multi-attribute control, with implications for diverse applications spanning NLP and bioinformatics.
  arXiv  Detail & Related papers  (2024-12-26T12:36:39Z)
• Generating Features with Increased Crop-related Diversity for Few-Shot Object Detection [35.652092907690694]
  Two-stage object detectors generate object proposals and classify them to detect objects in images. Proposals often do not contain the objects perfectly but overlap with them in many possible ways. We propose a novel variational autoencoder based data generation model, which is capable of generating data with increased crop-related diversity.
  arXiv  Detail & Related papers  (2023-04-11T09:47:21Z)
• String-based Molecule Generation via Multi-decoder VAE [56.465033997245776]
  We investigate the problem of string-based molecular generation via variational autoencoders (VAEs) We propose a simple, yet effective idea to improve the performance of VAE for the task. In our experiments, the proposed VAE model particularly performs well for generating a sample from out-of-domain distribution.
  arXiv  Detail & Related papers  (2022-08-23T03:56:30Z)
• Controllable Dialogue Generation with Disentangled Multi-grained Style Specification and Attribute Consistency Reward [47.96949534259019]
  We propose a controllable dialogue generation model to steer response generation under multi-attribute constraints. We categorize the commonly used control attributes into global and local ones, which possess different granularities of effects on response generation. Our model can significantly outperform competitive baselines in terms of response quality, content diversity and controllability.
  arXiv  Detail & Related papers  (2021-09-14T14:29:38Z)
• Is Disentanglement enough? On Latent Representations for Controllable Music Generation [78.8942067357231]
  In the absence of a strong generative decoder, disentanglement does not necessarily imply controllability. The structure of the latent space with respect to the VAE-decoder plays an important role in boosting the ability of a generative model to manipulate different attributes.
  arXiv  Detail & Related papers  (2021-08-01T18:37:43Z)
• Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learning [19.854565192491123]
  We present a novel multi-modal data augmentation approach named Dizygotic Conditional Variational AutoEncoder (DCVAE) DCVAE conducts feature synthesis via pairing two Conditional Variational AutoEncoders (CVAEs) with the same seed but different modality conditions in a dizygotic symbiosis manner. The generated features of two CVAEs are adaptively combined to yield the final feature, which can be converted back into its paired conditions.
  arXiv  Detail & Related papers  (2021-06-28T08:29:55Z)
• Transformer-based Conditional Variational Autoencoder for Controllable Story Generation [39.577220559911055]
  We investigate large-scale latent variable models (LVMs) for neural story generation with objectives in two threads: generation effectiveness and controllability. We advocate to revive latent variable modeling, essentially the power of representation learning, in the era of Transformers. Specifically, we integrate latent representation vectors with a Transformer-based pre-trained architecture to build conditional variational autoencoder (CVAE)
  arXiv  Detail & Related papers  (2021-01-04T08:31:11Z)
• Relaxed-Responsibility Hierarchical Discrete VAEs [3.976291254896486]
  We introduce textitRelaxed-Responsibility Vector-Quantisation, a novel way to parameterise discrete latent variables. We achieve state-of-the-art bits-per-dim results for various standard datasets.
  arXiv  Detail & Related papers  (2020-07-14T19:10:05Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)
• Variational Transformers for Diverse Response Generation [71.53159402053392]
  Variational Transformer (VT) is a variational self-attentive feed-forward sequence model. VT combines the parallelizability and global receptive field computation of the Transformer with the variational nature of the CVAE. We explore two types of VT: 1) modeling the discourse-level diversity with a global latent variable; and 2) augmenting the Transformer decoder with a sequence of finegrained latent variables.
  arXiv  Detail & Related papers  (2020-03-28T07:48:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.