Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation

• URL: http://arxiv.org/abs/2004.14983v2
• Date: Fri, 2 Oct 2020 12:23:16 GMT
• Title: Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation
• Authors: Giuseppe Russo, Nora Hollenstein, Claudiu Musat, Ce Zhang
• Abstract summary: We introduce CGA, a conditional VAE architecture, to control, generate, and augment text. We show the value of the individual model components in an ablation study. We show high quality, diversity and attribute control in the generated sentences through a series of automatic and human assessments.
• Score: 22.70189685469752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce CGA, a conditional VAE architecture, to control, generate, and augment text. CGA is able to generate natural English sentences controlling multiple semantic and syntactic attributes by combining adversarial learning with a context-aware loss and a cyclical word dropout routine. We demonstrate the value of the individual model components in an ablation study. The scalability of our approach is ensured through a single discriminator, independently of the number of attributes. We show high quality, diversity and attribute control in the generated sentences through a series of automatic and human assessments. As the main application of our work, we test the potential of this new NLG model in a data augmentation scenario. In a downstream NLP task, the sentences generated by our CGA model show significant improvements over a strong baseline, and a classification performance often comparable to adding same amount of additional real data.

Related papers

• Contextualization Distillation from Large Language Model for Knowledge Graph Completion [51.126166442122546]
  We introduce the Contextualization Distillation strategy, a plug-in-and-play approach compatible with both discriminative and generative KGC frameworks. Our method begins by instructing large language models to transform compact, structural triplets into context-rich segments. Comprehensive evaluations across diverse datasets and KGC techniques highlight the efficacy and adaptability of our approach.
  arXiv  Detail & Related papers  (2024-01-28T08:56:49Z)
• UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning [86.91893533388628]
  This paper presents UniDiff, a unified multi-modal model that integrates image-text contrastive learning (ITC), text-conditioned image synthesis learning (IS), and reciprocal semantic consistency modeling (RSC) UniDiff demonstrates versatility in both multi-modal understanding and generative tasks.
  arXiv  Detail & Related papers  (2023-06-01T15:39:38Z)
• DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation [6.844825905212349]
  We propose a new CTG approach, namely DisCup, which incorporates the attribute knowledge of discriminator to optimize the control-prompts. DisCup can achieve a new state-of-the-art control performance while maintaining an efficient and high-quality text generation, only relying on around 10 virtual tokens.
  arXiv  Detail & Related papers  (2022-10-18T02:59:06Z)
• FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training [25.75982440355576]
  Controllable text generation systems often leverage control codes to direct various properties of the output like style and length. Inspired by recent work on causal inference for NLP, this paper reveals a previously overlooked flaw in these control code-based conditional text generation algorithms. We propose two simple techniques to reduce these correlations in training sets.
  arXiv  Detail & Related papers  (2022-10-06T19:00:51Z)
• To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP [0.0]
  We investigate three categories of text augmentation methodologies which perform changes on the syntax. We compare them on part-of-speech tagging, dependency parsing and semantic role labeling for a diverse set of language families. Our results suggest that the augmentation techniques can further improve over strong baselines based on mBERT.
  arXiv  Detail & Related papers  (2021-11-18T10:52:48Z)
• Incorporating Linguistic Knowledge for Abstractive Multi-document Summarization [20.572283625521784]
  We develop a neural network based abstractive multi-document summarization (MDS) model. We process the dependency information into the linguistic-guided attention mechanism. With the help of linguistic signals, sentence-level relations can be correctly captured.
  arXiv  Detail & Related papers  (2021-09-23T08:13:35Z)
• Is Disentanglement enough? On Latent Representations for Controllable Music Generation [78.8942067357231]
  In the absence of a strong generative decoder, disentanglement does not necessarily imply controllability. The structure of the latent space with respect to the VAE-decoder plays an important role in boosting the ability of a generative model to manipulate different attributes.
  arXiv  Detail & Related papers  (2021-08-01T18:37:43Z)
• SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
  We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
  arXiv  Detail & Related papers  (2021-01-02T01:15:57Z)
• CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation [20.27052525082402]
  We present a Controlled Adversarial Text Generation (CAT-Gen) model that generates adversarial texts through controllable attributes. Experiments on real-world NLP datasets demonstrate that our method can generate more diverse and fluent adversarial texts.
  arXiv  Detail & Related papers  (2020-10-05T21:07:45Z)
• Contextualized Perturbation for Textual Adversarial Attack [56.370304308573274]
  Adversarial examples expose the vulnerabilities of natural language processing (NLP) models. This paper presents CLARE, a ContextuaLized AdversaRial Example generation model that produces fluent and grammatical outputs.
  arXiv  Detail & Related papers  (2020-09-16T06:53:15Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.