Controllable Text Generation with Focused Variation

• URL: http://arxiv.org/abs/2009.12046v1
• Date: Fri, 25 Sep 2020 06:31:06 GMT
• Title: Controllable Text Generation with Focused Variation
• Authors: Lei Shu, Alexandros Papangelis, Yi-Chia Wang, Gokhan Tur, Hu Xu, Zhaleh Feizollahi, Bing Liu, Piero Molino
• Abstract summary: Focused-Variation Network (FVN) is a novel model to control language generation. FVN learns disjoint discrete latent spaces for each attribute inside codebooks, which allows for both controllability and diversity. We evaluate FVN on two text generation datasets with annotated content and style, and show state-of-the-art performance as assessed by automatic and human evaluations.
• Score: 71.07811310799664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work introduces Focused-Variation Network (FVN), a novel model to control language generation. The main problems in previous controlled language generation models range from the difficulty of generating text according to the given attributes, to the lack of diversity of the generated texts. FVN addresses these issues by learning disjoint discrete latent spaces for each attribute inside codebooks, which allows for both controllability and diversity, while at the same time generating fluent text. We evaluate FVN on two text generation datasets with annotated content and style, and show state-of-the-art performance as assessed by automatic and human evaluations.

Related papers

• Detecting Machine-Generated Long-Form Content with Latent-Space Variables [54.07946647012579]
  Existing zero-shot detectors primarily focus on token-level distributions, which are vulnerable to real-world domain shifts. We propose a more robust method that incorporates abstract elements, such as event transitions, as key deciding factors to detect machine versus human texts.
  arXiv  Detail & Related papers  (2024-10-04T18:42:09Z)
• Exploring Precision and Recall to assess the quality and diversity of LLMs [82.21278402856079]
  We introduce a novel evaluation framework for Large Language Models (LLMs) such as textscLlama-2 and textscMistral. This approach allows for a nuanced assessment of the quality and diversity of generated text without the need for aligned corpora.
  arXiv  Detail & Related papers  (2024-02-16T13:53:26Z)
• Successor Features for Efficient Multisubject Controlled Text Generation [48.37713738712319]
  We introduce SF-GEN, which is grounded in two primary concepts: successor features (SFs) and language model rectification. SF-GEN seamlessly integrates the two to enable dynamic steering of text generation with no need to alter the LLM's parameters. To the best of our knowledge, our research represents the first application of successor features in text generation.
  arXiv  Detail & Related papers  (2023-11-03T00:17:08Z)
• MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space [110.85888003111653]
  Multi-aspect controllable text generation aims to generate fluent sentences that possess multiple desired attributes simultaneously. We introduce a novel approach for multi-aspect control, namely MacLaSa, that estimates compact latent space for multiple aspects. We show that MacLaSa outperforms several strong baselines on attribute relevance and textual quality while maintaining a high inference speed.
  arXiv  Detail & Related papers  (2023-05-22T07:30:35Z)
• DuNST: Dual Noisy Self Training for Semi-Supervised Controllable Text Generation [34.49194429157166]
  Self-training (ST) has prospered again in language understanding by augmenting the fine-tuning of pre-trained language models when labeled data is insufficient. It remains challenging to incorporate ST into attribute-controllable language generation.
  arXiv  Detail & Related papers  (2022-12-16T21:44:34Z)
• An Overview on Controllable Text Generation via Variational Auto-Encoders [15.97186478109836]
  Recent advances in neural-based generative modeling have reignited the hopes of having computer systems capable of conversing with humans. Latent variable models (LVM) such as variational auto-encoders (VAEs) are designed to characterize the distributional pattern of textual data. This overview gives an introduction to existing generation schemes, problems associated with text variational auto-encoders, and a review of several applications about the controllable generation.
  arXiv  Detail & Related papers  (2022-11-15T07:36:11Z)
• FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training [25.75982440355576]
  Controllable text generation systems often leverage control codes to direct various properties of the output like style and length. Inspired by recent work on causal inference for NLP, this paper reveals a previously overlooked flaw in these control code-based conditional text generation algorithms. We propose two simple techniques to reduce these correlations in training sets.
  arXiv  Detail & Related papers  (2022-10-06T19:00:51Z)
• Attribute Alignment: Controlling Text Generation from Pre-trained Language Models [46.19190007510232]
  We propose a simple and flexible method for controlling text generation by aligning disentangled attribute representations. In contrast to recent efforts on training a discriminator to perturb the token level distribution for an attribute, we use the same data to learn an alignment function to guide the pre-trained, non-controlled language model to generate texts with the target attribute without changing the original language model parameters.
  arXiv  Detail & Related papers  (2021-03-20T01:51:32Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.