FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training

• URL: http://arxiv.org/abs/2210.03167v1
• Date: Thu, 6 Oct 2022 19:00:51 GMT
• Title: FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training
• Authors: Junyi Chai, Reid Pryzant, Victor Ye Dong, Konstantin Golobokov, Chenguang Zhu, Yi Liu
• Abstract summary: Controllable text generation systems often leverage control codes to direct various properties of the output like style and length. Inspired by recent work on causal inference for NLP, this paper reveals a previously overlooked flaw in these control code-based conditional text generation algorithms. We propose two simple techniques to reduce these correlations in training sets.
• Score: 25.75982440355576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Controllable text generation systems often leverage control codes to direct various properties of the output like style and length. Inspired by recent work on causal inference for NLP, this paper reveals a previously overlooked flaw in these control code-based conditional text generation algorithms. Spurious correlations in the training data can lead models to incorrectly rely on parts of the input other than the control code for attribute selection, significantly undermining downstream generation quality and controllability. We demonstrate the severity of this issue with a series of case studies and then propose two simple techniques to reduce these correlations in training sets. The first technique is based on resampling the data according to an example's propensity towards each linguistic attribute (IPS). The second produces multiple counterfactual versions of each example and then uses an additional feedback mechanism to remove noisy examples (feedback aware self-training, FAST). We evaluate on 3 tasks -- news headline, meta review, and search ads generation -- and demonstrate that FAST can significantly improve the controllability and language quality of generated outputs when compared to state-of-the-art controllable text generation approaches.

Related papers

• Improving Sampling Methods for Fine-tuning SentenceBERT in Text Streams [49.3179290313959]
  This study explores the efficacy of seven text sampling methods designed to selectively fine-tune language models. We precisely assess the impact of these methods on fine-tuning the SBERT model using four different loss functions. Our findings indicate that Softmax loss and Batch All Triplets loss are particularly effective for text stream classification.
  arXiv  Detail & Related papers  (2024-03-18T23:41:52Z)
• Reinforcement Learning with Token-level Feedback for Controllable Text Generation [16.117006822479407]
  We propose a novel reinforcement learning algorithm named TOLE which formulates TOken-LEvel rewards for controllable text generation. Experimental results show that our algorithm can achieve superior performance on both single-attribute and multi-attribute control tasks.
  arXiv  Detail & Related papers  (2024-03-18T08:18:37Z)
• LiFi: Lightweight Controlled Text Generation with Fine-Grained Control Codes [46.74968005604948]
  We present LIFI, which offers a lightweight approach with fine-grained control for controlled text generation. We evaluate LIFI on two conventional tasks -- sentiment control and topic control -- and one newly proposed task -- stylistic novel writing.
  arXiv  Detail & Related papers  (2024-02-10T11:53:48Z)
• Text2Data: Low-Resource Data Generation with Textual Control [104.38011760992637]
  Natural language serves as a common and straightforward control signal for humans to interact seamlessly with machines. We propose Text2Data, a novel approach that utilizes unlabeled data to understand the underlying data distribution through an unsupervised diffusion model. It undergoes controllable finetuning via a novel constraint optimization-based learning objective that ensures controllability and effectively counteracts catastrophic forgetting.
  arXiv  Detail & Related papers  (2024-02-08T03:41:39Z)
• Air-Decoding: Attribute Distribution Reconstruction for Decoding-Time Controllable Text Generation [58.911255139171075]
  Controllable text generation (CTG) aims to generate text with desired attributes. We propose a novel lightweight decoding framework named Air-Decoding. Our method achieves a new state-of-the-art control performance.
  arXiv  Detail & Related papers  (2023-10-23T12:59:11Z)
• Controllable Data Augmentation for Few-Shot Text Mining with Chain-of-Thought Attribute Manipulation [35.33340453046864]
  Chain-of-Thought Attribute Manipulation (CoTAM) is a novel approach that generates new data from existing examples. We leverage the chain-of-thought prompting to directly edit the text in three steps, (1) attribute decomposition, (2) manipulation proposal, and (3) sentence reconstruction.
  arXiv  Detail & Related papers  (2023-07-14T00:10:03Z)
• KEST: Kernel Distance Based Efficient Self-Training for Improving Controllable Text Generation [24.47531522553703]
  We propose KEST, a novel and efficient self-training framework to handle these problems. KEST utilizes a kernel-based loss, rather than standard cross entropy, to learn from the soft pseudo text produced by a shared non-autoregressive generator. Experiments on three controllable generation tasks demonstrate that KEST significantly improves control accuracy while maintaining comparable text fluency and generation diversity against several strong baselines.
  arXiv  Detail & Related papers  (2023-06-17T19:40:57Z)
• STA: Self-controlled Text Augmentation for Improving Text Classifications [2.9669250132689164]
  A number of text augmentation techniques have emerged in the field of Natural Language Processing (NLP) We introduce a state-of-the-art approach for Self-Controlled Text Augmentation (STA) Our approach tightly controls the generation process by introducing a self-checking procedure to ensure that generated examples retain the semantic content of the original text.
  arXiv  Detail & Related papers  (2023-02-24T17:54:12Z)
• Classifiers are Better Experts for Controllable Text Generation [63.17266060165098]
  We show that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and sentiment accuracy based on the external classifier of generated texts. The same time, it is also easier to implement and tune, and has significantly fewer restrictions and requirements.
  arXiv  Detail & Related papers  (2022-05-15T12:58:35Z)
• Controllable Text Generation with Focused Variation [71.07811310799664]
  Focused-Variation Network (FVN) is a novel model to control language generation. FVN learns disjoint discrete latent spaces for each attribute inside codebooks, which allows for both controllability and diversity. We evaluate FVN on two text generation datasets with annotated content and style, and show state-of-the-art performance as assessed by automatic and human evaluations.
  arXiv  Detail & Related papers  (2020-09-25T06:31:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.