Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation

• URL: http://arxiv.org/abs/2204.13362v1
• Date: Thu, 28 Apr 2022 09:09:45 GMT
• Title: Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation
• Authors: Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Mingfeng Xue, Boxing Chen, Jun Xie
• Abstract summary: Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes. We propose Tailor, which represents each attribute as a pre-trained continuous vector (i.e., single-attribute prompt) and guides the generation of a fixed PLM switch to a pre-specified attribute. Experiments on 11 attribute-specific generation tasks demonstrate strong performances of Tailor on both single-attribute and multi-attribute CTG, with 0.08% training parameters of a GPT-2.
• Score: 47.09041767447308
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes (e.g., emotions and topics). Existing works often utilize fine-tuning or resort to extra attribute classifiers, yet suffer from storage and inference time increases. To address these concerns, we explore attribute-based CTG in a prompt-based manner. In short, the proposed Tailor represents each attribute as a pre-trained continuous vector (i.e., single-attribute prompt) and guides the generation of a fixed PLM switch to a pre-specified attribute. We experimentally find that these prompts can be simply concatenated as a whole to multi-attribute CTG without any re-training, yet raises problems of fluency decrease and position sensitivity. To this end, Tailor provides a multi-attribute prompt mask and a re-indexing position-ids sequence to bridge the gap between the training (one prompt for each task) and testing stage (concatenating more than one prompt). To further enhance such single-attribute prompt combinations, Tailor also introduces a trainable prompt connector, which can be concatenated with any two single-attribute prompts to multi-attribute text generation. Experiments on 11 attribute-specific generation tasks demonstrate strong performances of Tailor on both single-attribute and multi-attribute CTG, with 0.08\% training parameters of a GPT-2.

Related papers

• SEP: Self-Enhanced Prompt Tuning for Visual-Language Model [68.68025991850115]
  We introduce a novel approach named Self-Enhanced Prompt Tuning (SEP) SEP explicitly incorporates discriminative prior knowledge to enhance both textual-level and visual-level embeddings. Comprehensive evaluations across various benchmarks and tasks confirm SEP's efficacy in prompt tuning.
  arXiv  Detail & Related papers  (2024-05-24T13:35:56Z)
• Multi-Prompts Learning with Cross-Modal Alignment for Attribute-based Person Re-Identification [18.01407937934588]
  We present a new framework called Multi-Prompts ReID (MP-ReID) based on prompt learning and language models. MP-ReID learns to hallucinate diverse, informative, and promptable sentences for describing the query images. Explicit prompts are obtained by ensembling generation models, such as ChatGPT and VQA models.
  arXiv  Detail & Related papers  (2023-12-28T03:00:19Z)
• Air-Decoding: Attribute Distribution Reconstruction for Decoding-Time Controllable Text Generation [58.911255139171075]
  Controllable text generation (CTG) aims to generate text with desired attributes. We propose a novel lightweight decoding framework named Air-Decoding. Our method achieves a new state-of-the-art control performance.
  arXiv  Detail & Related papers  (2023-10-23T12:59:11Z)
• MACSum: Controllable Summarization with Mixed Attributes [56.685735509260276]
  MACSum is the first human-annotated summarization dataset for controlling mixed attributes. We propose two simple and effective parameter-efficient approaches for the new task of mixed controllable summarization.
  arXiv  Detail & Related papers  (2022-11-09T17:17:37Z)
• DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation [6.844825905212349]
  We propose a new CTG approach, namely DisCup, which incorporates the attribute knowledge of discriminator to optimize the control-prompts. DisCup can achieve a new state-of-the-art control performance while maintaining an efficient and high-quality text generation, only relying on around 10 virtual tokens.
  arXiv  Detail & Related papers  (2022-10-18T02:59:06Z)
• Composable Text Controls in Latent Space with ODEs [97.12426987887021]
  This paper proposes a new efficient approach for composable text operations in the compact latent space of text. By connecting pretrained LMs to the latent space through efficient adaption, we then decode the sampled vectors into desired text sequences. Experiments show that composing those operators within our approach manages to generate or edit high-quality text.
  arXiv  Detail & Related papers  (2022-08-01T06:51:45Z)
• Controllable Dialogue Generation with Disentangled Multi-grained Style Specification and Attribute Consistency Reward [47.96949534259019]
  We propose a controllable dialogue generation model to steer response generation under multi-attribute constraints. We categorize the commonly used control attributes into global and local ones, which possess different granularities of effects on response generation. Our model can significantly outperform competitive baselines in terms of response quality, content diversity and controllability.
  arXiv  Detail & Related papers  (2021-09-14T14:29:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.