Prompt-Based Length Controlled Generation with Multiple Control Types

• URL: http://arxiv.org/abs/2406.10278v1
• Date: Wed, 12 Jun 2024 01:49:54 GMT
• Title: Prompt-Based Length Controlled Generation with Multiple Control Types
• Authors: Renlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang, Qun Liu,
• Abstract summary: We propose a prompt-based method to achieve length controlled generation under different control types with high accuracy. In particular, we adopt reinforcement learning (RL) and sample filtering with the reward signal given by rule-based reward models. Experiments show that our method significantly improves the accuracy of prompt-based length control on popular summarization datasets like CNNDM and NYT.
• Score: 45.202705040391734
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have attracted great attention given their strong performance on a wide range of NLP tasks. In practice, users often expect generated texts to fall within a specific length range, making length controlled generation an important topic, especially for GPT-style models. Existing length control methods mostly focus on a simple control type of "equal to" a target length. Different from them, we propose a prompt-based method to achieve length controlled generation under different control types with high accuracy. In particular, we adopt reinforcement learning (RL) and sample filtering with the reward signal given by rule-based reward models, which enhances the length control ability of models by rewarding outputs that follow certain control instructions. In addition, we introduce a standard prompt extractor to parse arbitrary users' input into standard control instructions. Experiments show that our method significantly improves the accuracy of prompt-based length control on popular summarization datasets like CNNDM and NYT under multiple control types. Moreover, both the standard prompt extractor and RL-tuned model show strong generalization to unseen control prompt templates.

Related papers

• Ruler: A Model-Agnostic Method to Control Generated Length for Large Language Models [14.175953642749649]
  Large language models often struggle to generate responses of a specific length. We introduce a novel, model-agnostic approach called Ruler to enhance the instruction-following ability of large language models under length-constrained instructions.
  arXiv  Detail & Related papers  (2024-09-27T17:44:58Z)
• LiFi: Lightweight Controlled Text Generation with Fine-Grained Control Codes [46.74968005604948]
  We present LIFI, which offers a lightweight approach with fine-grained control for controlled text generation. We evaluate LIFI on two conventional tasks -- sentiment control and topic control -- and one newly proposed task -- stylistic novel writing.
  arXiv  Detail & Related papers  (2024-02-10T11:53:48Z)
• Fine-grained Controllable Video Generation via Object Appearance and Context [74.23066823064575]
  We propose fine-grained controllable video generation (FACTOR) to achieve detailed control. FACTOR aims to control objects' appearances and context, including their location and category. Our method achieves controllability of object appearances without finetuning, which reduces the per-subject optimization efforts for the users.
  arXiv  Detail & Related papers  (2023-12-05T17:47:33Z)
• Prompt-Based Length Controlled Generation with Reinforcement Learning [48.49553921757085]
  We propose a prompt-based length control method to achieve high-accuracy length controlled generation. We adopt reinforcement learning with the reward signal given by either trainable or rule-based reward models. Our method significantly improves the accuracy of prompt-based length control for summarization task on popular datasets like CNNDM and NYT.
  arXiv  Detail & Related papers  (2023-08-23T09:43:10Z)
• Latent Prompt Tuning for Text Summarization [95.85520030785139]
  We propose Lotus (shorthand for Latent Prompt Tuning for Summarization), which is a single model that can be applied in both controlled and uncontrolled modes. During training, Lotus learns latent prompt representations from prompts with gold control signals using a contrastive learning objective. Experiments show Lotus in uncontrolled mode consistently improves upon strong (uncontrollable) summarization models across four different summarization datasets.
  arXiv  Detail & Related papers  (2022-11-03T14:18:48Z)
• Posterior Control of Blackbox Generation [126.33511630879713]
  We consider augmenting neural generation models with discrete control states learned through a structured latent-variable approach. We find that this method improves over standard benchmarks, while also providing fine-grained control.
  arXiv  Detail & Related papers  (2020-05-10T03:22:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.