Harnessing the Plug-and-Play Controller by Prompting
- URL: http://arxiv.org/abs/2402.04160v1
- Date: Tue, 6 Feb 2024 17:18:25 GMT
- Title: Harnessing the Plug-and-Play Controller by Prompting
- Authors: Hao Wang, Lei Sha
- Abstract summary: This paper introduces a novel method for flexible attribute control in text generation using pre-trained language models (PLMs)
The proposed approach aims to enhance the fluency of generated text by guiding the generation process with PPCs.
- Score: 12.705251690623495
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Controllable text generation is a growing field within natural language
generation (NLG) that focuses on producing text that meets specific constraints
in real-world applications. Previous approaches, such as plug-and-play
controllers (PPCs), aimed to steer the properties of generated text in a
flexible manner. However, these methods often compromised the integrity of the
language model's decoding process, resulting in less smooth text generation.
Alternatively, other techniques utilized multiple attribute prompts to align
the generated text with desired attributes, but this approach required prompt
design for each attribute and was dependent on the size of the language model.
This paper introduces a novel method for flexible attribute control in text
generation using pre-trained language models (PLMs). The proposed approach aims
to enhance the fluency of generated text by guiding the generation process with
PPCs. The key idea is to dynamically adjust the distribution of generated text
by modifying prompts, effectively constraining the output space of the language
model and influencing the desired attribute. To enable smooth cooperation
between the PLM and the PPC, our work innovatively proposes a new model
fine-tuning method: Reinforcement Learning with Dynamic Adjust Feedback
(RLDAF).This fine-tuning process adapts a small subset of the language model's
parameters based on the generating actions taken during the PPC control
process. The resulting harmonious collaboration between the PLM and PPC leads
to improved smoothness in text generation during inference. Extensive
experiments were conducted on the SST2 dataset, and the proposed method
outperformed previous approaches in various evaluation metrics, including text
fluency and attribute consistency.
Related papers
- Enhancing Text Generation in Joint NLG/NLU Learning Through Curriculum Learning, Semi-Supervised Training, and Advanced Optimization Techniques [0.0]
This research paper developed a novel approach to improve text generation in the context of joint Natural Language Generation (NLG) and Natural Language Understanding (NLU) learning.
The data is prepared by gathering and preprocessing annotated datasets, including cleaning, tokenization, stemming, and stop-word removal.
Transformer-based encoders and decoders, capturing long range dependencies and improving source-target sequence modelling.
Reinforcement learning with policy gradient techniques, semi-supervised training, improved attention mechanisms, and differentiable approximations are employed to fine-tune the models and handle complex linguistic tasks effectively.
arXiv Detail & Related papers (2024-10-17T12:43:49Z) - Successor Features for Efficient Multisubject Controlled Text Generation [48.37713738712319]
We introduce SF-GEN, which is grounded in two primary concepts: successor features (SFs) and language model rectification.
SF-GEN seamlessly integrates the two to enable dynamic steering of text generation with no need to alter the LLM's parameters.
To the best of our knowledge, our research represents the first application of successor features in text generation.
arXiv Detail & Related papers (2023-11-03T00:17:08Z) - Composable Text Controls in Latent Space with ODEs [97.12426987887021]
This paper proposes a new efficient approach for composable text operations in the compact latent space of text.
By connecting pretrained LMs to the latent space through efficient adaption, we then decode the sampled vectors into desired text sequences.
Experiments show that composing those operators within our approach manages to generate or edit high-quality text.
arXiv Detail & Related papers (2022-08-01T06:51:45Z) - Text Revision by On-the-Fly Representation Optimization [76.11035270753757]
Current state-of-the-art methods formulate these tasks as sequence-to-sequence learning problems.
We present an iterative in-place editing approach for text revision, which requires no parallel data.
It achieves competitive and even better performance than state-of-the-art supervised methods on text simplification.
arXiv Detail & Related papers (2022-04-15T07:38:08Z) - Control Prefixes for Text Generation [17.682443394199375]
We propose a dynamic method, Control Prefixes, which allows for the inclusion of conditional input-dependent information in each prompt.
We present state-of-the-art results on several data-to-text datasets, including WebNLG.
arXiv Detail & Related papers (2021-10-15T19:32:17Z) - Attribute Alignment: Controlling Text Generation from Pre-trained
Language Models [46.19190007510232]
We propose a simple and flexible method for controlling text generation by aligning disentangled attribute representations.
In contrast to recent efforts on training a discriminator to perturb the token level distribution for an attribute, we use the same data to learn an alignment function to guide the pre-trained, non-controlled language model to generate texts with the target attribute without changing the original language model parameters.
arXiv Detail & Related papers (2021-03-20T01:51:32Z) - SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation.
Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
arXiv Detail & Related papers (2021-01-02T01:15:57Z) - Progressive Generation of Long Text with Pretrained Language Models [83.62523163717448]
Large-scale language models (LMs) pretrained on massive corpora of text, such as GPT-2, are powerful open-domain text generators.
It is still challenging for such models to generate coherent long passages of text, especially when the models are fine-tuned to the target domain on a small corpus.
We propose a simple but effective method of generating text in a progressive manner, inspired by generating images from low to high resolution.
arXiv Detail & Related papers (2020-06-28T21:23:05Z) - POINTER: Constrained Progressive Text Generation via Insertion-based
Generative Pre-training [93.79766670391618]
We present POINTER, a novel insertion-based approach for hard-constrained text generation.
The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner.
The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
arXiv Detail & Related papers (2020-05-01T18:11:54Z) - Syntax-driven Iterative Expansion Language Models for Controllable Text
Generation [2.578242050187029]
We propose a new paradigm for introducing a syntactic inductive bias into neural text generation.
Our experiments show that this paradigm is effective at text generation, with quality between LSTMs and Transformers, and comparable diversity.
arXiv Detail & Related papers (2020-04-05T14:29:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.