GeDi: Generative Discriminator Guided Sequence Generation
- URL: http://arxiv.org/abs/2009.06367v2
- Date: Thu, 22 Oct 2020 14:14:09 GMT
- Title: GeDi: Generative Discriminator Guided Sequence Generation
- Authors: Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish
Keskar, Shafiq Joty, Richard Socher, Nazneen Fatema Rajani
- Abstract summary: We propose GeDi as an efficient method for using smaller LMs as generative discriminators to guide generation from large LMs.
We find that GeDi gives stronger controllability than the state of the art method while also achieving generation speeds more than 30 times faster.
- Score: 53.15651536569169
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: While large-scale language models (LMs) are able to imitate the distribution
of natural language well enough to generate realistic text, it is difficult to
control which regions of the distribution they generate. This is especially
problematic because datasets used for training large LMs usually contain
significant toxicity, hate, bias, and negativity. We propose GeDi as an
efficient method for using smaller LMs as generative discriminators to guide
generation from large LMs to make them safer and more controllable. GeDi guides
generation at each step by computing classification probabilities for all
possible next tokens via Bayes rule by normalizing over two class-conditional
distributions; one conditioned on the desired attribute, or control code, and
another conditioned on the undesired attribute, or anti control code. We find
that GeDi gives stronger controllability than the state of the art method while
also achieving generation speeds more than 30 times faster. Additionally,
training GeDi on only four topics allows us to controllably generate new topics
zero-shot from just a keyword, unlocking a new capability that previous
controllable generation methods do not have. Lastly, we show that GeDi can make
GPT-2 (1.5B parameters) significantly less toxic without sacrificing linguistic
quality, making it by far the most practical existing method for detoxifying
large language models while maintaining a fast generation speed.
Related papers
- Controlled LLM Decoding via Discrete Auto-regressive Biasing [9.843359827321194]
Controlled text generation allows for enforcing user-defined constraints on large language model outputs.
We propose Discrete Auto-regressive Biasing, a controlled decoding algorithm that leverages gradients while operating entirely in the discrete text domain.
Our method significantly improves constraint satisfaction while maintaining comparable or better fluency, all with even lower computational costs.
arXiv Detail & Related papers (2025-02-06T00:14:43Z) - Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection [49.15148871877941]
Next-token distribution outputs offer a theoretically appealing approach for detection of large language models (LLMs)
We propose the Perplexity Attention Weighted Network (PAWN), which uses the last hidden states of the LLM and positions to weight the sum of a series of features based on metrics from the next-token distribution across the sequence length.
PAWN shows competitive and even better performance in-distribution than the strongest baselines with a fraction of their trainable parameters.
arXiv Detail & Related papers (2025-01-07T17:00:49Z) - Reinforcement Learning with Token-level Feedback for Controllable Text Generation [16.117006822479407]
We propose a novel reinforcement learning algorithm named TOLE which formulates TOken-LEvel rewards for controllable text generation.
Experimental results show that our algorithm can achieve superior performance on both single-attribute and multi-attribute control tasks.
arXiv Detail & Related papers (2024-03-18T08:18:37Z) - Prompt Optimization via Adversarial In-Context Learning [51.18075178593142]
adv-ICL is implemented as a two-player game between a generator and a discriminator.
The generator tries to generate realistic enough output to fool the discriminator.
We show that adv-ICL results in significant improvements over state-of-the-art prompt optimization techniques.
arXiv Detail & Related papers (2023-12-05T09:44:45Z) - KEST: Kernel Distance Based Efficient Self-Training for Improving
Controllable Text Generation [24.47531522553703]
We propose KEST, a novel and efficient self-training framework to handle these problems.
KEST utilizes a kernel-based loss, rather than standard cross entropy, to learn from the soft pseudo text produced by a shared non-autoregressive generator.
Experiments on three controllable generation tasks demonstrate that KEST significantly improves control accuracy while maintaining comparable text fluency and generation diversity against several strong baselines.
arXiv Detail & Related papers (2023-06-17T19:40:57Z) - Extrapolating Multilingual Understanding Models as Multilingual
Generators [82.1355802012414]
This paper explores methods to empower multilingual understanding models the generation abilities to get a unified model.
We propose a textbfSemantic-textbfGuided textbfAlignment-then-Denoising (SGA) approach to adapt an encoder to a multilingual generator with a small number of new parameters.
arXiv Detail & Related papers (2023-05-22T15:33:21Z) - Controlled Text Generation with Natural Language Instructions [74.88938055638636]
InstructCTG is a controlled text generation framework that incorporates different constraints.
We first extract the underlying constraints of natural texts through a combination of off-the-shelf NLP tools and simple verbalizes.
By prepending natural language descriptions of the constraints and a few demonstrations, we fine-tune a pre-trained language model to incorporate various types of constraints.
arXiv Detail & Related papers (2023-04-27T15:56:34Z) - Bridging the Gap Between Training and Inference of Bayesian Controllable
Language Models [58.990214815032495]
Large-scale pre-trained language models have achieved great success on natural language generation tasks.
BCLMs have been shown to be efficient in controllable language generation.
We propose a "Gemini Discriminator" for controllable language generation which alleviates the mismatch problem with a small computational cost.
arXiv Detail & Related papers (2022-06-11T12:52:32Z) - Generating texts under constraint through discriminator-guided MCTS [1.3750624267664153]
We formalize constrained generation as a tree exploration process guided by a discriminator.
Using a discriminator to guide this generation, rather than fine-tuning the LM, allows to apply the constraint more finely and dynamically.
We show that our methods achieves state-of-the-art results in constrained generation, without having to tune the language model.
arXiv Detail & Related papers (2021-09-28T09:29:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.