Efficient Guided Generation for Large Language Models

• URL: http://arxiv.org/abs/2307.09702v4
• Date: Sat, 19 Aug 2023 21:27:51 GMT
• Title: Efficient Guided Generation for Large Language Models
• Authors: Brandon T. Willard and R\'emi Louf
• Abstract summary: We show how the problem of neural text generation can be constructively reformulated in terms of transitions between the states of a finite-state machine. This framework leads to an efficient approach to guiding text generation with regular expressions and context-free grammars.
• Score: 0.21485350418225244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this article we show how the problem of neural text generation can be constructively reformulated in terms of transitions between the states of a finite-state machine. This framework leads to an efficient approach to guiding text generation with regular expressions and context-free grammars by allowing the construction of an index over a language model's vocabulary. The approach is model agnostic, allows one to enforce domain-specific knowledge and constraints, and enables the construction of reliable interfaces by guaranteeing the structure of the generated text. It adds little overhead to the token sequence generation process and significantly outperforms existing solutions. An implementation is provided in the open source Python library Outlines

Related papers

• XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models [3.9417976759908573]
  Context-free grammar is a flexible approach to enable structured generation via constrained decoding. XGrammar is a flexible and efficient structure generation engine for large language models. XGrammar can achieve up to 100x speedup over existing solutions.
  arXiv  Detail & Related papers  (2024-11-22T18:01:37Z)
• Retrieval is Accurate Generation [99.24267226311157]
  We introduce a novel method that selects context-aware phrases from a collection of supporting documents. Our model achieves the best performance and the lowest latency among several retrieval-augmented baselines.
  arXiv  Detail & Related papers  (2024-02-27T14:16:19Z)
• Instruct-SCTG: Guiding Sequential Controlled Text Generation through Instructions [42.67608830386934]
  Instruct-SCTG is a sequential framework that harnesses instruction-tuned language models to generate structurally coherent text. Our framework generates articles in a section-by-section manner, aligned with the desired human structure using natural language instructions.
  arXiv  Detail & Related papers  (2023-12-19T16:20:49Z)
• Text-Blueprint: An Interactive Platform for Plan-based Conditional Generation [84.95981645040281]
  Planning can be a useful intermediate step to render conditional generation less opaque and more grounded. We present a web browser-based demonstration for query-focused summarization that uses a sequence of question-answer pairs.
  arXiv  Detail & Related papers  (2023-04-28T18:14:48Z)
• The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding [65.34601470417967]
  We describe a hybrid architecture for dialogue response generation that combines the strengths of neural language modeling and rule-based generation. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.
  arXiv  Detail & Related papers  (2022-09-16T09:00:49Z)
• Classifiers are Better Experts for Controllable Text Generation [63.17266060165098]
  We show that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and sentiment accuracy based on the external classifier of generated texts. The same time, it is also easier to implement and tune, and has significantly fewer restrictions and requirements.
  arXiv  Detail & Related papers  (2022-05-15T12:58:35Z)
• On Efficient Training, Controllability and Compositional Generalization of Insertion-based Language Generators [18.98725770517241]
  InsNet is an insertion-based sequence model that can be trained as efficiently as transformer decoders. We evaluate InsNet on story generation and CleVR-CoGENT captioning.
  arXiv  Detail & Related papers  (2021-02-12T11:05:02Z)
• Breaking Writer's Block: Low-cost Fine-tuning of Natural Language Generation Models [62.997667081978825]
  We describe a system that fine-tunes a natural language generation model for the problem of solving Writer's Block. The proposed fine-tuning obtains excellent results, even with a small number of epochs and a total cost of USD 150.
  arXiv  Detail & Related papers  (2020-12-19T11:19:11Z)
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training [93.79766670391618]
  We present POINTER, a novel insertion-based approach for hard-constrained text generation. The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner. The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2020-05-01T18:11:54Z)
• Syntax-driven Iterative Expansion Language Models for Controllable Text Generation [2.578242050187029]
  We propose a new paradigm for introducing a syntactic inductive bias into neural text generation. Our experiments show that this paradigm is effective at text generation, with quality between LSTMs and Transformers, and comparable diversity.
  arXiv  Detail & Related papers  (2020-04-05T14:29:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.