AutoTemplate: A Simple Recipe for Lexically Constrained Text Generation

• URL: http://arxiv.org/abs/2211.08387v2
• Date: Fri, 9 Aug 2024 23:34:26 GMT
• Title: AutoTemplate: A Simple Recipe for Lexically Constrained Text Generation
• Authors: Hayate Iso,
• Abstract summary: We introduce AutoTemplate, a simple yet effective lexically constrained text generation framework. We conduct experiments on two tasks: keywords-to-sentence generations and entity-guided summarization. Experimental results show that the AutoTemplate outperforms the competitive baselines on both tasks.
• Score: 2.7763177595791655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lexically constrained text generation is one of the constrained text generation tasks, which aims to generate text that covers all the given constraint lexicons. While the existing approaches tackle this problem using a lexically constrained beam search algorithm or dedicated model using non-autoregressive decoding, there is a trade-off between the generated text quality and the hard constraint satisfaction. We introduce AutoTemplate, a simple yet effective lexically constrained text generation framework divided into template generation and lexicalization tasks. The template generation is to generate the text with the placeholders, and lexicalization replaces them into the constraint lexicons to perform lexically constrained text generation. We conducted the experiments on two tasks: keywords-to-sentence generations and entity-guided summarization. Experimental results show that the AutoTemplate outperforms the competitive baselines on both tasks while satisfying the hard lexical constraints. The code is available at https://github.com/megagonlabs/autotemplate

Related papers

• Efficient Guided Generation for Large Language Models [0.21485350418225244]
  We show how the problem of neural text generation can be constructively reformulated in terms of transitions between the states of a finite-state machine. This framework leads to an efficient approach to guiding text generation with regular expressions and context-free grammars.
  arXiv  Detail & Related papers  (2023-07-19T01:14:49Z)
• Copy Is All You Need [66.00852205068327]
  We formulate text generation as progressively copying text segments from an existing text collection. Our approach achieves better generation quality according to both automatic and human evaluations. Our approach attains additional performance gains by simply scaling up to larger text collections.
  arXiv  Detail & Related papers  (2023-07-13T05:03:26Z)
• Most Language Models can be Poets too: An AI Writing Assistant and Constrained Text Generation Studio [0.5097809301149341]
  We find that most language models generate compelling text even under significant constraints. We present a technique for modifying the output of a language model by compositionally applying filter functions to the language models vocabulary. We also present a Huggingface space web-app presenting this technique called Gadsby.
  arXiv  Detail & Related papers  (2023-06-28T05:10:51Z)
• Text-Blueprint: An Interactive Platform for Plan-based Conditional Generation [84.95981645040281]
  Planning can be a useful intermediate step to render conditional generation less opaque and more grounded. We present a web browser-based demonstration for query-focused summarization that uses a sequence of question-answer pairs.
  arXiv  Detail & Related papers  (2023-04-28T18:14:48Z)
• Tractable Control for Autoregressive Language Generation [82.79160918147852]
  We propose to use tractable probabilistic models (TPMs) to impose lexical constraints in autoregressive text generation models. We show that GeLaTo achieves state-of-the-art performance on challenging benchmarks for constrained text generation. Our work opens up new avenues for controlling large language models and also motivates the development of more expressive TPMs.
  arXiv  Detail & Related papers  (2023-04-15T00:19:44Z)
• Event Transition Planning for Open-ended Text Generation [55.729259805477376]
  Open-ended text generation tasks require models to generate a coherent continuation given limited preceding context. We propose a novel two-stage method which explicitly arranges the ensuing events in open-ended text generation. Our approach can be understood as a specially-trained coarse-to-fine algorithm.
  arXiv  Detail & Related papers  (2022-04-20T13:37:51Z)
• Text Editing by Command [82.50904226312451]
  A prevailing paradigm in neural text generation is one-shot generation, where text is produced in a single step. We address this limitation with an interactive text generation setting in which the user interacts with the system by issuing commands to edit existing text. We show that our Interactive Editor, a transformer-based model trained on this dataset, outperforms baselines and obtains positive results in both automatic and human evaluations.
  arXiv  Detail & Related papers  (2020-10-24T08:00:30Z)
• POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training [93.79766670391618]
  We present POINTER, a novel insertion-based approach for hard-constrained text generation. The proposed method operates by progressively inserting new tokens between existing tokens in a parallel manner. The resulting coarse-to-fine hierarchy makes the generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2020-05-01T18:11:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.