Constraints First: A New MDD-based Model to Generate Sentences Under Constraints

• URL: http://arxiv.org/abs/2309.12415v1
• Date: Thu, 21 Sep 2023 18:29:52 GMT
• Title: Constraints First: A New MDD-based Model to Generate Sentences Under Constraints
• Authors: Alexandre Bonlarron, Aur\'elie Calabr\`ese, Pierre Kornprobst, Jean-Charles R\'egin
• Abstract summary: This paper introduces a new approach to generating strongly constrained texts. We use multivalued decision diagrams (MDD), a well-known data structure to deal with constraints. We get hundreds of bona-fide candidate sentences when compared with the few dozen sentences usually available in the well-known vision screening test (MNREAD)
• Score: 45.498315114762484
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces a new approach to generating strongly constrained texts. We consider standardized sentence generation for the typical application of vision screening. To solve this problem, we formalize it as a discrete combinatorial optimization problem and utilize multivalued decision diagrams (MDD), a well-known data structure to deal with constraints. In our context, one key strength of MDD is to compute an exhaustive set of solutions without performing any search. Once the sentences are obtained, we apply a language model (GPT-2) to keep the best ones. We detail this for English and also for French where the agreement and conjugation rules are known to be more complex. Finally, with the help of GPT-2, we get hundreds of bona-fide candidate sentences. When compared with the few dozen sentences usually available in the well-known vision screening test (MNREAD), this brings a major breakthrough in the field of standardized sentence generation. Also, as it can be easily adapted for other languages, it has the potential to make the MNREAD test even more valuable and usable. More generally, this paper highlights MDD as a convincing alternative for constrained text generation, especially when the constraints are hard to satisfy, but also for many other prospects.

Related papers

• Intertwining CP and NLP: The Generation of Unreasonably Constrained Sentences [49.86129209397701]
  This paper presents the Constraints First Framework to remedy this issue. It is solved by a constraint programming method that combines linguistic properties with more classical constraints. The effectiveness of this approach is demonstrated by tackling a new more tediously constrained text generation problem.
  arXiv  Detail & Related papers  (2024-06-15T17:40:49Z)
• Markov Constraint as Large Language Model Surrogate [49.86129209397701]
  NgramMarkov is dedicated to text generation in constraint programming (CP) It limits the product of the probabilities of the n-gram of a sentence. A real-world problem has been solved for the first time using 4-grams instead of 5-grams.
  arXiv  Detail & Related papers  (2024-06-11T16:09:53Z)
• BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generation [60.77990074569754]
  We present a computation-efficient framework that steers a frozen Pre-Trained Language Model towards more commonsensical generation. Specifically, we first construct a reference-free evaluator that assigns a sentence with a commonsensical score. We then use the scorer as the oracle for commonsense knowledge, and extend the controllable generation method called NADO to train an auxiliary head.
  arXiv  Detail & Related papers  (2023-10-25T23:32:12Z)
• Reference Free Domain Adaptation for Translation of Noisy Questions with Question Specific Rewards [22.297433705607464]
  Translating questions using Neural Machine Translation poses more challenges in noisy environments. We propose a training methodology that fine-tunes the NMT system only using source-side data. Our approach balances adequacy and fluency by utilizing a loss function that combines BERTScore and Masked Language Model (MLM) Score.
  arXiv  Detail & Related papers  (2023-10-23T18:08:01Z)
• DICTDIS: Dictionary Constrained Disambiguation for Improved NMT [50.888881348723295]
  We present DictDis, a lexically constrained NMT system that disambiguates between multiple candidate translations derived from dictionaries. We demonstrate the utility of DictDis via extensive experiments on English-Hindi and English-German sentences in a variety of domains including regulatory, finance, engineering.
  arXiv  Detail & Related papers  (2022-10-13T13:04:16Z)
• Entity Disambiguation with Entity Definitions [50.01142092276296]
  Local models have recently attained astounding performances in Entity Disambiguation (ED) Previous works limited their studies to using, as the textual representation of each candidate, only its Wikipedia title. In this paper, we address this limitation and investigate to what extent more expressive textual representations can mitigate it. We report a new state of the art on 2 out of 6 benchmarks we consider and strongly improve the generalization capability over unseen patterns.
  arXiv  Detail & Related papers  (2022-10-11T17:46:28Z)
• Generating texts under constraint through discriminator-guided MCTS [1.3750624267664153]
  We formalize constrained generation as a tree exploration process guided by a discriminator. Using a discriminator to guide this generation, rather than fine-tuning the LM, allows to apply the constraint more finely and dynamically. We show that our methods achieves state-of-the-art results in constrained generation, without having to tune the language model.
  arXiv  Detail & Related papers  (2021-09-28T09:29:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.