MAP's not dead yet: Uncovering true language model modes by conditioning away degeneracy

• URL: http://arxiv.org/abs/2311.08817v2
• Date: Thu, 8 Aug 2024 01:46:19 GMT
• Title: MAP's not dead yet: Uncovering true language model modes by conditioning away degeneracy
• Abstract summary: We argue that mixing even a tiny amount of low-entropy noise with a population text distribution can cause the data distribution's mode to become degenerate. We empirically verify that the length-conditional modes of machine translation models and language models are indeed more fluent and topical than their unconditional modes.
• Score: 26.464639238671054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It has been widely observed that exact or approximate MAP (mode-seeking) decoding from natural language generation (NLG) models consistently leads to degenerate outputs (Holtzman et al., 2019; Stahlberg and Byrne, 2019). Prior work has attributed this behavior to either a fundamental and unavoidable inadequacy of modes in probabilistic models or weaknesses in language modeling. Contrastingly, we argue that degenerate modes can even occur in the absence of any modeling error, due to contamination of the training data. Specifically, we argue that mixing even a tiny amount of low-entropy noise with a population text distribution can cause the data distribution's mode to become degenerate. We therefore propose to apply MAP decoding to the model's true conditional distribution where the conditioning variable explicitly avoids specific degenerate behavior. Using exact search, we empirically verify that the length-conditional modes of machine translation models and language models are indeed more fluent and topical than their unconditional modes. For the first time, we also share many examples of exact modal sequences from these models, and from several variants of the LLaMA-7B model. Notably, we observe that various kinds of degenerate modes persist, even at the scale of LLaMA-7B. Although we cannot tractably address these degeneracies with exact search, we perform a classifier-based approximate search on LLaMA-7B, a model which was not trained for instruction following, and find that we are able to elicit reasonable outputs without any finetuning.

Related papers

• The Efficiency Gap in Byte Modeling [101.18202046105718]
  Two alternative paradigms have challenged the use of subword tokenization and autoregressive ordering.<n>We show that removing these structural priors incurs a significant computational cost.<n>Our findings suggest that future modality-agnostic designs must incorporate alternative structural biases to maintain viable scaling trajectories.
  arXiv  Detail & Related papers  (2026-05-13T03:03:30Z)
• Model Stealing for Any Low-Rank Language Model [25.16701867917684]
  We build a theoretical understanding of stealing language models by studying a simple and mathematically tractable setting. Our main result is an efficient algorithm in the conditional query model, for learning any low-rank distribution. This is an interesting example where, at least theoretically, allowing a machine learning model to solve more complex problems at inference time can lead to drastic improvements in its performance.
  arXiv  Detail & Related papers  (2024-11-12T04:25:31Z)
• Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines [74.42485647685272]
  We focus on Generative Masked Language Models (GMLMs) We train a model to fit conditional probabilities of the data distribution via masking, which are subsequently used as inputs to a Markov Chain to draw samples from the model. We adapt the T5 model for iteratively-refined parallel decoding, achieving 2-3x speedup in machine translation with minimal sacrifice in quality.
  arXiv  Detail & Related papers  (2024-07-22T18:00:00Z)
• Understanding Deep Generative Models with Generalized Empirical Likelihoods [3.7978679293562587]
  We show how to combine techniques from Maximum Mean Discrepancy and Generalized Empirical Likelihood to create distribution tests that retain per-sample interpretability. We find that such tests predict the degree of mode dropping and mode imbalance up to 60% better than metrics such as improved precision/recall.
  arXiv  Detail & Related papers  (2023-06-16T11:33:47Z)
• Tailoring Language Generation Models under Total Variation Distance [55.89964205594829]
  The standard paradigm of neural language generation adopts maximum likelihood estimation (MLE) as the optimizing method. We develop practical bounds to apply it to language generation. We introduce the TaiLr objective that balances the tradeoff of estimating TVD.
  arXiv  Detail & Related papers  (2023-02-26T16:32:52Z)
• Hierarchical Latent Structure for Multi-Modal Vehicle Trajectory Forecasting [0.0]
  We introduce a hierarchical latent structure into a VAE-based trajectory forecasting model. Our model is capable of generating clear multi-modal trajectory distributions and outperforms the state-of-the-art (SOTA) models in terms of prediction accuracy.
  arXiv  Detail & Related papers  (2022-07-11T04:52:28Z)
• Discrete Denoising Flows [87.44537620217673]
  We introduce a new discrete flow-based model for categorical random variables: Discrete Denoising Flows (DDFs) In contrast with other discrete flow-based models, our model can be locally trained without introducing gradient bias. We show that DDFs outperform Discrete Flows on modeling a toy example, binary MNIST and Cityscapes segmentation maps, measured in log-likelihood.
  arXiv  Detail & Related papers  (2021-07-24T14:47:22Z)
• Argmax Flows and Multinomial Diffusion: Towards Non-Autoregressive Language Models [76.22217735434661]
  This paper introduces two new classes of generative models for categorical data: Argmax Flows and Multinomial Diffusion. We demonstrate that our models perform competitively on language modelling and modelling of image segmentation maps.
  arXiv  Detail & Related papers  (2021-02-10T11:04:17Z)
• Goal-directed Generation of Discrete Structures with Conditional Generative Models [85.51463588099556]
  We introduce a novel approach to directly optimize a reinforcement learning objective, maximizing an expected reward. We test our methodology on two tasks: generating molecules with user-defined properties and identifying short python expressions which evaluate to a given target value.
  arXiv  Detail & Related papers  (2020-10-05T20:03:13Z)
• Characterizing and Avoiding Problematic Global Optima of Variational Autoencoders [28.36260646471421]
  Variational Auto-encoders (VAEs) are deep generative latent variable models. Recent work shows that traditional training methods tend to yield solutions that violate desiderata. We show that both issues stem from the fact that the global optima of the VAE training objective often correspond to undesirable solutions.
  arXiv  Detail & Related papers  (2020-03-17T15:14:25Z)
• On the Discrepancy between Density Estimation and Sequence Generation [92.70116082182076]
  log-likelihood is highly correlated with BLEU when we consider models within the same family. We observe no correlation between rankings of models across different families.
  arXiv  Detail & Related papers  (2020-02-17T20:13:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.