Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective

• URL: http://arxiv.org/abs/2506.05754v1
• Date: Fri, 06 Jun 2025 05:28:20 GMT
• Title: Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective
• Authors: Emmanuel Anaya Gonzalez, Sairam Vaidya, Kanghee Park, Ruyi Ji, Taylor Berg-Kirkpatrick, Loris D'Antoni,
• Abstract summary: Constrained decoding enables Language Models to produce samples that provably satisfy hard constraints.<n>Existing constrained-decoding approaches distort the underlying model distribution.<n>We propose a new constrained sampling framework based on Markov Chain Monte Carlo.
• Score: 31.37618506317961
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Constrained decoding enables Language Models (LMs) to produce samples that provably satisfy hard constraints. However, existing constrained-decoding approaches often distort the underlying model distribution, a limitation that is especially problematic in applications like program fuzzing, where one wants to generate diverse and valid program inputs for testing purposes. We propose a new constrained sampling framework based on Markov Chain Monte Carlo (MCMC) that simultaneously satisfies three core desiderata: constraint satisfying (every sample satisfies the constraint), monotonically converging (the sampling process converges to the true conditional distribution), and efficient (high-quality samples emerge in few steps). Our method constructs a proposal distribution over valid outputs and applies a Metropolis-Hastings acceptance criterion based on the LM's likelihood, ensuring principled and efficient exploration of the constrained space. Empirically, our sampler outperforms existing methods on both synthetic benchmarks and real-world program fuzzing tasks.

Related papers

• Test-Time Alignment of Discrete Diffusion Models with Sequential Monte Carlo [19.81513273510523]
  We propose a training-free method based on Sequential Monte Carlo (SMC) to sample from the reward-aligned target distribution at the test time.<n>Our approach leverages twisted SMC with an approximate locally optimal proposal, obtained via a first-order Taylor expansion of the reward function.<n>To address the challenge of ill-defined gradients in discrete spaces, we incorporate a Gumbel-Softmax relaxation, enabling efficient gradient-based approximation within the discrete generative framework.
  arXiv  Detail & Related papers  (2025-05-28T16:12:03Z)
• Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling [90.86991492288487]
  evaluating constraint on every token can be prohibitively expensive.<n> LCD can distort the global distribution over strings, sampling tokens based only on local information.<n>We show that our approach is superior to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-07T18:30:18Z)
• Controllable Generation via Locally Constrained Resampling [77.48624621592523]
  We propose a tractable probabilistic approach that performs Bayesian conditioning to draw samples subject to a constraint. Our approach considers the entire sequence, leading to a more globally optimal constrained generation than current greedy methods. We show that our approach is able to steer the model's outputs away from toxic generations, outperforming similar approaches to detoxification.
  arXiv  Detail & Related papers  (2024-10-17T00:49:53Z)
• A Block Metropolis-Hastings Sampler for Controllable Energy-based Text Generation [78.81021361497311]
  We develop a novel Metropolis-Hastings (MH) sampler that proposes re-writes of the entire sequence in each step via iterative prompting of a large language model. Our new sampler allows for more efficient and accurate sampling from a target distribution and (b) allows generation length to be determined through the sampling procedure rather than fixed in advance.
  arXiv  Detail & Related papers  (2023-12-07T18:30:15Z)
• Exact and general decoupled solutions of the LMC Multitask Gaussian Process model [28.32223907511862]
  The Linear Model of Co-regionalization (LMC) is a very general model of multitask gaussian process for regression or classification. Recent work has shown that under some conditions the latent processes of the model can be decoupled, leading to a complexity that is only linear in the number of said processes. We here extend these results, showing from the most general assumptions that the only condition necessary to an efficient exact computation of the LMC is a mild hypothesis on the noise model.
  arXiv  Detail & Related papers  (2023-10-18T15:16:24Z)
• Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs [60.58434523646137]
  A popular approach for improving the correctness of output from large language models (LLMs) is Self-Consistency. We introduce Adaptive-Consistency, a cost-efficient, model-agnostic technique that dynamically adjusts the number of samples per question. Our experiments show that Adaptive-Consistency reduces sample budget by up to 7.9 times with an average accuracy drop of less than 0.1%.
  arXiv  Detail & Related papers  (2023-05-19T17:49:25Z)
• Learning Sampling Distributions for Model Predictive Control [36.82905770866734]
  Sampling-based approaches to Model Predictive Control (MPC) have become a cornerstone of contemporary approaches to MPC. We propose to carry out all operations in the latent space, allowing us to take full advantage of the learned distribution. Specifically, we frame the learning problem as bi-level optimization and show how to train the controller with backpropagation-through-time.
  arXiv  Detail & Related papers  (2022-12-05T20:35:36Z)
• Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models [65.52639709094963]
  Methods such as beam search and Gumbel top-k sampling can guarantee a different output for each element of the beam, but are not easy to parallelize. We present a framework for sampling according to an arithmetic code book implicitly defined by a large language model.
  arXiv  Detail & Related papers  (2022-10-18T22:19:41Z)
• Approximating Constraint Manifolds Using Generative Models for Sampling-Based Constrained Motion Planning [8.924344714683814]
  This paper presents a learning-based sampling strategy for constrained motion planning problems. We use Conditional Variversaational Autoencoder (CVAE) and Conditional Generative Adrial Net (CGAN) to generate constraint-satisfying sample configurations. We evaluate the efficiency of these two generative models in terms of their sampling accuracy and coverage of sampling distribution.
  arXiv  Detail & Related papers  (2022-04-14T07:08:30Z)
• Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization [61.26619639722804]
  We propose a conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms. The proposed method, equipped with an average gradient (SAG) estimator, requires only one sample per iteration. Nevertheless, it guarantees fast convergence rates on par with more sophisticated variance reduction techniques.
  arXiv  Detail & Related papers  (2022-02-26T19:10:48Z)
• Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set [3.862247454265944]
  We develop a framework to develop calibration schemes that satisfy rigorous frequentist statistical guarantees. We demonstrate our methodology on several numerical examples, including an application to calibration of a limit order book market simulator.
  arXiv  Detail & Related papers  (2021-05-27T00:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.