Reasoning with Sampling: Cutting at Decision Points

• URL: http://arxiv.org/abs/2605.30327v1
• Date: Thu, 28 May 2026 17:57:32 GMT
• Title: Reasoning with Sampling: Cutting at Decision Points
• Abstract summary: We show that sampling from a sharpened version of the base model's distribution elicits comparable reasoning without additional training, curated datasets, or verifiers.<n>A sampler needs to "mix" to the power distribution, which necessitates moving between modes of the target distribution.<n>We introduce an algorithm that uses the base model's next-token entropy as a proxy to identify key decision points and resample from those positions.
• Score: 7.438249964230582
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Frontier reasoning models are produced by posttraining base language models with reinforcement learning. Recent work has challenged this by showing that sampling from a sharpened version of the base model's distribution, a so-called power distribution, elicits comparable reasoning without additional training, curated datasets, or verifiers. However, making this method practical requires efficiently sampling from the power distribution. A sampler needs to "mix" to the power distribution, which necessitates moving between modes of the target distribution; intuitively, e.g., trying different reasoning strategies. The samplers proposed in prior works repeatedly select a "cut" position in the current reasoning trace uniformly at random and resample the suffix from that position onward. However, reasoning traces typically contain a few consequential decisions (e.g., the choice of proof strategy or algorithm), and we observe that a uniformly chosen cut tends to rewrite local details rather than revisit decision points. We introduce an algorithm (Entropy-Cut Metropolis-Hastings) that uses the base model's next-token entropy as a proxy to identify key decision points and resample from those positions. We empirically verify that entropy jumps are a useful proxy for decision points and, in a stylized model of reasoning, prove that our method's mixing time scales with the number of decisions in a trace rather than with the number of tokens, which can be much larger. Across MATH500, HumanEval, GPQA Diamond, and AIME26, our method consistently improves over baselines and RL-trained models.

Related papers

• Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models [14.588291638328329]
  TraFL is a trajectory-balance objective that trains the policy toward a reward-tilted target distribution anchored to a frozen reference model.<n>We make this practical for diffusion language models with a diffusion-compatible sequence-level surrogate and a learned prompt-dependent normalization.
  arXiv  Detail & Related papers  (2026-05-13T16:14:46Z)
• Reasoning with Sampling: Your Base Model is Smarter Than You Think [52.639108524651846]
  We propose a simple iterative sampling algorithm leveraging the base models' own likelihoods.<n>We show that our algorithm offers substantial boosts in reasoning that nearly match and even outperform those from RL.<n>Our method does not require training, curated datasets, or a verifier.
  arXiv  Detail & Related papers  (2025-10-16T17:18:11Z)
• Correct Reasoning Paths Visit Shared Decision Pivots [11.42890083738685]
  We introduce the idea of decision pivots-minimal, verifiable checkpoints that any correct reasoning path must visit.<n>We propose a self-training pipeline that samples diverse reasoning paths and mines shared decision pivots.<n>The proposed method aligns reasoning without ground truth reasoning data or external metrics.
  arXiv  Detail & Related papers  (2025-09-25T20:34:45Z)
• Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts [64.34482582690927]
  We provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models.<n>We propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality.
  arXiv  Detail & Related papers  (2025-03-04T17:46:51Z)
• Mirror Bridges Between Probability Measures [16.359542985936713]
  Resampling from a target measure whose density is unknown is a fundamental problem in mathematical statistics and machine learning.<n>We propose a new model called mirror bridges to solve this problem of conditional resampling.
  arXiv  Detail & Related papers  (2024-10-09T15:48:56Z)
• DistPred: A Distribution-Free Probabilistic Inference Method for Regression and Forecasting [14.390842560217743]
  We propose a novel approach called DistPred for regression and forecasting tasks.<n>We transform proper scoring rules that measure the discrepancy between the predicted distribution and the target distribution into a differentiable discrete form.<n>This allows the model to sample numerous samples in a single forward pass to estimate the potential distribution of the response variable.
  arXiv  Detail & Related papers  (2024-06-17T10:33:00Z)
• Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations [67.40641255908443]
  We identify limitations of model-randomization-based sanity checks for the purpose of evaluating explanations. Top-down model randomization preserves scales of forward pass activations with high probability.
  arXiv  Detail & Related papers  (2022-11-22T18:52:38Z)
• Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
  We take a two-step approach by first modeling the probability distribution and then sampling from that model. We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
  arXiv  Detail & Related papers  (2021-10-20T12:25:22Z)
• Distributional Reinforcement Learning via Moment Matching [54.16108052278444]
  We formulate a method that learns a finite set of statistics from each return distribution via neural networks. Our method can be interpreted as implicitly matching all orders of moments between a return distribution and its Bellman target. Experiments on the suite of Atari games show that our method outperforms the standard distributional RL baselines.
  arXiv  Detail & Related papers  (2020-07-24T05:18:17Z)
• Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning [61.32992639292889]
  Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks. We introduce a new scoring method that casts a plausibility ranking task in a full-text format. We show that our method provides a much more stable training phase across random restarts.
  arXiv  Detail & Related papers  (2020-04-29T10:54:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.