Sample Where You Struggle: Sharpening Base Model Reasoning via Entropy-Guided Power Sampling

• URL: http://arxiv.org/abs/2606.09926v1
• Date: Sun, 07 Jun 2026 14:06:20 GMT
• Title: Sample Where You Struggle: Sharpening Base Model Reasoning via Entropy-Guided Power Sampling
• Abstract summary: Entropy-Guided Power Sampling (EGPS) is a training-free and verifier-free sampler.<n>EGPS skips deterministic blocks, localizes each MCMC move to a high-entropy neighborhood, and applies Multiple-Try Metropolis at decision points.
• Score: 7.8682919561927065
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sampling from the sequence-level power distribution $p^α$ elicits RL-level reasoning from base language models without any parameter updates, but the standard Metropolis--Hastings (MH), a Markov Chain Monte Carlo (MCMC) sampler, is both expensive and slow-mixing. We trace both to a structural mismatch: $p^α$ mainly departs from $p$ at a sparse, spatially clustered set of high-entropy decision points, yet MH proposes resampling positions uniformly along the prefix -- wasting compute on near-degenerate conditionals while under-mixing precisely where modes diverge. We propose Entropy-Guided Power Sampling (EGPS), a training-free and verifier-free sampler that re-derives its proposal from token-level entropy already in the forward pass. EGPS skips deterministic blocks, localizes each MCMC move to a high-entropy neighborhood, and applies Multiple-Try Metropolis at decision points -- making sampling cost scale with \emph{entropy mass rather than sequence length}. On Qwen2.5-Math-7B, EGPS reaches best or tied-best accuracy on all three benchmarks (MATH500 $75.8\%$, HumanEval $62.2\%$, GPQA $42.4\%$) at up to a $12.6\times$ wall-clock speedup over the MH baseline.

Related papers

• Entropy-Gated Latent Recursion [9.65821666936513]
  In-time scaling has become the dominant lever for improving language-model reasoning.<n>Existing methods derive rollout diversity from a single source: token-level sampling.<n>We argue that this single-axis sampling space is fundamentally limiting.<n>We show that the $L$-axis is genuinely complementary to temperature.
  arXiv  Detail & Related papers  (2026-06-15T12:14:01Z)
• Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning [11.356198488445488]
  We introduce Power-SMC, a training-free Sequential Monte Carlo scheme that targets the same objective while remaining close to standard decoding latency.<n>On MATH500, Power-SMC matches or exceeds MH power sampling while reducing latency from $16$--$28times$ to $1.4$--$3.3times$ over baseline decoding.
  arXiv  Detail & Related papers  (2026-02-10T20:31:40Z)
• Amortized Simulation-Based Inference in Generalized Bayes via Neural Posterior Estimation [1.096028999747108]
  Generalized Bayesian Inference (GBI) tempers a loss with a temperature $>0$ to mitigate overconfidence and improve under model misspecification.<n>We give the first fully amortized variational approximation to the tempered posterior family $p_(mid x) propto (),p(x mid )$ by training a single $(x,)$-conditioned neural posterior estimator $q_(mid x,)$ that enables sampling in a single forward pass.
  arXiv  Detail & Related papers  (2026-01-29T22:20:47Z)
• Phase-space entropy at acquisition reflects downstream learnability [54.4100065023873]
  We propose an acquisition-level scalar $S_mathcal B$ based on instrument-resolved phase space.<n>We show theoretically that (S_mathcal B) correctly identifies the phase-space coherence of periodic sampling.<n>$|S_mathcal B|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty emphwithout training.
  arXiv  Detail & Related papers  (2025-12-22T10:03:51Z)
• Polynomial Mixing Times of Simulated Tempering for Mixture Targets by Conductance Decomposition [4.008356608627647]
  We study the theoretical complexity of simulated tempering for sampling from mixtures of log-concave components differing only by location shifts.<n>The main result establishes the first-time guarantee for simulated tempering combined with the Metropolis- Langevin algorithm (MALA)<n>The proof builds on a general state decomposition theorem for $s$-conductance, applied to an auxiliary Markov chain constructed on an augmented space.
  arXiv  Detail & Related papers  (2025-11-01T21:16:35Z)
• CarBoN: Calibrated Best-of-N Sampling Improves Test-time Reasoning [62.56541355300587]
  We introduce a general test-time calibration framework that adaptively modifies the model toward high-reward reasoning paths.<n>Within this framework, we propose CarBoN, a two-phase method that first explores the solution space and then learns a calibration of the logits.<n>Experiments on MATH-500 and AIME-2024 show that CarBoN improves efficiency, with up to $4times$ fewer rollouts to reach the same accuracy.
  arXiv  Detail & Related papers  (2025-10-17T14:04:37Z)
• Spectral gap of Metropolis-within-Gibbs under log-concavity [1.4466802614938334]
  The Metropolis-within-Gibbs (MwG) algorithm is a widely used Markov Chain Monte Carlo method for sampling from high-dimensional distributions.<n>We study MwG with Random Walk Metropolis (RWM) updates, using proposal variances tuned to match the target's conditional variances.<n>The result shows that MwG can mix substantially faster with variance-adaptive proposals and that its mixing performance is just a constant factor worse than that of the exact Gibbs sampler.
  arXiv  Detail & Related papers  (2025-09-30T12:31:22Z)
• Simultaneous approximation of multiple degenerate states using a single neural network quantum state [0.0]
  We propose a single-trunk multi-head (ST-MH) NQS ensemble that share a feature extracting trunk while attaching lightweight heads for each target state.<n>We prove that ST-MH can represent every eigenstate exactly whenever the feature map of latent width $h$, augmented with a constant, has column space.<n>We demonstrate that ST-MH attains high fidelity and energy accuracy across degenerate ground state while using significantly lower computing resources.
  arXiv  Detail & Related papers  (2025-09-02T17:49:33Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Fast Rates for Maximum Entropy Exploration [52.946307632704645]
  We address the challenge of exploration in reinforcement learning (RL) when the agent operates in an unknown environment with sparse or no rewards. We study the maximum entropy exploration problem two different types. For visitation entropy, we propose a game-theoretic algorithm that has $widetildemathcalO(H3S2A/varepsilon2)$ sample complexity. For the trajectory entropy, we propose a simple algorithm that has a sample of complexity of order $widetildemathcalO(mathrmpoly(S,
  arXiv  Detail & Related papers  (2023-03-14T16:51:14Z)
• STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning [58.6792963686231]
  Federated Learning (FL) refers to the paradigm where multiple worker nodes (WNs) build a joint model by using local data. It is not clear how to choose the WNs' minimum update directions, the first minibatch sizes, and the local update frequency. We show that there is a trade-off curve between local update frequencies and local mini sizes, on which the above complexities can be maintained.
  arXiv  Detail & Related papers  (2021-06-19T06:13:45Z)
• A New Framework for Variance-Reduced Hamiltonian Monte Carlo [88.84622104944503]
  We propose a new framework of variance-reduced Hamiltonian Monte Carlo (HMC) methods for sampling from an $L$-smooth and $m$-strongly log-concave distribution. We show that the unbiased gradient estimators, including SAGA and SVRG, based HMC methods achieve highest gradient efficiency with small batch size. Experimental results on both synthetic and real-world benchmark data show that our new framework significantly outperforms the full gradient and gradient HMC approaches.
  arXiv  Detail & Related papers  (2021-02-09T02:44:24Z)
• Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction [63.41789556777387]
  Asynchronous Q-learning aims to learn the optimal action-value function (or Q-function) of a Markov decision process (MDP) We show that the number of samples needed to yield an entrywise $varepsilon$-accurate estimate of the Q-function is at most on the order of $frac1mu_min (1-gamma)5varepsilon2+ fract_mixmu_min (1-gamma)$ up to some logarithmic factor.
  arXiv  Detail & Related papers  (2020-06-04T17:51:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.