Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Sampling

• URL: http://arxiv.org/abs/2607.20791v1
• Date: Wed, 22 Jul 2026 23:33:51 GMT
• Title: Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Sampling
• Abstract summary: We study how temperature influences refusal behavior in LLMs.<n>We propose an efficient sequential decoding approach which preserves a model's greedy decoding refusal response at high temperatures.<n>Our work demonstrates how refusal behavior can be maintained in an efficient manner for applications which require high-temperature sampling.
• Score: 7.887862284767784
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-temperature sampling is one of the primary mechanisms for increasing diversity in LLMs. Recent advances in truncation-based sampling techniques have helped mitigate drawbacks of high-temperature sampling such as neural text degeneration, thereby enabling greater diversity in LLM outputs without sacrificing coherence. However, increasing the entropy of the token probability distribution via high temperatures has also been shown to weaken model guardrails by reducing the model's refusal response in the presence of harmful prompts. Despite the potential benefits of high-temperature sampling and the importance of maintaining model safety, there is a lack of existing solutions for maintaining the refusal behavior of LLMs under a higher entropy regime. To address this gap, we systematically study how temperature influences refusal behavior in LLMs and propose an efficient sequential decoding approach which preserves a model's greedy decoding refusal response at high temperatures while incurring minimal additional latency. Through extensive experiments, we show that our approach preserves 91-99% of the greedy decoding refusal behavior across three benchmark datasets without compromising the model's high-temperature response for safe prompts. Our work demonstrates how refusal behavior can be maintained in an efficient manner for applications which require high-temperature sampling.

Related papers

• Internalize the Temperature: On-Policy Self-Distillation as Policy Reheater for Reinforcement Learning [43.29500828702542]
  Temperature-Scaled On-Policy Self-Distillation (TS-OPSD) is a lightweight policy reheating method that internalizes the exploratory effect of temperature into model parameters.<n>TS-OPSD constructs a self-teacher by applying high-temperature scaling to the model's own logits, then distills the resulting smoother distribution back into the student.
  arXiv  Detail & Related papers  (2026-05-30T14:44:57Z)
• Improving LLM-based Recommendation with Self-Hard Negatives from Intermediate Layers [80.55429742713623]
  ILRec is a novel preference fine-tuning framework for LLM-based recommender systems.<n>We introduce a lightweight collaborative filtering model to assign token-level rewards for negative signals.<n>Experiments on three datasets demonstrate ILRec's effectiveness in enhancing the performance of LLM-based recommender systems.
  arXiv  Detail & Related papers  (2026-02-19T14:37:43Z)
• Energy-Driven Steering: Reducing False Refusals in Large Language Models [80.09252175869858]
  We introduce Energy-Driven Steering (EDS), a novel, fine-tuning free framework designed to resolve this challenge through dynamic, inference-time intervention.<n>We trained a lightweight, external Energy-Based Model (EBM) to assign high energy to undesirable (false refusal or jailbreak) states and low energy to desirable (helpful response or safe reject) ones.<n>We use the gradient of the energy function to dynamically steer the LLM's hidden states to low energy regions, correcting the model to generate a desirable response in real-time without modifying its weights.
  arXiv  Detail & Related papers  (2025-10-09T06:01:41Z)
• Let it Calm: Exploratory Annealed Decoding for Verifiable Reinforcement Learning [29.277754405630205]
  Reinforcement learning with verifiable rewards (RLVR) is a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs)<n>Standard fixed-temperature sampling is simple, but it struggles to balance these competing demands, as high temperatures degrade sample quality and low temperatures limit discovery.<n>We propose a simpler and more effective strategy, Exploratory Annealed Decoding (EAD), grounded in the insight that exploration is most impactful on early tokens.
  arXiv  Detail & Related papers  (2025-10-06T18:15:43Z)
• Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs [26.477037145228735]
  Temperature-based sampling is a common strategy to increase diversity.<n>But uncontrolled high temperature sampling, e.g., min-$p$ or top-$p$, degrades reasoning quality.<n>We propose textbfselective sampling, a method that switches between greedy and high-temperature sampling.
  arXiv  Detail & Related papers  (2025-09-20T15:16:27Z)
• Progressive Tempering Sampler with Diffusion [50.06039228068602]
  We propose a neural sampler that trains diffusion models sequentially across temperatures.<n>We also introduce a novel method to combine high-temperature diffusion models to generate approximate lower-temperature samples.<n>Our method significantly improves target evaluation efficiency, outperforming diffusion-based neural samplers.
  arXiv  Detail & Related papers  (2025-06-05T16:46:04Z)
• Monte Carlo Temperature: a robust sampling strategy for LLM's uncertainty quantification methods [1.3892342684177872]
  We propose a robust sampling strategy that eliminates the need for temperature calibration.<n>MCT provides more robust uncertainty estimates across a wide range of temperatures.<n>MCT achieves statistical parity with oracle temperatures, which represent the ideal outcome of a well-tuned but computationally expensive HPO process.
  arXiv  Detail & Related papers  (2025-02-25T17:33:20Z)
• Adaptive Decoding via Latent Preference Optimization [55.70602730588745]
  We introduce Adaptive Decoding, a layer added to the model to select the sampling temperature dynamically at inference time. Our method outperforms all fixed decoding temperatures across a range of tasks that require different temperatures.
  arXiv  Detail & Related papers  (2024-11-14T18:31:39Z)
• REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy [93.8400683020273]
  Decoding methods for large language models (LLMs) usually struggle with the tradeoff between ensuring factuality and maintaining diversity. We propose REAL sampling, a decoding method that improved factuality and diversity over nucleus sampling.
  arXiv  Detail & Related papers  (2024-06-11T21:44:49Z)
• Extreme Miscalibration and the Illusion of Adversarial Robustness [66.29268991629085]
  Adversarial Training is often used to increase model robustness. We show that this observed gain in robustness is an illusion of robustness (IOR) We urge the NLP community to incorporate test-time temperature scaling into their robustness evaluations.
  arXiv  Detail & Related papers  (2024-02-27T13:49:12Z)
• KL-Divergence Guided Temperature Sampling [5.726259957909055]
  As temperature increases, the prediction becomes diverse but also vulnerable to hallucinations. One common approach to mitigate hallucinations is to provide source/grounding documents. We propose to relax the constraint of having a fixed temperature over decoding steps, and a mechanism to guide the dynamic temperature according to its relevance to the source.
  arXiv  Detail & Related papers  (2023-06-02T06:11:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.