The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior

• URL: http://arxiv.org/abs/2512.12066v2
• Date: Tue, 16 Dec 2025 03:45:08 GMT
• Title: The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior
• Abstract summary: Current safety evaluations of large language models rely on single-shot testing.<n>We investigate the stability of safety refusal decisions across random seeds and temperature settings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current safety evaluations of large language models rely on single-shot testing, implicitly assuming that model responses are deterministic and representative of the model's safety alignment. We challenge this assumption by investigating the stability of safety refusal decisions across random seeds and temperature settings. Testing four instruction-tuned models from three families (Llama 3.1 8B, Qwen 2.5 7B, Qwen 3 8B, Gemma 3 12B) on 876 harmful prompts across 20 different sampling configurations (4 temperatures x 5 random seeds), we find that 18-28% of prompts exhibit decision flips--the model refuses in some configurations but complies in others--depending on the model. Our Safety Stability Index (SSI) reveals that higher temperatures significantly reduce decision stability (Friedman chi-squared = 396.81, p < 0.001), with mean within-temperature SSI dropping from 0.977 at temperature 0.0 to 0.942 at temperature 1.0. We validate our findings across all model families using Claude 3.5 Haiku as a unified external judge, achieving 89.0% inter-judge agreement with our primary Llama 70B judge (Cohen's kappa = 0.62). Within each model, prompts with higher compliance rates exhibit lower stability (Spearman rho = -0.47 to -0.70, all p < 0.001), indicating that models "waver" more on borderline requests. These findings demonstrate that single-shot safety evaluations are insufficient for reliable safety assessment and that evaluation protocols must account for stochastic variation in model behavior. We show that single-shot evaluation agrees with multi-sample ground truth only 92.4% of the time when pooling across temperatures (94.2-97.7% at fixed temperature depending on setting), and recommend using at least 3 samples per prompt for reliable safety assessment.

Related papers

• Condition-Stratified Robustness Analysis of Post-Hoc Calibration Methods for Probabilistic Classifiers [0.0]
  We compare temperature scaling (TEMP) and isotonic regression (ISO) across four controlled conditions.<n>Four hypothesis groups are evaluated: discrimination deltas with Holm-corrected control (H1), Brier score differences (H2), calibration slope outcomes (H3), and AUROC differences under best-condition setups (H4)
  arXiv  Detail & Related papers  (2026-07-13T13:28:31Z)
• The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis [0.0]
  LLM deployments routinely compress models and raise sampling temperature to reduce cost, latency, or repetition.<n>This leaves a practical uncertainty: does a model that is safe at FP16 and greedy decoding remain safe after it is quantized and sampled?<n>We study this question with a factorial evaluation of 9 instruction-tuned models from six families, 3 precisions, and 6 temperatures.
  arXiv  Detail & Related papers  (2026-06-28T19:52:25Z)
• Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion [0.0]
  Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify.<n>We show that at temperature zero, can draft-side behavior leak into safety-scored outputs?
  arXiv  Detail & Related papers  (2026-06-23T19:06:42Z)
• Evaluation Awareness Is Not One Capability: Evidence from Open Language Models [1.758143872501506]
  Safety benchmarks assume that test-condition behavior predicts deployment behavior, an assumption that fails if models detect evaluation cues and adapt.<n>This opens a gap between benchmark performance and deployment behavior.<n>We characterize this evaluation awareness through eight experiments across 37 open-weight models and seven families.
  arXiv  Detail & Related papers  (2026-06-22T16:48:43Z)
• Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment [0.0]
  Five large language models are tested to assess multisensor physical hazard data.<n>All tested models consistently produced no precautionary warning signal across tested scenarios.<n>These findings have direct implications for practitioners deploying the tested models in physical safety monitoring systems.
  arXiv  Detail & Related papers  (2026-05-25T05:43:37Z)
• Adaptive Rigor in AI System Evaluation using Temperature-Controlled Verdict Aggregation via Generalized Power Mean [51.56484100374058]
  This paper presents a method that combines a five-level verdict-scoring system with generalized power-mean aggregation and an intuitive temperature parameter T [0.1, 1.0] to control evaluation rigor.<n>Low temperatures yield pessimistic scores suited for safety-critical domains; high temperatures produce lenient scores appropriate for conversational AI.
  arXiv  Detail & Related papers  (2026-04-04T09:04:38Z)
• Evaluating LLM Safety Under Repeated Inference via Accelerated Prompt Stress Testing [0.0]
  We introduce Accelerated Prompt Stress Testing (APST), a depth-oriented evaluation framework inspired by reliability engineering.<n>APST repeatedly samples identical prompts under controlled operational conditions to surface latent failure modes.<n>We find that models with similar benchmark-aligned scores can exhibit substantially different empirical failure rates under repeated sampling.
  arXiv  Detail & Related papers  (2026-02-12T10:09:13Z)
• Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling [50.872910438715486]
  Large Language Models (LLMs) are typically evaluated for safety under single-shot or low-budget adversarial prompting.<n>We propose a scaling-aware Best-of-N estimation of risk, SABER, for modeling jailbreak vulnerability under Best-of-N sampling.
  arXiv  Detail & Related papers  (2026-01-30T06:54:35Z)
• Prompt Injection Evaluations: Refusal Boundary Instability and Artifact-Dependent Compliance in GPT-4-Series Models [0.0]
  We evaluate two models, GPT-4.1 and GPT-4o, using 3,274 perturbation runs derived from refusal-inducing prompt injection attempts.<n>We find that artifact type is a stronger predictor of refusal failure than perturbation style.<n>We conclude that refusal behavior is a probabilistic, artifact-dependent boundary phenomenon rather than a stable binary property.
  arXiv  Detail & Related papers  (2026-01-25T17:14:33Z)
• Causal Judge Evaluation: Calibrated Surrogate Metrics for LLM Systems [0.29465623430708904]
  Uncalibrated scores can invert preferences, naive confidence intervals on uncalibrated scores achieve near-0% coverage, and importance-weighted estimators collapse under limited overlap.<n>We introduce Causal Judge Evaluation, a framework that fixes all three failures.
  arXiv  Detail & Related papers  (2025-12-11T22:16:24Z)
• Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92% [4.693270291878929]
  Large language models (LLMs) produce fluent but unsupported answers - hallucinations.<n>We propose ECLIPSE, a framework that treats hallucination as a mismatch between a model's semantic entropy and the capacity of available evidence.
  arXiv  Detail & Related papers  (2025-12-02T05:25:48Z)
• VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety [3.1109025622085693]
  We present Vision Language Safety Understanding, a comprehensive framework to evaluate multimodal safety.<n>Our evaluation of eleven state-of-the-art models reveals systematic joint understanding failures.<n>Our framework exposes weaknesses in joint image-text understanding and alignment gaps in current models.
  arXiv  Detail & Related papers  (2025-10-21T01:30:31Z)
• Rethinking Safety in LLM Fine-tuning: An Optimization Perspective [56.31306558218838]
  We show that poor optimization choices, rather than inherent trade-offs, often cause safety problems, measured as harmful responses to adversarial prompts.<n>We propose a simple exponential moving average (EMA) momentum technique in parameter space that preserves safety performance.<n>Our experiments on the Llama families across multiple datasets demonstrate that safety problems can largely be avoided without specialized interventions.
  arXiv  Detail & Related papers  (2025-08-17T23:46:36Z)
• Extreme Miscalibration and the Illusion of Adversarial Robustness [66.29268991629085]
  Adversarial Training is often used to increase model robustness. We show that this observed gain in robustness is an illusion of robustness (IOR) We urge the NLP community to incorporate test-time temperature scaling into their robustness evaluations.
  arXiv  Detail & Related papers  (2024-02-27T13:49:12Z)
• Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval [51.83967175585896]
  UAL aims at providing reliability-aware predictions by considering data uncertainty and model uncertainty simultaneously. Data uncertainty captures the noise" inherent in the sample, while model uncertainty depicts the model's confidence in the sample's prediction.
  arXiv  Detail & Related papers  (2022-10-24T17:53:20Z)
• Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty [68.00748155945047]
  Capturing uncertainty in models of complex dynamical systems is crucial to designing safe controllers. Several approaches use formal abstractions to synthesize policies that satisfy temporal specifications related to safety and reachability. Our contribution is a novel abstraction-based controller method for continuous-state models with noise, uncertain parameters, and external disturbances.
  arXiv  Detail & Related papers  (2022-10-12T07:57:03Z)
• On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification [47.13680267076843]
  We show that explicitly accounting for aleatoric uncertainty significantly improves the performance of Bayesian neural networks. We find that a cold posterior, tempered by a power greater than one, often more honestly reflects our beliefs about aleatoric uncertainty than no tempering.
  arXiv  Detail & Related papers  (2022-03-30T17:17:50Z)
• Quadrature Squeezing And Temperature Estimation From The Fock Distribution [0.0]
  Squeezing and temperature are essential parameters states used in various quantum communication and sensing protocols. Our method allows estimation without a phase reference, by using for example a photon-number-resolving detector.
  arXiv  Detail & Related papers  (2021-11-09T03:07:55Z)
• Pointwise Feasibility of Gaussian Process-based Safety-Critical Control under Model Uncertainty [77.18483084440182]
  Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) are popular tools for enforcing safety and stability of a controlled system, respectively. We present a Gaussian Process (GP)-based approach to tackle the problem of model uncertainty in safety-critical controllers that use CBFs and CLFs.
  arXiv  Detail & Related papers  (2021-06-13T23:08:49Z)
• A Bayesian Monte-Carlo Uncertainty Model for Assessment of Shear Stress Entropy [0.0]
  We present a novel method to evaluate the uncertainty of four popular entropy models, including Shannon, Shannon-Power Low (PL), Tsallis, and Renyi, in shear stress estimation in circular channels. We developed a new statistic index called as FREEopt-based OCB (FOCB) using the statistical indices Forecasting Range of Error Estimation (FREE) and the percentage of observed data in the CB (Nin) The Shannon and Shannon PL entropies had the highest certainty in the calculation of shear stress values in circular channels followed by traditional uniform flow shear stress and Tsallis models with close values
  arXiv  Detail & Related papers  (2020-01-10T22:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.