The Illusion of Stochasticity in LLMs

• URL: http://arxiv.org/abs/2604.06543v1
• Date: Wed, 08 Apr 2026 00:44:14 GMT
• Title: The Illusion of Stochasticity in LLMs
• Abstract summary: We demonstrate that reliable sampling is a fundamental yet unfulfilled requirement for Large Language Models (LLMs) operating as agents.<n>We show that while powerful frontier models can convert provided random seeds to target distributions, their ability to sample directly from specific distributions is fundamentally flawed.
• Score: 21.172079837071724
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we demonstrate that reliable stochastic sampling is a fundamental yet unfulfilled requirement for Large Language Models (LLMs) operating as agents. Agentic systems are frequently required to sample from distributions, often inferred from observed data, a process which needs to be emulated by the LLM. This leads to a distinct failure point: while standard RL agents rely on external sampling mechanisms, LLMs fail to map their internal probability estimates to their stochastic outputs. Through rigorous empirical analysis across multiple model families, model sizes, prompting styles, and distributions, we demonstrate the extent of this failure. Crucially, we show that while powerful frontier models can convert provided random seeds to target distributions, their ability to sample directly from specific distributions is fundamentally flawed.

Related papers

• UnpredictaBench: A Benchmark for Evaluating Distributional Randomness in LLMs [21.05820098186702]
  We introduce UnpredictaBench, an evaluation that tests the ability of large language models to capture true underlying distributions.<n>We find that even simple distributional simulation remains challenging, making it a necessary first step toward using LLMs as stand-ins for complex systems.
  arXiv  Detail & Related papers  (2026-06-04T18:18:02Z)
• Multi-Objective Learning for Diffusion Models: A Statistical Theory under Semi-Supervised Learning [83.65494573724904]
  We develop a principled MOL framework for diffusion models with limited data.<n>We propose a two-stage training procedure that first fits lightweight specialist models from limited paired data.<n>We establish bounds generalization showing that the required number of paired samples only depends on the complexity of the specialist model classes.
  arXiv  Detail & Related papers  (2026-05-24T18:19:40Z)
• D-Models and E-Models: Diversity-Stability Trade-offs in the Sampling Behavior of Large Language Models [91.21455683212224]
  In large language models (LLMs), the probability of relevance for the next piece of information is linked to the probability of relevance for the next product.<n>But whether fine-grained sampling probabilities faithfully align with task requirements remains an open question.<n>We identify two model types: D-models, whose P_token exhibits large step-to-step variability and poor alignment with P_task; and E-models, whose P_token is more stable and better aligned with P_task.
  arXiv  Detail & Related papers  (2026-01-25T14:59:09Z)
• Training-Free Distribution Adaptation for Diffusion Models via Maximum Mean Discrepancy Guidance [17.353524034156205]
  MMD Guidance augments the reverse diffusion process with gradients of the Maximum Mean Discrepancy (MMD) between generated samples and a reference dataset.<n>Our framework naturally extends to prompt-aware adaptation in conditional generation models via product kernels.<n>Experiments on synthetic and real-world benchmarks demonstrate that MMD Guidance can achieve distributional alignment while preserving sample fidelity.
  arXiv  Detail & Related papers  (2026-01-13T09:42:57Z)
• Large Language Models Are Bad Dice Players: LLMs Struggle to Generate Random Numbers from Statistical Distributions [50.1404916337174]
  We present the first large-scale, statistically powered audit of native probabilistic sampling in large language models (LLMs)<n>We show that batch generation achieves only modest statistical validity, with a 13% median pass rate, while independent requests collapse almost entirely.<n>We conclude that current LLMs lack a functional internal sampler, necessitating the use of external tools for applications requiring statistical guarantees.
  arXiv  Detail & Related papers  (2026-01-08T22:33:12Z)
• Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling [59.133428586090226]
  Large language models (LLMs) can often accurately describe probability distributions using natural language.<n>This mismatch limits their use in tasks requiring reliableity, such as Monte Carlo methods, agent-based simulations, and randomized decision-making.<n>We introduce Verbalized Rejection Sampling (VRS), a natural-language adaptation of classical rejection sampling.
  arXiv  Detail & Related papers  (2025-06-11T17:59:58Z)
• Evaluating Binary Decision Biases in Large Language Models: Implications for Fair Agent-Based Financial Simulations [15.379345372327375]
  Large Language Models (LLMs) are increasingly being used to simulate human-like decision making in agent-based financial market models.<n>We test three state-of-the-art GPT models for bias using two model sampling approaches: one-shot and few-shot API queries.
  arXiv  Detail & Related papers  (2025-01-20T10:36:51Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers.<n>We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.<n>This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation [73.58618024960968]
  An increasing number of studies are employing large language models (LLMs) as agents to emulate the sequential decision-making processes of humans.<n>This arouses curiosity regarding the capacity of LLM agents to comprehend probability distributions.<n>Our analysis indicates that LLM agents can understand probabilities, but they struggle with probability sampling.
  arXiv  Detail & Related papers  (2024-04-13T16:59:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.