Language models suffer from a curse of ambiguity

• URL: http://arxiv.org/abs/2608.15448v1
• Date: Sat, 15 Aug 2026 23:22:01 GMT
• Title: Language models suffer from a curse of ambiguity
• Abstract summary: Large language models increasingly rely on sampling as a driver of their own improvement.<n>In this work, we identify a curse of ambiguity: in large language models, the more ambiguous a next-token distribution is, the harder it is to learn accurately.
• Score: 4.7682723427536144
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever. Yet, not all distributions are equally easy to learn. In this work, we identify a curse of ambiguity: in large language models, and more broadly in all neural networks that produce discrete probability distributions, the more ambiguous a next-token distribution is, the harder it is to learn accurately. Through an extensive theoretical analysis, we trace this curse to architectural and learning roots. More ambiguous distributions require more capacity to be stored, larger embeddings to be represented, more steps to be fitted, and amplify token-sampling noise. We validate these findings on synthetic tasks with controlled ground truth and observe the same signatures in language models trained on real data. Our results provide a new perspective on the statistical capabilities of large language models and a practical framework for when to trust their output distribution.

Related papers

• Shape of Thought: When Distribution Matters More than Correctness in Reasoning Tasks [24.55929874173401]
  We show that a language model's reasoning capabilities can be improved by training on datasets of chain-of-thought traces from more capable models.<n>Experiments show this approach can yield better performance on reasoning tasks than training on human-annotated datasets.
  arXiv  Detail & Related papers  (2025-12-24T07:35:55Z)
• Emergence of Linear Truth Encodings in Language Models [64.86571541830598]
  Large language models exhibit linear subspaces that separate true from false statements, yet the mechanism behind their emergence is unclear.<n>We introduce a transparent, one-layer transformer toy model that reproduces such truth subspaces end-to-end.<n>We study one simple setting in which truth encoding can emerge, encouraging the model to learn this distinction in order to lower the LM loss on future tokens.
  arXiv  Detail & Related papers  (2025-10-17T16:30:07Z)
• Collapsed Language Models Promote Fairness [88.48232731113306]
  We find that debiased language models exhibit collapsed alignment between token representations and word embeddings.<n>We design a principled fine-tuning method that can effectively improve fairness in a wide range of debiasing methods.
  arXiv  Detail & Related papers  (2024-10-06T13:09:48Z)
• What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages [78.1866280652834]
  Large language models (LM) are distributions over strings.<n>We investigate the learnability of regular LMs (RLMs) by RNN and Transformer LMs.<n>We find that the complexity of the RLM rank is strong and significant predictors of learnability for both RNNs and Transformers.
  arXiv  Detail & Related papers  (2024-06-06T17:34:24Z)
• Forcing Diffuse Distributions out of Language Models [70.28345569190388]
  Despite being trained specifically to follow user instructions, today's instructiontuned language models perform poorly when instructed to produce random outputs. We propose a fine-tuning method that encourages language models to output distributions that are diffuse over valid outcomes.
  arXiv  Detail & Related papers  (2024-04-16T19:17:23Z)
• Distinguishing the Knowable from the Unknowable with Language Models [15.471748481627143]
  In the absence of ground-truth probabilities, we explore a setting where, in order to disentangle a given uncertainty, a significantly larger model stands in as a proxy for the ground truth. We show that small linear probes trained on the embeddings of frozen, pretrained models accurately predict when larger models will be more confident at the token level. We propose a fully unsupervised method that achieves non-trivial accuracy on the same task.
  arXiv  Detail & Related papers  (2024-02-05T22:22:49Z)
• Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning [52.22611035186903]
  We show that scaling diffusion language models can effectively make them strong language learners.<n>We build competent diffusion language models at scale by first acquiring knowledge from massive data.<n>Experiments show that scaling diffusion language models consistently improves performance across downstream language tasks.
  arXiv  Detail & Related papers  (2023-08-23T16:01:12Z)
• A Latent Space Theory for Emergent Abilities in Large Language Models [5.033924641692716]
  We show that languages are not created randomly but rather to communicate information. A strong association between languages and their underlying meanings, resulting in a sparse joint distribution. With the advent of LLMs trained on big data and large models, we can now precisely assess the marginal distribution of languages.
  arXiv  Detail & Related papers  (2023-04-19T20:45:01Z)
• Lexical Generalization Improves with Larger Models and Longer Training [42.024050065980845]
  We analyze the use of lexical overlaps in natural language inference, paraphrase detection, and reading comprehension. We find that larger models are much less susceptible to adopting lexical overlaps.
  arXiv  Detail & Related papers  (2022-10-23T09:20:11Z)
• Distributionally Robust Recurrent Decoders with Random Network Distillation [93.10261573696788]
  We propose a method based on OOD detection with Random Network Distillation to allow an autoregressive language model to disregard OOD context during inference. We apply our method to a GRU architecture, demonstrating improvements on multiple language modeling (LM) datasets.
  arXiv  Detail & Related papers  (2021-10-25T19:26:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.