Greedy or not, here I come: Language production under vocabulary constraints in humans and resource-rational models

• URL: http://arxiv.org/abs/2605.15365v1
• Date: Thu, 14 May 2026 19:45:02 GMT
• Title: Greedy or not, here I come: Language production under vocabulary constraints in humans and resource-rational models
• Abstract summary: We investigate how humans respond to a broad array of questions under variable vocabulary limitations, consisting of only 250 highly frequent words at the most restrictive.<n>We compare greedy and globally optimal sampling algorithms using Sequential Monte Carlo inference with large language models.<n>We discuss the results and their broader implications for resource-rational cognition, psycholinguistics, L2 communication, and language impairments.
• Score: 6.407877420659298
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Communicating using only a limited vocabulary is a common but challenging cognitive phenomenon, requiring an ideal communicator to plan carefully to optimize for intelligibility while circumventing a constrained lexicon. In this work, we investigate how humans respond to a broad array of questions under variable vocabulary limitations, consisting of only 250 highly frequent words at the most restrictive. We provide theoretically motivated comparisons to greedy and globally optimal sampling algorithms using Sequential Monte Carlo inference with large language models. Humans generally resemble greedy sampling more than globally optimal sampling, though more skilled humans are more likely to backtrack and revise -- a non-greedy behavior. An observed human pattern of leaning on semantically light words in high-constraint settings falls out of both greedy and globally optimal sampling. We discuss the results and their broader implications for resource-rational cognition, psycholinguistics, L2 communication, and language impairments.

Related papers

• How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Framework [0.47954782871066737]
  We propose a context-aware evaluation framework in which human-likeness is assessed.<n>We implement this framework using the Maximum Mean Discrepancy (MMD) and the 67 lexico-grammatical features introduced by Biber.
  arXiv  Detail & Related papers  (2026-05-22T14:04:25Z)
• A suite of LMs comprehend puzzle statements as well as humans [13.386647125288516]
  We report a preregistered study comparing human responses in two conditions: one allowed rereading, and one that restricted rereading.<n>Human accuracy dropped significantly when rereading was restricted, falling below that of Falcon-180B-Chat and GPT-4.<n>Results suggest shared pragmatic sensitivities rather than model-specific deficits.
  arXiv  Detail & Related papers  (2025-05-13T22:18:51Z)
• Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling [90.86991492288487]
  evaluating constraint on every token can be prohibitively expensive.<n> LCD can distort the global distribution over strings, sampling tokens based only on local information.<n>We show that our approach is superior to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-07T18:30:18Z)
• Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning [72.21876989058858]
  We present a model of pragmatic language understanding, where utterances are produced and understood by searching for regularized equilibria of signaling games. In this model speakers and listeners search for contextually appropriate utterance--meaning mappings that are both close to game-theoretically optimal conventions and close to a shared, ''default'' semantics.
  arXiv  Detail & Related papers  (2023-11-16T09:42:36Z)
• UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations [62.71847873326847]
  We investigate the ability to model unusual, unexpected, and unlikely situations. Given a piece of context with an unexpected outcome, this task requires reasoning abductively to generate an explanation. We release a new English language corpus called UNcommonsense.
  arXiv  Detail & Related papers  (2023-11-14T19:00:55Z)
• A fine-grained comparison of pragmatic language understanding in humans and language models [2.231167375820083]
  We compare language models and humans on seven pragmatic phenomena. We find that the largest models achieve high accuracy and match human error patterns. Preliminary evidence that models and humans are sensitive to similar linguistic cues.
  arXiv  Detail & Related papers  (2022-12-13T18:34:59Z)
• Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks [39.39138995087475]
  We ask how much of human-like thinking can be captured by learning statistical patterns in language alone. Our benchmark contains two problem-solving domains (planning and explanation generation) and is designed to require generalization. We find that humans are far more robust than LLMs on this benchmark.
  arXiv  Detail & Related papers  (2022-05-11T18:14:33Z)
• Testing the limits of natural language models for predicting human language judgments [0.27309692684728604]
  We compared the model-human consistency of diverse language models using controversial sentence pairs. Controversial sentence pairs proved highly effective at revealing model failures. The most human-consistent model tested was GPT-2, although experiments also revealed significant shortcomings of its alignment with human perception.
  arXiv  Detail & Related papers  (2022-04-07T17:12:57Z)
• Locally Typical Sampling [84.62530743899025]
  We show that today's probabilistic language generators fall short when it comes to producing coherent and fluent text.<n>We propose a simple and efficient procedure for enforcing this criterion when generating from probabilistic models.
  arXiv  Detail & Related papers  (2022-02-01T18:58:45Z)
• Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training [59.571632468137075]
  We find that many languages are under-represented in recent cross-lingual language models due to the limited vocabulary capacity. We propose an algorithm VoCap to determine the desired vocabulary capacity of each language. In order to address the issues, we propose k-NN-based target sampling to accelerate the expensive softmax.
  arXiv  Detail & Related papers  (2021-09-15T14:04:16Z)
• Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning [69.1137074774244]
  Leveraging language interactions effectively requires addressing limitations in the two most common approaches to language grounding. We introduce the idea of neural abstructions: a set of constraints on the inference procedure of a label-conditioned generative model. We show that with this method a user population is able to build a semantic modification for an open-ended house task in Minecraft.
  arXiv  Detail & Related papers  (2021-07-20T07:01:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.