Modeling the human lexicon under temperature variations: linguistic factors, diversity and typicality in LLM word associations

• URL: http://arxiv.org/abs/2603.18171v1
• Date: Wed, 18 Mar 2026 18:10:02 GMT
• Title: Modeling the human lexicon under temperature variations: linguistic factors, diversity and typicality in LLM word associations
• Abstract summary: This study compares human and large language models (LLMs)<n>We examine the influence of lexical factors such as word frequency and concreteness on cue-response pairs.<n>Results show that all models mirror human trends for frequency and concreteness but differ in response variability and typicality.
• Score: 2.3950779725796765
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large language models (LLMs) achieve impressive results in terms of fluency in text generation, yet the nature of their linguistic knowledge - in particular the human-likeness of their internal lexicon - remains uncertain. This study compares human and LLM-generated word associations to evaluate how accurately models capture human lexical patterns. Using English cue-response pairs from the SWOW dataset and newly generated associations from three LLMs (Mistral-7B, Llama-3.1-8B, and Qwen-2.5-32B) across multiple temperature settings, we examine (i) the influence of lexical factors such as word frequency and concreteness on cue-response pairs, and (ii) the variability and typicality of LLM responses compared to human responses. Results show that all models mirror human trends for frequency and concreteness but differ in response variability and typicality. Larger models such as Qwen tend to emulate a single "prototypical" human participant, generating highly typical but minimally variable responses, while smaller models such as Mistral and Llama produce more variable yet less typical responses. Temperature settings further influence this trade-off, with higher values increasing variability but decreasing typicality. These findings highlight both the similarities and differences between human and LLM lexicons, emphasizing the need to account for model size and temperature when probing LLM lexical representations.

Related papers

• Evaluating LLMs as Human Surrogates in Controlled Experiments [4.233590639227564]
  Large language models (LLMs) are increasingly used to simulate human responses in behavioral research.<n>It remains unclear when LLM-generated data support the same experimental inferences as human data.<n>We evaluate this by comparing off-the-shelf LLM-generated responses with human responses from a canonical survey experiment on accuracy perception.
  arXiv  Detail & Related papers  (2026-03-08T16:03:27Z)
• Who are you, ChatGPT? Personality and Demographic Style in LLM-Generated Content [5.515596385935823]
  Generative large language models (LLMs) have become central to everyday life, producing human-like text across diverse domains.<n>A growing body of research investigates whether these models also exhibit personality- and demographic-like characteristics in their language.<n>We introduce a novel, data-driven methodology for assessing LLM personality without relying on self-report questionnaires.<n>Applying instead automatic personality and gender classifiers to model replies on open-ended questions collected from Reddit.
  arXiv  Detail & Related papers  (2025-10-13T14:06:17Z)
• Can LLMs Simulate Human Behavioral Variability? A Case Study in the Phonemic Fluency Task [0.0]
  Large language models (LLMs) are increasingly explored as substitutes for human participants in cognitive tasks.<n>This study examines whether LLMs can approximate individual differences in the phonemic fluency task.
  arXiv  Detail & Related papers  (2025-05-22T03:08:27Z)
• Leveraging Human Production-Interpretation Asymmetries to Test LLM Cognitive Plausibility [7.183662547358301]
  We examine whether large language models process language similarly to humans.<n>We find that some LLMs do quantitatively and qualitatively reflect human-like asymmetries between production and interpretation.
  arXiv  Detail & Related papers  (2025-03-21T23:25:42Z)
• ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models [75.05436691700572]
  We introduce ExpliCa, a new dataset for evaluating Large Language Models (LLMs) in explicit causal reasoning.<n>We tested seven commercial and open-source LLMs on ExpliCa through prompting and perplexity-based metrics.<n>Surprisingly, models tend to confound temporal relations with causal ones, and their performance is also strongly influenced by the linguistic order of the events.
  arXiv  Detail & Related papers  (2025-02-21T14:23:14Z)
• Large Language Models as Neurolinguistic Subjects: Discrepancy between Performance and Competence [49.60849499134362]
  This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning)<n>We introduce a neurolinguistic approach, utilizing a novel method that combines minimal pair and diagnostic probing to analyze activation patterns across model layers.<n>We found: (1) Psycholinguistic and neurolinguistic methods reveal that language performance and competence are distinct; (2) Direct probability measurement may not accurately assess linguistic competence; and (3) Instruction tuning won't change much competence but improve performance.
  arXiv  Detail & Related papers  (2024-11-12T04:16:44Z)
• One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversity [2.5975241792179378]
  Researchers have proposed using large language models (LLMs) as replacements for humans in behavioral research.<n>It is debated whether post-training alignment (RLHF or RLAIF) affects models' internal diversity.<n>We use a new way of measuring the conceptual diversity of synthetically-generated LLM "populations" by relating the internal variability of simulated individuals to the population-level variability.
  arXiv  Detail & Related papers  (2024-11-07T04:38:58Z)
• What Goes Into a LM Acceptability Judgment? Rethinking the Impact of Frequency and Length [61.71625297655583]
  We show that MORCELA outperforms a commonly used linking theory for acceptability.<n>Larger models require a lower relative degree of adjustment for unigram frequency.<n>Our analysis shows that larger LMs' lower susceptibility to frequency effects can be explained by an ability to better predict rarer words in context.
  arXiv  Detail & Related papers  (2024-11-04T19:05:49Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• High-Dimension Human Value Representation in Large Language Models [60.33033114185092]
  We propose UniVaR, a high-dimensional neural representation of symbolic human value distributions in LLMs.<n>This is a continuous and scalable representation, self-supervised from the value-relevant output of 8 LLMs.<n>We explore how LLMs prioritize different values in 25 languages and cultures, shedding light on complex interplay between human values and language modeling.
  arXiv  Detail & Related papers  (2024-04-11T16:39:00Z)
• Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment [84.32768080422349]
  Alignment with human preference prevents large language models from generating misleading or toxic content. We propose a new formulation of prompt diversity, implying a linear correlation with the final performance of LLMs after fine-tuning.
  arXiv  Detail & Related papers  (2024-03-17T07:08:55Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• Multi-timescale Representation Learning in LSTM Language Models [69.98840820213937]
  Language models must capture statistical dependencies between words at timescales ranging from very short to very long. We derived a theory for how the memory gating mechanism in long short-term memory language models can capture power law decay. Experiments showed that LSTM language models trained on natural English text learn to approximate this theoretical distribution.
  arXiv  Detail & Related papers  (2020-09-27T02:13:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.