Machine individuality: Separating genuine idiosyncrasy from response bias in large language models

• URL: http://arxiv.org/abs/2604.16755v2
• Date: Tue, 21 Apr 2026 13:23:23 GMT
• Title: Machine individuality: Separating genuine idiosyncrasy from response bias in large language models
• Abstract summary: Large language models (LLMs) are increasingly integrated into daily life, in roles ranging from high-stakes decision support to companionship.<n>Here, we apply random crossed models to 74.9 million ratings provided by 10 open-weight LLMs for over 100,000 words across 14 psycholinguistic norms.<n>On average, 16.9% of variance is attributable to stimulus-specific individuality, robustly exceeding a statistical null model.
• Score: 1.4323566945483497
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models (LLMs) are increasingly integrated into daily life, in roles ranging from high-stakes decision support to companionship, understanding their behavioral dispositions becomes critical. A growing literature uses psychometric inventories and cognitive paradigms to profile LLM dispositions. However, these approaches cannot determine whether behavioral differences reflect stable, stimulus-specific individuality or global response biases and stochastic noise. Here, we apply crossed random-effects models -- widely used in psychometrics to separate systematic effects -- to 74.9 million ratings provided by 10 open-weight LLMs for over 100,000 words across 14 psycholinguistic norms. On average, 16.9% of variance is attributable to stimulus-specific individuality, robustly exceeding a statistical null model. Cross-norm prediction analyses reveal this individuality as a coherent fingerprint, unique to each model. These results identify individual differences among LLMs that cannot be attributed to response biases or stochastic noise. We term these differences machine individuality.

Related papers

• Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact [1.3277433989254843]
  Psychological instruments designed for humans are increasingly used to assign large language models (LLMs) stable psychological profiles.<n>Using a formal psychometric framework, we show that these profiles are largely a measurement artifact.
  arXiv  Detail & Related papers  (2026-06-18T13:18:37Z)
• The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models [12.654067061944074]
  Applications based on large language models (LLMs) require population diversity among agents.<n>We identify a pervasive failure mode we term emphPersona Collapse: agents each assigned a distinct profile nonetheless converge into a narrow behavioral mode, producing a homogeneous simulated population.
  arXiv  Detail & Related papers  (2026-04-27T17:01:48Z)
• From Five Dimensions to Many: Large Language Models as Precise and Interpretable Psychological Profilers [14.983442449498739]
  This study investigates whether and how Large Language Models can model the correlational structure of human psychological traits from minimal quantitative inputs.<n>We prompted various LLMs with Big Five Personality Scale responses from 816 human individuals to role-play their responses on nine other psychological scales.<n>LLMs demonstrated remarkable accuracy in capturing human psychological structure.
  arXiv  Detail & Related papers  (2025-11-05T06:51:13Z)
• Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond) [90.45301024940329]
  Language models (LMs) often struggle to generate diverse, human-like creative content.<n>We introduce Infinity-Chat, a large-scale dataset of 26K diverse, real-world, open-ended user queries.<n>We present a large-scale study of mode collapse in LMs, revealing a pronounced Artificial Hivemind effect.
  arXiv  Detail & Related papers  (2025-10-27T03:16:21Z)
• The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs [60.15472325639723]
  Personality traits have long been studied as predictors of human behavior.<n>Recent advances in Large Language Models (LLMs) suggest similar patterns may emerge in artificial systems.
  arXiv  Detail & Related papers  (2025-09-03T21:27:10Z)
• IROTE: Human-like Traits Elicitation of Large Language Model via In-Context Self-Reflective Optimization [66.6349183886101]
  We propose IROTE, a novel in-context method for stable and transferable trait elicitation.<n>We show that one single IROTE-generated self-reflection can induce LLMs' stable impersonation of the target trait across diverse downstream tasks.
  arXiv  Detail & Related papers  (2025-08-12T08:04:28Z)
• Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation History [7.58175460763641]
  Even 400B+ models exhibit substantial response variability.<n> Interventions expected to stabilize behavior, such as chain-of-thought reasoning, detailed personas instruction, inclusion of conversation history, can paradoxically increase variability.<n>For safety-critical applications requiring predictable behavior, these findings indicate that personality-based alignment strategies may be fundamentally inadequate.
  arXiv  Detail & Related papers  (2025-08-06T19:11:33Z)
• Evaluating the Simulation of Human Personality-Driven Susceptibility to Misinformation with LLMs [0.18416014644193066]
  Large language models (LLMs) make it possible to generate synthetic behavioural data at scale.<n>We evaluate the capacity of LLM agents, conditioned on Big-Five profiles, to reproduce personality-based variation in susceptibility to misinformation.
  arXiv  Detail & Related papers  (2025-06-30T08:16:07Z)
• Can LLMs Simulate Human Behavioral Variability? A Case Study in the Phonemic Fluency Task [0.0]
  Large language models (LLMs) are increasingly explored as substitutes for human participants in cognitive tasks.<n>This study examines whether LLMs can approximate individual differences in the phonemic fluency task.
  arXiv  Detail & Related papers  (2025-05-22T03:08:27Z)
• Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes [50.544186914115045]
  Large language models (LLMs) are increasingly embedded in everyday applications.<n> Ensuring their alignment with the diverse preferences of individual users has become a critical challenge.<n>We present a novel framework for few-shot steerable alignment.
  arXiv  Detail & Related papers  (2024-12-18T16:14:59Z)
• When Prompting Fails to Sway: Inertia in Moral and Value Judgments of Large Language Models [4.906478894661688]
  Large Language Models (LLMs) exhibit non-deterministic behavior, and prompting has emerged as a primary method for steering their outputs toward desired directions.<n>One popular strategy involves assigning a specific "persona" to the model to induce more varied and context-sensitive responses, akin to the diversity found in human perspectives.<n>Our experiments demonstrate that LLMs maintain consistent value orientations.<n>In particular, we observe a persistent inertia in their responses, where certain moral and value dimensions, especially harm avoidance and fairness, remain distinctly skewed in one direction despite varied persona settings.
  arXiv  Detail & Related papers  (2024-08-16T23:24:10Z)
• Evaluating Large Language Models with Psychometrics [59.821829073478376]
  This paper offers a comprehensive benchmark for quantifying psychological constructs of Large Language Models (LLMs)<n>Our work identifies five key psychological constructs -- personality, values, emotional intelligence, theory of mind, and self-efficacy -- assessed through a suite of 13 datasets.<n>We uncover significant discrepancies between LLMs' self-reported traits and their response patterns in real-world scenarios, revealing complexities in their behaviors.
  arXiv  Detail & Related papers  (2024-06-25T16:09:08Z)
• Zero-shot causal learning [64.9368337542558]
  CaML is a causal meta-learning framework which formulates the personalized prediction of each intervention's effect as a task. We show that CaML is able to predict the personalized effects of novel interventions that do not exist at the time of training.
  arXiv  Detail & Related papers  (2023-01-28T20:14:11Z)
• Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization? [90.35044668396591]
  A recurring theme in machine learning is algorithmic monoculture: the same systems, or systems that share components, are deployed by multiple decision-makers. We propose the component-sharing hypothesis: if decision-makers share components like training data or specific models, then they will produce more homogeneous outcomes. We test this hypothesis on algorithmic fairness benchmarks, demonstrating that sharing training data reliably exacerbates homogenization. We conclude with philosophical analyses of and societal challenges for outcome homogenization, with an eye towards implications for deployed machine learning systems.
  arXiv  Detail & Related papers  (2022-11-25T09:33:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.