LLM Agents Display Human Biases but Exhibit Distinct Learning Patterns

• URL: http://arxiv.org/abs/2503.10248v1
• Date: Thu, 13 Mar 2025 10:47:03 GMT
• Title: LLM Agents Display Human Biases but Exhibit Distinct Learning Patterns
• Authors: Idan Horowitz, Ori Plonsky,
• Abstract summary: We investigate the choice patterns of Large Language Models (LLMs) in the context of Decisions from Experience tasks.<n>We find that on the aggregate, LLMs appear to display behavioral biases similar to humans.<n>However, more nuanced analyses of the choice patterns reveal that this happens for very different reasons.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate the choice patterns of Large Language Models (LLMs) in the context of Decisions from Experience tasks that involve repeated choice and learning from feedback, and compare their behavior to human participants. We find that on the aggregate, LLMs appear to display behavioral biases similar to humans: both exhibit underweighting rare events and correlation effects. However, more nuanced analyses of the choice patterns reveal that this happens for very different reasons. LLMs exhibit strong recency biases, unlike humans, who appear to respond in more sophisticated ways. While these different processes may lead to similar behavior on average, choice patterns contingent on recent events differ vastly between the two groups. Specifically, phenomena such as ``surprise triggers change" and the ``wavy recency effect of rare events" are robustly observed in humans, but entirely absent in LLMs. Our findings provide insights into the limitations of using LLMs to simulate and predict humans in learning environments and highlight the need for refined analyses of their behavior when investigating whether they replicate human decision making tendencies.

Related papers

• If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs [55.8331366739144]
  We introduce LIFESTATE-BENCH, a benchmark designed to assess lifelong learning in large language models (LLMs) Our fact checking evaluation probes models' self-awareness, episodic memory retrieval, and relationship tracking, across both parametric and non-parametric approaches.
  arXiv  Detail & Related papers  (2025-03-30T16:50:57Z)
• Cognitive Biases in Large Language Models: A Survey and Mitigation Experiments [24.15688619889342]
  Large Language Models (LLMs) are trained on large corpora written by humans and demonstrate high performance on various tasks. As humans are susceptible to cognitive biases, LLMs can also be influenced by these biases, leading to irrational decision-making.
  arXiv  Detail & Related papers  (2024-11-30T02:37:59Z)
• How far can bias go? -- Tracing bias from pretraining data to alignment [54.51310112013655]
  This study examines the correlation between gender-occupation bias in pre-training data and their manifestation in LLMs. Our findings reveal that biases present in pre-training data are amplified in model outputs.
  arXiv  Detail & Related papers  (2024-11-28T16:20:25Z)
• AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment [37.985947029716016]
  Large language models (LLMs) have shown advanced understanding capabilities but may inherit human biases from their training data. We investigated whether LLMs are influenced by the threshold priming effect in relevance judgments.
  arXiv  Detail & Related papers  (2024-09-24T12:23:15Z)
• Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personas [14.650234624251716]
  Large language models (LLMs) are increasingly being used in human-centered social scientific tasks. These tasks are highly subjective and dependent on human factors, such as one's environment, attitudes, beliefs, and lived experiences. We examine the role of prompting LLMs with human-like personas and ask the models to answer as if they were a specific human.
  arXiv  Detail & Related papers  (2024-06-20T16:24:07Z)
• Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment [84.32768080422349]
  Alignment with human preference prevents large language models from generating misleading or toxic content. We propose a new formulation of prompt diversity, implying a linear correlation with the final performance of LLMs after fine-tuning.
  arXiv  Detail & Related papers  (2024-03-17T07:08:55Z)
• Exploring Value Biases: How LLMs Deviate Towards the Ideal [57.99044181599786]
  Large-Language-Models (LLMs) are deployed in a wide range of applications, and their response has an increasing social impact. We show that value bias is strong in LLMs across different categories, similar to the results found in human studies.
  arXiv  Detail & Related papers  (2024-02-16T18:28:43Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks [49.60689355674541]
  A rich literature in cognitive science has studied people's causal and moral intuitions. This work has revealed a number of factors that systematically influence people's judgments. We test whether large language models (LLMs) make causal and moral judgments about text-based scenarios that align with human participants.
  arXiv  Detail & Related papers  (2023-10-30T15:57:32Z)
• Studying and improving reasoning in humans and machines [0.0]
  We investigate and compare reasoning in large language models (LLM) and humans. Our results show that most of the included models presented reasoning errors akin to those frequently ascribed to error-prone, induce-based human reasoning.
  arXiv  Detail & Related papers  (2023-09-21T21:02:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.