From Minds to Models: The Intersection of Psychology and LLM Behaviours

• URL: http://arxiv.org/abs/2607.27579v1
• Date: Thu, 30 Jul 2026 01:44:00 GMT
• Title: From Minds to Models: The Intersection of Psychology and LLM Behaviours
• Abstract summary: Large language models (LLMs) are often compared with the human mind because their decision-making is complex, non-linear and difficult to interpret.<n>This study tested whether ChatGPT produced sentiment differences across racial conditions in open-ended text.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are often compared with the human mind because their decision-making is complex, non-linear and difficult to interpret. Psychological methods developed to investigate unobservable mental processes may therefore help examine LLM behaviour, particularly in government and healthcare. Building on prompt-based adaptations of the Implicit Association Test, this study tested whether ChatGPT produced sentiment differences across racial conditions in open-ended text. Fourteen base questions were crossed with eight racial categories and a race-agnostic control, producing 126 prompts. Each was submitted once to GPT-3.5T, GPT-4 and GPT-4T, yielding 378 responses. Sentiment scores were derived from categorical labels and source scores: positive labels retained the source score, negative labels were assigned its negative, and neutral responses were coded zero. A two-way ANOVA found a small main effect of racial condition, F(8, 351) = 2.04, p = .042, partial-eta squared = .044, but no effect of model, F(2, 351) = 0.07, p = .933, and no interaction, F(16, 351) = 0.23, p = .999. However, the effect was not retained in a rank-transformed sensitivity analysis, F(8, 351) = 1.53, p = .145, and Tukey-corrected comparisons found no significant pairwise differences. An uncorrected European-Indigenous Australian comparison was significant, but was selected post hoc and is reported only as hypothesis-generating. Evidence for sentiment differences was therefore weak and analysis-dependent. Sentiment scoring also cannot distinguish evaluative bias from the valence of historical content elicited by a prompt. We outline design changes needed to address these limitations and argue for interdisciplinary development of behavioural measures of model bias. Keywords: Implicit Bias, Psychological Research Methods, Artificial Intelligence, ChatGPT, Large Language Models, Sentiment Analysis

Related papers

• Natural Language Processing Psychometrics [0.0]
  We treat psychological prediction from text as a psychometric problem, linking scores to interpretable linguistic evidence and testing beyond the training text format.<n>We extracted emotional profiles and syntactic-semantic structure via textual forma mentis networks, combined with personality and sociodemographic variables in ablated random forest (RF) regressors.<n>RF models explained up to 70.8% of variance in life satisfaction (SWLS), 55.7% in depression (PHQ-9), and, for DASS-21, 68.5% depression, 76.4% anxiety, 72.4% stress.
  arXiv  Detail & Related papers  (2026-08-07T15:10:57Z)
• Sentiment Classification of Gaza War Headlines: A Comparative Analysis of Large Language Models and Arabic Fine-Tuned BERT Models [0.0]
  This study examines how different artificial intelligence architectures interpret sentiment in conflict-related media discourse.<n> Drawing on a corpus of 10,990 Arabic news headlines (Eleraqi 2026), the research conducts a comparative analysis between three large language models and six fine-tuned Arabic BERT models.<n>The results reveal pronounced and non-random divergence in sentiment distributions.
  arXiv  Detail & Related papers  (2026-03-18T16:19:51Z)
• Alignment Backfire: Language-Dependent Reversal of Safety Interventions Across 16 Languages in LLM Multi-Agent Systems [0.0]
  In perpetrator treatment, offenders articulate remorse yet behavioral change does not follow.<n>We show that alignment interventions produce a structurally analogous phenomenon: surface safety that masks or generates collective pathology and internal dissociation.<n>These findings reframe alignment as a behavioral intervention subject to risk homeostasis and iatrogenesis.
  arXiv  Detail & Related papers  (2026-03-05T07:46:59Z)
• A Multifaceted Analysis of Negative Bias in Large Language Models through the Lens of Parametric Knowledge [48.00855840536793]
  Negative bias refers to the tendency of large language models to excessively generate negative responses in binary decision tasks.<n>We show that large language models exhibit format-level negative bias, meaning the prompt format more influences their responses than the semantics of the negative response.<n>We identify a shortcut behavior in which models tend to generate negative responses when they lack sufficient knowledge to answer a yes-no question.
  arXiv  Detail & Related papers  (2025-11-14T01:18:18Z)
• Discrimination by LLMs: Cross-lingual Bias Assessment and Mitigation in Decision-Making and Summarisation [0.19336815376402716]
  The rapid integration of Large Language Models (LLMs) into various domains raises concerns about societal inequalities and information bias.<n>This study examines biases in LLMs related to background, gender, and age, with a focus on their impact on decision-making and summarisation tasks.<n>Various demographic variables, instructions, salience levels, and languages were tested on GPT-3.5 and GPT-4o.
  arXiv  Detail & Related papers  (2025-09-10T14:25:09Z)
• ChatGPT for automated grading of short answer questions in mechanical ventilation [0.0]
  Large language models (LLMs) simulate conversational language and interpret unstructured free-text responses.<n>We evaluated ChatGPT 4o to grade SAQs in a postgraduate medical setting using data from 215 students.
  arXiv  Detail & Related papers  (2025-05-05T19:04:25Z)
• Trusting CHATGPT: how minor tweaks in the prompts lead to major differences in sentiment classification [0.0]
  This study tests the hypothesis that subtle changes in the structure of prompts do not produce significant variations in the classification results of sentiment polarity analysis.<n>Using a dataset of 100.000 comments in Spanish on four Latin American presidents, the model classified the comments as positive, negative, or neutral on 10 occasions, varying the prompts slightly each time.
  arXiv  Detail & Related papers  (2025-04-16T15:37:09Z)
• CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense [61.78357530675446]
  Humans are difficult to be cheated by subtle manipulations, since we make judgments only based on essential factors.<n>Inspired by this observation, we attempt to model label generation with essential label-causative factors and incorporate label-non-causative factors to assist data generation.<n>For an adversarial example, we aim to discriminate perturbations as non-causative factors and make predictions only based on the label-causative factors.
  arXiv  Detail & Related papers  (2024-10-30T15:06:44Z)
• The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models [78.69526166193236]
  Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases. We propose sc Social Bias Neurons to accurately pinpoint units (i.e., neurons) in a language model that can be attributed to undesirable behavior, such as social bias. As measured by prior metrics from StereoSet, our model achieves a higher degree of fairness while maintaining language modeling ability with low cost.
  arXiv  Detail & Related papers  (2024-06-14T15:41:06Z)
• Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis [69.07674653828565]
  Machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. In this paper, we examine the implications of spurious correlations through a novel perspective called neighborhood analysis. We propose a family of regularization methods, NFL (doN't Forget your Language) to mitigate spurious correlations in text classification.
  arXiv  Detail & Related papers  (2023-05-23T03:55:50Z)
• Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models [74.07684768317705]
  LMs are highly sensitive to markers of certainty in prompts, with accuies varying more than 80%. We find that expressions of high certainty result in a decrease in accuracy as compared to low expressions; similarly, factive verbs hurt performance, while evidentials benefit performance. These associations may suggest that LMs is based on observed language use, rather than truly reflecting uncertainty.
  arXiv  Detail & Related papers  (2023-02-26T23:46:29Z)
• Naturalistic Causal Probing for Morpho-Syntax [76.83735391276547]
  We suggest a naturalistic strategy for input-level intervention on real world data in Spanish. Using our approach, we isolate morpho-syntactic features from counfounders in sentences. We apply this methodology to analyze causal effects of gender and number on contextualized representations extracted from pre-trained models.
  arXiv  Detail & Related papers  (2022-05-14T11:47:58Z)
• Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests [87.60900567941428]
  A spurious correlation' is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character. We study stress testing using the tools of causal inference.
  arXiv  Detail & Related papers  (2021-05-31T14:39:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.