GPT-4o Lacks Core Features of Theory of Mind

• URL: http://arxiv.org/abs/2602.12150v2
• Date: Fri, 13 Feb 2026 19:21:26 GMT
• Title: GPT-4o Lacks Core Features of Theory of Mind
• Authors: John Muchovej, Amanda Royka, Shane Lee, Julian Jara-Ettinger,
• Abstract summary: We use a cognitively-grounded definition of ToM to develop and test a new evaluation framework.<n>We find that even though LLMs succeed in approxing human judgments in a simple ToM paradigm, they fail at a logically equivalent task.
• Score: 0.09320657506524145
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Do Large Language Models (LLMs) possess a Theory of Mind (ToM)? Research into this question has focused on evaluating LLMs against benchmarks and found success across a range of social tasks. However, these evaluations do not test for the actual representations posited by ToM: namely, a causal model of mental states and behavior. Here, we use a cognitively-grounded definition of ToM to develop and test a new evaluation framework. Specifically, our approach probes whether LLMs have a coherent, domain-general, and consistent model of how mental states cause behavior -- regardless of whether that model matches a human-like ToM. We find that even though LLMs succeed in approximating human judgments in a simple ToM paradigm, they fail at a logically equivalent task and exhibit low consistency between their action predictions and corresponding mental state inferences. As such, these findings suggest that the social proficiency exhibited by LLMs is not the result of a domain-general or consistent ToM.

Related papers

• SocialEval: Evaluating Social Intelligence of Large Language Models [70.90981021629021]
  Social Intelligence (SI) equips humans with interpersonal abilities to behave wisely in navigating social interactions to achieve social goals.<n>This presents an operational evaluation paradigm: outcome-oriented goal achievement evaluation and process-oriented interpersonal ability evaluation.<n>We propose SocialEval, a script-based bilingual SI benchmark, integrating outcome- and process-oriented evaluation by manually crafting narrative scripts.
  arXiv  Detail & Related papers  (2025-06-01T08:36:51Z)
• Effects of Theory of Mind and Prosocial Beliefs on Steering Human-Aligned Behaviors of LLMs in Ultimatum Games [17.615991993062455]
  We investigate the role of theory-of-mind (ToM) reasoning in aligning agentic behaviors with human norms in negotiation tasks.<n>ToM reasoning enhances behavior alignment, decision-making consistency, and negotiation outcomes.<n>Our findings contribute to the understanding of ToM's role in enhancing human-AI interaction and cooperative decision-making.
  arXiv  Detail & Related papers  (2025-05-30T06:23:52Z)
• Rethinking Theory of Mind Benchmarks for LLMs: Towards A User-Centered Perspective [24.27038998164743]
  Theory-of-Mind (ToM) tasks are designed for humans to benchmark LLM's ToM capabilities.<n>This approach has a number of limitations.<n>Taking a human-computer interaction (HCI) perspective, these limitations prompt us to rethink the definition and criteria of ToM in ToM benchmarks.
  arXiv  Detail & Related papers  (2025-04-15T03:44:43Z)
• Re-evaluating Theory of Mind evaluation in large language models [3.262532929657758]
  We take inspiration from cognitive science to re-evaluate the state of ToM evaluation in large language models.<n>A major reason for the disagreement on whether LLMs have ToM is a lack of clarity on whether models should be expected to match human behaviors.<n>We conclude by discussing several directions for future research, including the relationship between ToM and pragmatic communication.
  arXiv  Detail & Related papers  (2025-02-28T14:36:57Z)
• Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models [76.6028674686018]
  We introduce thought-tracing, an inference-time reasoning algorithm to trace the mental states of agents.<n>Our algorithm is modeled after the Bayesian theory-of-mind framework.<n>We evaluate thought-tracing on diverse theory-of-mind benchmarks, demonstrating significant performance improvements.
  arXiv  Detail & Related papers  (2025-02-17T15:08:50Z)
• Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models [51.91448005607405]
  We evaluate key human ToM precursors by annotating characters' perceptions on ToMi and FANToM. We present PercepToM, a novel ToM method leveraging LLMs' strong perception inference capability while supplementing their limited perception-to-belief inference.
  arXiv  Detail & Related papers  (2024-07-08T14:58:29Z)
• ToMBench: Benchmarking Theory of Mind in Large Language Models [41.565202027904476]
  ToM is the cognitive capability to perceive and ascribe mental states to oneself and others.<n>Existing ToM evaluations are hindered by challenges such as constrained scope, subjective judgment, and unintended contamination.<n>We introduce ToMBench with three key characteristics: a systematic evaluation framework encompassing 8 tasks and 31 abilities in social cognition, a multiple-choice question format to support automated and unbiased evaluation, and a build-from-scratch bilingual inventory to strictly avoid data leakage.
  arXiv  Detail & Related papers  (2024-02-23T02:05:46Z)
• CLOMO: Counterfactual Logical Modification with Large Language Models [109.60793869938534]
  We introduce a novel task, Counterfactual Logical Modification (CLOMO), and a high-quality human-annotated benchmark. In this task, LLMs must adeptly alter a given argumentative text to uphold a predetermined logical relationship. We propose an innovative evaluation metric, the Self-Evaluation Score (SES), to directly evaluate the natural language output of LLMs.
  arXiv  Detail & Related papers  (2023-11-29T08:29:54Z)
• FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions [94.61530480991627]
  Theory of mind evaluations currently focus on testing models using passive narratives that inherently lack interactivity. We introduce FANToM, a new benchmark designed to stress-test ToM within information-asymmetric conversational contexts via question answering.
  arXiv  Detail & Related papers  (2023-10-24T00:24:11Z)
• Understanding Social Reasoning in Language Models with Language Models [34.068368860882586]
  We present a novel framework for generating evaluations with Large Language Models (LLMs) by populating causal templates. We create a new social reasoning benchmark (BigToM) for LLMs which consists of 25 controls and 5,000 model-written evaluations. We find that human participants rate the quality of our benchmark higher than previous crowd-sourced evaluations and comparable to expert-written evaluations.
  arXiv  Detail & Related papers  (2023-06-21T16:42:15Z)
• Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models [82.50173296858377]
  Many anecdotal examples were used to suggest newer large language models (LLMs) like ChatGPT and GPT-4 exhibit Neural Theory-of-Mind (N-ToM) We investigate the extent of LLMs' N-ToM through an extensive evaluation on 6 tasks and find that while LLMs exhibit certain N-ToM abilities, this behavior is far from being robust.
  arXiv  Detail & Related papers  (2023-05-24T06:14:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.