Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions

• URL: http://arxiv.org/abs/2605.00226v1
• Date: Thu, 30 Apr 2026 21:04:38 GMT
• Title: Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions
• Abstract summary: Large language models (LLMs) are increasingly tasked with strategic decision-making under incomplete information.<n>We show two fundamental gaps in the internal mechanisms underlying the decision-making of LLMs in incomplete-information games.
• Score: 22.817330994735254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly tasked with strategic decision-making under incomplete information, such as in negotiation and policymaking. While LLMs can excel at many such tasks, they also fail in ways that are poorly understood. We shed light on these failures by uncovering two fundamental gaps in the internal mechanisms underlying the decision-making of LLMs in incomplete-information games, supported by experiments with open-weight models Llama 3.1, Qwen3, and gpt-oss. First, an observation-belief gap: LLMs encode internal beliefs about latent game states that are substantially more accurate than their own verbal reports, yet these beliefs are brittle. In particular, the belief accuracy degrades with multi-hop reasoning, exhibits primacy and recency biases, and drifts away from Bayesian coherence over extended interactions. Second, a belief-action gap: The implicit conversion of internal beliefs into actions is weaker than that of the beliefs externalized in the prompt, yet neither belief-conditioning consistently achieves higher game payoffs. These results show how analyzing LLMs' internal processes can expose systematic vulnerabilities that warrant caution before deploying LLMs in strategic domains without robust guardrails.

Related papers

• Steering LLM Viewpoints through Fabricated Evidence Injection [61.22850755490993]
  We introduce a two-phase attack framework that first repackages misleading statements with fabricated rationales, then instruct target LLMs to incorporate these viewpoints when responding to relevant queries.<n>Experiments on BBQ, ToxiGen, and our specialized dataset reveal that commercial LLMs without external safety classifiers remain highly vulnerable.<n>Building on this, we explore multiple defense strategies, among which a tailored safety policy enablesgpt-oss-safeguard to achieve 81% detection rate.
  arXiv  Detail & Related papers  (2026-06-04T14:48:23Z)
• Are Reasoning LLMs Robust to Interventions on Their Chain-of-Thought? [79.86483056611105]
  Reasoning LLMs generate step-by-step chains of thought before giving an answer.<n>How robust are these reasoning traces to disruptions that occur within them?<n>We introduce a controlled evaluation framework that perturbs a model's own CoT at fixed timesteps.
  arXiv  Detail & Related papers  (2026-02-07T10:02:58Z)
• The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence [49.94160400740222]
  We introduce MisBelief, a framework that generates misleading evidence via collaborative, multi-round interactions.<n>Using MisBelief, we generate 4,800 instances across three difficulty levels to evaluate 7 representative LLMs.<n>Results indicate that while models are robust to direct misinformation, they are highly sensitive to this refined evidence.<n>We propose Deceptive Intent Shielding (DIS), a governance mechanism that provides an early warning signal by inferring the deceptive intent behind evidence.
  arXiv  Detail & Related papers  (2026-01-09T02:28:00Z)
• Incoherent Beliefs & Inconsistent Actions in Large Language Models [33.54139088666698]
  Real-world tasks and environments exhibit differences from the static datasets that large language models (LLMs) are typically evaluated on.<n>We examine two critical components of LLM performance: the ability of LLMs to coherently update their beliefs, and the extent to which the actions they take are consistent with those beliefs.<n>Our results highlight the difficulties of predicting LLM behavior in complex real-world settings.
  arXiv  Detail & Related papers  (2025-11-17T11:04:00Z)
• Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing [63.96040994220329]
  We find that SFT-based methods, e.g., Reason-KE, suffer from a "faithfulness gap"<n>This gap enables the LLM's powerful parametric priors to override new contextual facts.<n>We propose Reason-KE++, an SFT+RL framework that instills process-level faithfulness.
  arXiv  Detail & Related papers  (2025-11-16T15:49:01Z)
• Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts [79.1081247754018]
  Large Language Models (LLMs) are widely deployed in reasoning, planning, and decision-making tasks.<n>We propose a framework based on Contact Searching Questions(CSQ) to quantify the likelihood of deception.
  arXiv  Detail & Related papers  (2025-08-08T14:46:35Z)
• How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation [35.365004091470944]
  Large Language Models (LLMs) are widely deployed in diverse scenarios.<n>The extent to which they could tacitly spread misinformation emerges as a critical safety concern.<n>We curated EchoMist, the first benchmark for implicit misinformation.
  arXiv  Detail & Related papers  (2025-03-12T17:59:18Z)
• Aligning Large Language Models for Faithful Integrity Against Opposing Argument [71.33552795870544]
  Large Language Models (LLMs) have demonstrated impressive capabilities in complex reasoning tasks.<n>They can be easily misled by unfaithful arguments during conversations, even when their original statements are correct.<n>We propose a novel framework, named Alignment for Faithful Integrity with Confidence Estimation.
  arXiv  Detail & Related papers  (2025-01-02T16:38:21Z)
• Dissecting the Ullman Variations with a SCALPEL: Why do LLMs fail at Trivial Alterations to the False Belief Task? [1.4936946857731093]
  We introduce SCALPEL -- a technique to incrementally modify stimuli to test different hypotheses about why LLMs fail.<n>Our results suggest that LLMs often do poorly because they fail to make essential common-sense inferences.<n>We conclude that while modern LLMs go beyond mere pattern matching, they still fall short of robust human-like ToM.
  arXiv  Detail & Related papers  (2024-06-20T21:02:30Z)
• GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations [87.99872683336395]
  Large Language Models (LLMs) are integrated into critical real-world applications. This paper evaluates LLMs' reasoning abilities in competitive environments. We first propose GTBench, a language-driven environment composing 10 widely recognized tasks.
  arXiv  Detail & Related papers  (2024-02-19T18:23:36Z)
• Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models [26.11408084129897]
  Large Language Models (LLMs) are deployed as powerful tools for several natural language processing (NLP) applications. Recent works show that modern LLMs can generate self-explanations (SEs), which elicit their intermediate reasoning steps for explaining their behavior. We discuss the dichotomy between faithfulness and plausibility in SEs generated by LLMs.
  arXiv  Detail & Related papers  (2024-02-07T06:32:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.