Tracing the ongoing emergence of human-like reasoning in Large Language Models

• URL: http://arxiv.org/abs/2605.21299v1
• Date: Wed, 20 May 2026 15:28:52 GMT
• Title: Tracing the ongoing emergence of human-like reasoning in Large Language Models
• Abstract summary: Large Language Models show human-like performance on many tasks.<n>It remains unclear whether they reason like humans.<n>We find that humans enrich logical reasoning through pragmatic inferences across languages.
• Score: 0.13048920509133807
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans effortlessly go beyond literal meanings: If you mow the lawn, I will give you fifty dollars, is typically understood as implying that the speaker will pay only if the lawn is mowed, whereas If you are hungry, there is pizza in the oven implies that pizza is available regardless of the hearers hunger. Large Language Models - LLMs - show human-like performance on many tasks, yet it remains unclear whether they reason like humans. To address this, we conducted a population-matching experiment assessing how twentyfive LLMs compute conditional inferences across four languages, compared to an equal number of humans per language. We find that humans enrich logical reasoning through pragmatic inferences across languages. Model behavior is more variable. Some LLMs perfectly follow the truth-table of conditionals but they ignore pragmatic inferences, while others deviate from the truth-table, adhering to a single interpretation across the board, thus reflecting accurate rule-based processing but not human-like reasoning. Overall, LLMs are accurate semantic operators, but fail to capture the pragmatic enrichments characteristic of human reasoning. Crucially, LLM accuracy is neither predicted nor boosted by open vs. closed status, training orientation, or architecture type, suggesting that pragmatic reasoning is still an emerging ability in the cognitive toolkit of artificial systems.

Related papers

• Chatbots Output Meaningful (but Problematic) Language [1.246150324257064]
  Most ordinary users, as well as AI engineers, take the answer to be trivially "yes"<n>But many cognitive scientists, linguists, and philosophers of language argue that dominant intentionalist accounts of language and meaning deliver the opposite conclusion.<n>Theorists more sympathetic to ordinary users' intuitions have therefore advocated a radical "de-anthropomorphization" of language.
  arXiv  Detail & Related papers  (2026-06-02T00:24:42Z)
• On the Thinking-Language Modeling Gap in Large Language Models [68.83670974539108]
  We show that there is a significant gap between the modeling of languages and thoughts.<n>We propose a new prompt technique termed Language-of-Thoughts (LoT) to demonstrate and alleviate this gap.
  arXiv  Detail & Related papers  (2025-05-19T09:31:52Z)
• Intermediate Languages Matter: Formal Choice Drives Neurosymbolic LLM Reasoning [50.99811144731619]
  We show that the choice of formal language affects both the syntactic and the semantic reasoning capability.<n>We conclude that on average, context-aware encodings help LLMs to reason, while there is no apparent effect of using comments or markdown syntax.
  arXiv  Detail & Related papers  (2025-02-24T14:49:52Z)
• Non-literal Understanding of Number Words by Language Models [33.24263583093367]
  Humans naturally interpret numbers non-literally, combining context, world knowledge, and speaker intent.<n>We investigate whether large language models (LLMs) interpret numbers similarly, focusing on hyperbole and pragmatic halo effects.
  arXiv  Detail & Related papers  (2025-02-10T07:03:00Z)
• Are LLMs good pragmatic speakers? [3.4113474745671923]
  Large language models (LLMs) are trained on data assumed to include natural language pragmatics, but do they actually behave like pragmatic speakers? We attempt to answer this question using the Rational Speech Act (RSA) framework, which models pragmatic reasoning in human communication. We find that while scores from the LLM have some positive correlation with those from RSA, there isn't sufficient evidence to claim that it behaves like a pragmatic speaker.
  arXiv  Detail & Related papers  (2024-11-03T13:23:18Z)
• Large Language Models Assume People are More Rational than We Really are [10.857040292234984]
  In order for AI to communicate effectively with people, they must understand how we make decisions.<n>Previous empirical evidence seems to suggest that these implicit models are accurate.<n>We find that this is actually not the case when both simulating and predicting people's choices.
  arXiv  Detail & Related papers  (2024-06-24T18:15:27Z)
• Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting [87.30837365008931]
  Large language models (LLMs) equipped with Chain-of-Thought (CoT) prompting are able to make accurate incremental predictions even on unscalable tasks. This study examines the impact of LLMs' step-by-step predictions on gender bias in unscalable tasks.
  arXiv  Detail & Related papers  (2024-01-28T06:50:10Z)
• Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations [62.61495090463084]
  Large language models (LLMs) are trained to imitate humans to explain human decisions. We evaluate whether an explanation can enable humans to precisely infer the model's outputs on diverse counterfactuals. We found that LLM's explanations have low precision and that precision does not correlate with plausibility.
  arXiv  Detail & Related papers  (2023-07-17T17:41:47Z)
• The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputs [50.32802502923367]
  We study the process of language driving and influencing social reasoning in a probabilistic goal inference domain. We propose a neuro-symbolic model that carries out goal inference from linguistic inputs of agent scenarios. Our model closely matches human response patterns and better predicts human judgements than using an LLM alone.
  arXiv  Detail & Related papers  (2023-06-25T19:38:01Z)
• Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners [75.85554779782048]
  Large Language Models (LLMs) have excited the natural language and machine learning community over recent years. Despite of numerous successful applications, the underlying mechanism of such in-context capabilities still remains unclear. In this work, we hypothesize that the learned textitsemantics of language tokens do the most heavy lifting during the reasoning process.
  arXiv  Detail & Related papers  (2023-05-24T07:33:34Z)
• The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs [26.118193748582197]
  We evaluate four categories of widely used state-of-the-art models. We find that, despite only evaluating on utterances that require a binary inference, models in three of these categories perform close to random. These results suggest that certain fine-tuning strategies are far better at inducing pragmatic understanding in models.
  arXiv  Detail & Related papers  (2022-10-26T19:04:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.