Beyond Memorization: Distinguishing between Reductive and Epistemic Reasoning in LLMs using Classic Logic Puzzles

• URL: http://arxiv.org/abs/2603.21350v1
• Date: Sun, 22 Mar 2026 18:18:17 GMT
• Title: Beyond Memorization: Distinguishing between Reductive and Epistemic Reasoning in LLMs using Classic Logic Puzzles
• Abstract summary: We argue that memorization is better understood as a special case of reduction, where a new instance is mapped onto a known problem.<n>We find that while some large models succeed via reduction, other models fail early, and all models struggle once epistemic reasoning is required.
• Score: 14.690035415630433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Epistemic reasoning requires agents to infer the state of the world from partial observations and information about other agents' knowledge. Prior work evaluating LLMs on canonical epistemic puzzles interpreted their behavior through a dichotomy between epistemic reasoning and brittle memorization. We argue that this framing is incomplete: in recent models, memorization is better understood as a special case of reduction, where a new instance is mapped onto a known problem. Instead, we introduce a reduction ladder, a sequence of modifications that progressively move instances away from a canonical epistemic puzzle, making reduction increasingly difficult while preserving the underlying logic. We find that while some large models succeed via reduction, other models fail early, and all models struggle once epistemic reasoning is required.

Related papers

• Consistency Is Not Always Correct: Towards Understanding the Role of Exploration in Post-Training Reasoning [75.79451512757844]
  Foundation models exhibit broad knowledge but limited task-specific reasoning.<n> RLVR and inference scaling motivate post-training strategies such as RLVR and inference scaling.<n>We show that RLVR induces a squeezing effect, reducing reasoning entropy and forgetting some correct paths.
  arXiv  Detail & Related papers  (2025-11-10T18:25:26Z)
• PHANTOM RECALL: When Familiar Puzzles Fool Smart Models [29.172155264798466]
  Large language models (LLMs) such as GPT, Gemini, and Claude often appear adept at solving classic logic puzzles.<n>Recent evidence suggests that these models frequently rely on memorized templates rather than reasoning from first principles.<n>Despite near-perfect accuracy on puzzles, models significantly underperform humans on unmodified ones.
  arXiv  Detail & Related papers  (2025-10-13T18:09:50Z)
• LAG: Logic-Augmented Generation from a Cartesian Perspective [7.2022636966543745]
  This paper introduces Logic-Augmented Generation (LAG), a novel paradigm that reframes knowledge augmentation through systematic question decomposition and dependency-aware reasoning.<n>Experiments on four benchmark datasets demonstrate that LAG significantly enhances reasoning robustness, reduces hallucination, and aligns LLM problem-solving with human cognition.
  arXiv  Detail & Related papers  (2025-08-07T15:42:00Z)
• Frontier LLMs Still Struggle with Simple Reasoning Tasks [53.497499123166804]
  This work studies the performance of frontier language models on a broad set of "easy" reasoning problems.<n>We create a suite of procedurally generated simple reasoning tasks, including counting, first-order logic, proof trees, and travel planning.<n>We show that even state-of-the-art thinking models consistently fail on such problems and for similar reasons.
  arXiv  Detail & Related papers  (2025-07-09T22:22:49Z)
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM [58.2298313720146]
  Multimodal hallucinations are multi-sourced and arise from diverse causes.<n>Existing benchmarks fail to adequately distinguish between perception-induced hallucinations and reasoning-induced hallucinations.
  arXiv  Detail & Related papers  (2025-05-30T05:54:36Z)
• Reasoning Effort and Problem Complexity: A Scaling Analysis in LLMs [26.494798719138526]
  We investigate how the reasoning effort of large language models scales with problem complexity.<n>Our results show that reasoning effort scales with problem size, but only up to a critical problem complexity.
  arXiv  Detail & Related papers  (2025-03-19T11:13:51Z)
• On Memorization of Large Language Models in Logical Reasoning [70.94164038947078]
  Large language models (LLMs) achieve good performance on challenging reasoning benchmarks, yet could also make basic reasoning mistakes.<n>One hypothesis is that the increasingly high and nearly saturated performance could be due to the memorization of similar problems.<n>We show that fine-tuning leads to heavy memorization, but it also consistently improves generalization performance.
  arXiv  Detail & Related papers  (2024-10-30T15:31:54Z)
• Temporal Knowledge Question Answering via Abstract Reasoning Induction [32.08799860090592]
  This study addresses the challenge of enhancing temporal knowledge reasoning in Large Language Models (LLMs) We propose Abstract Reasoning Induction (ARI) framework, which divides temporal reasoning into two distinct phases: Knowledge-agnostic and Knowledge-based. Our approach achieves remarkable improvements, with relative gains of 29.7% and 9.27% on two temporal QA datasets.
  arXiv  Detail & Related papers  (2023-11-15T17:46:39Z)
• DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy [76.58614128865652]
  We propose DetermLR, a novel perspective that rethinks the reasoning process as an evolution from indeterminacy to determinacy. First, we categorize known conditions into two types: determinate and indeterminate premises This provides an oveall direction for the reasoning process and guides LLMs in converting indeterminate data into progressively determinate insights. We automate the storage and extraction of available premises and reasoning paths with reasoning memory, preserving historical reasoning details for subsequent reasoning steps.
  arXiv  Detail & Related papers  (2023-10-28T10:05:51Z)
• Exposing Attention Glitches with Flip-Flop Language Modeling [55.0688535574859]
  This work identifies and analyzes the phenomenon of attention glitches in large language models. We introduce flip-flop language modeling (FFLM), a family of synthetic benchmarks designed to probe the extrapolative behavior of neural language models. We find that Transformer FFLMs suffer from a long tail of sporadic reasoning errors, some of which we can eliminate using various regularization techniques.
  arXiv  Detail & Related papers  (2023-06-01T17:44:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.