When Models Know More Than They Say: Probing Analogical Reasoning in LLMs

• URL: http://arxiv.org/abs/2604.03877v1
• Date: Sat, 04 Apr 2026 22:00:22 GMT
• Title: When Models Know More Than They Say: Probing Analogical Reasoning in LLMs
• Abstract summary: Analogical reasoning is a core cognitive faculty essential for narrative understanding.<n>We compare a model's probed representations with its prompted performance at detecting narrative analogies.
• Score: 5.76923079935053
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Analogical reasoning is a core cognitive faculty essential for narrative understanding. While LLMs perform well when surface and structural cues align, they struggle in cases where an analogy is not apparent on the surface but requires latent information, suggesting limitations in abstraction and generalisation. In this paper we compare a model's probed representations with its prompted performance at detecting narrative analogies, revealing an asymmetry: for rhetorical analogies, probing significantly outperforms prompting in open-source models, while for narrative analogies, they achieve a similar (low) performance. This suggests that the relationship between internal representations and prompted behavior is task-dependent and may reflect limitations in how prompting accesses available information.

Related papers

• Cognitive Profiling of LRMs' Reasoning Traces Using Bloom's Taxonomy [51.08110837378928]
  We introduce a framework for automatic annotation of reasoning steps through the lens of Bloom's taxonomy.<n>We perform a large-scale analysis across models and datasets, revealing both similarities and differences in thinking patterns across models and tasks.<n>Our findings establish a fine-grained framework for analyzing thinking patterns in LRMs and provide actionable insights for enhancing reasoning quality.
  arXiv  Detail & Related papers  (2026-08-24T12:55:51Z)
• Enhancing Structural Mapping with LLM-derived Abstractions for Analogical Reasoning in Narratives [13.352345916789337]
  Analogical reasoning is a key driver of human generalization in problem-solving and argumentation.<n>We propose a modular framework named YARN, which uses LLMs to decompose narratives into units, abstract these units, and then passes them to a mapping component that aligns elements across stories to perform analogical reasoning.<n>Our experiments reveal that abstractions consistently improve model performance, resulting in competitive or better performance than end-to-end LLM baselines.
  arXiv  Detail & Related papers  (2026-03-31T16:57:11Z)
• Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs [100.02824137397464]
  We investigate how Large Language Models adapt their internal representations when encountering inputs of increasing difficulty.<n>We reveal a consistent and quantifiable phenomenon: as task difficulty increases, the last hidden states of LLMs become substantially sparser.<n>This sparsity--difficulty relation is observable across diverse models and domains.
  arXiv  Detail & Related papers  (2026-03-03T18:48:15Z)
• The Curious Case of Analogies: Investigating Analogical Reasoning in Large Language Models [22.609819017261632]
  Analogical reasoning is at the core of human cognition, serving as an important foundation for a variety of intellectual activities.<n>While prior work has shown that LLMs can represent task patterns and surface-level concepts, it remains unclear whether these models can encode high-level relational concepts.
  arXiv  Detail & Related papers  (2025-11-25T14:23:58Z)
• Modeling Understanding of Story-Based Analogies Using Large Language Models [1.4999444543328293]
  Recent advancements in Large Language Models have brought them closer to matching human cognition across a variety of tasks.<n>How well do these models align with human performance in detecting and mapping analogies?
  arXiv  Detail & Related papers  (2025-07-15T03:40:21Z)
• LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning [74.0242521818214]
  This paper systematically investigates the comparative dynamics of inductive (System 1) versus abductive/deductive (System 2) inference in large language models (LLMs)<n>We utilize a controlled analogical reasoning environment, varying modality (textual, visual, symbolic), difficulty, and task format (MCQ / free-text)<n>Our analysis reveals System 2 pipelines generally excel, particularly in visual/symbolic modalities and harder tasks, while System 1 is competitive for textual and easier problems.
  arXiv  Detail & Related papers  (2025-02-16T15:54:53Z)
• Self-supervised Analogical Learning using Language Models [59.64260218737556]
  We propose SAL, a self-supervised analogical learning framework.<n> SAL mimics the human analogy process and trains models to explicitly transfer high-quality symbolic solutions.<n>We show that the resulting models outperform base language models on a wide range of reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-02-03T02:31:26Z)
• Failure Modes of LLMs for Causal Reasoning on Narratives [51.19592551510628]
  We investigate the interaction between world knowledge and logical reasoning.<n>We find that state-of-the-art large language models (LLMs) often rely on superficial generalizations.<n>We show that simple reformulations of the task can elicit more robust reasoning behavior.
  arXiv  Detail & Related papers  (2024-10-31T12:48:58Z)
• LLMs as Models for Analogical Reasoning [14.412456982731467]
  Analogical reasoning is fundamental to human cognition and learning.<n>Recent studies have shown that large language models can sometimes match humans in analogical reasoning tasks.
  arXiv  Detail & Related papers  (2024-06-19T20:07:37Z)
• Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers [49.80959223722325]
  We study the distinction between feed-forward and attention layers in large language models.<n>We find that feed-forward layers tend to learn simple distributional associations such as bigrams, while attention layers focus on in-context reasoning.
  arXiv  Detail & Related papers  (2024-06-05T08:51:08Z)
• ARN: Analogical Reasoning on Narratives [13.707344123755126]
  We develop a framework that operationalizes dominant theories of analogy, using narrative elements to create surface and system mappings. We show that while all LLMs can largely recognize near analogies, even the largest ones struggle with far analogies in a zero-shot setting.
  arXiv  Detail & Related papers  (2023-10-02T08:58:29Z)
• ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base [51.777618249271725]
  ANALOGYKB is a million-scale analogy knowledge base derived from existing knowledge graphs (KGs) It identifies two types of analogies from the KGs: 1) analogies of the same relations, which can be directly extracted from the KGs, and 2) analogies of analogous relations, which are identified with a selection and filtering pipeline enabled by large language models (LLMs)
  arXiv  Detail & Related papers  (2023-05-10T09:03:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.