Directional Attractors in LLM Reasoning: How Similarity Retrieval Steers Iterative Summarization Based Reasoning

• URL: http://arxiv.org/abs/2601.08846v1
• Date: Mon, 22 Dec 2025 00:26:54 GMT
• Title: Directional Attractors in LLM Reasoning: How Similarity Retrieval Steers Iterative Summarization Based Reasoning
• Authors: Cagatay Tekin, Charbel Barakat, Luis Joseph Luna Limgenco,
• Abstract summary: We introduce InftyThink with Cross-Chain Memory, an extension that augments iterative reasoning with an embedding-based semantic cache of previously successful reasoning patterns.<n> Experiments show that semantic lemma retrieval improves accuracy in structured domains while exposing failure modes in tests that include heterogeneous domains.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Iterative summarization based reasoning frameworks such as InftyThink enable long-horizon reasoning in large language models (LLMs) by controlling context growth, but they repeatedly regenerate similar reasoning strategies across tasks. We introduce InftyThink with Cross-Chain Memory, an extension that augments iterative reasoning with an embedding-based semantic cache of previously successful reasoning patterns. At each reasoning step, the model retrieves and conditions on the most semantically similar stored lemmas, guiding inference without expanding the context window indiscriminately. Experiments on MATH500, AIME2024, and GPQA-Diamond demonstrate that semantic lemma retrieval improves accuracy in structured domains while exposing failure modes in tests that include heterogeneous domains. Geometric analyses of reasoning trajectories reveal that cache retrieval induces directional biases in embedding space, leading to consistent fix (improve baseline accuracy) and break (degradation in baseline accuracy) attractors. Our results highlight both the benefits and limits of similarity-based memory for self-improving LLM reasoning.

Related papers

• Learning Structured Reasoning via Tractable Trajectory Control [99.75278337895024]
  Ctrl-R is a framework for learning structured reasoning via tractable trajectory control.<n>We show that Ctrl-R enables effective exploration and internalization of previously unattainable reasoning patterns.
  arXiv  Detail & Related papers  (2026-03-02T09:18:19Z)
• LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval [74.72139580745511]
  LaSER is a novel self-distillation framework that internalizes explicit reasoning into the latent space of retrievers.<n>Our method successfully combines the reasoning depth of explicit CoT pipelines with the inference efficiency of standard dense retrievers.
  arXiv  Detail & Related papers  (2026-03-02T04:11:18Z)
• CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching [50.65932158912512]
  We propose a new causal reasoning benchmark, CausalFlip, to encourage the development of new large language models.<n>CaulFlip consists of causal judgment questions built over event triples that could form different confounder, chain, and collider relations.<n>We evaluate LLMs under multiple training paradigms, including answer-only training, explicit Chain-of-Thought supervision, and a proposed internalized causal reasoning approach.
  arXiv  Detail & Related papers  (2026-02-23T18:06:15Z)
• Reinforced Efficient Reasoning via Semantically Diverse Exploration [73.41112984160992]
  Reinforcement learning with verifiable rewards (RLVR) has proven effective in enhancing the reasoning of large language models (LLMs)<n>We propose reinforced efficient reasoning via semantically diverse explorations, i.e., ROSE, for LLMs.<n>Our method incorporates a semantic-entropy-based branching strategy and an $varepsilon$-exploration mechanism.
  arXiv  Detail & Related papers  (2026-01-08T15:56:44Z)
• Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought [33.267497114389734]
  Large Language Models (LLMs) have demonstrated strong reasoning capabilities through emphChain-of-Thought (CoT) prompting.<n>CoT methods rely on discrete token-level reasoning processes prone to error propagation and limited by vocabulary.<n>We propose EBM-CoT, an Energy-Based Chain-of-Thought framework that refines latent thought representations through an energy-based model.
  arXiv  Detail & Related papers  (2025-11-10T14:10:58Z)
• From Implicit Exploration to Structured Reasoning: Leveraging Guideline and Refinement for LLMs [33.17712742134723]
  We propose a framework that shifts from implicit exploration to structured reasoning through guideline and refinement.<n>First, we extract structured reasoning patterns from successful trajectories and reflective signals from failures.<n>During inference, the model follows these guidelines step-by-step, with refinement applied after each step to correct errors and stabilize the reasoning process.
  arXiv  Detail & Related papers  (2025-09-08T02:11:49Z)
• Implicit Reasoning in Large Language Models: A Comprehensive Survey [67.53966514728383]
  Large Language Models (LLMs) have demonstrated strong generalization across a wide range of tasks.<n>Recent studies have shifted attention from explicit chain-of-thought prompting toward implicit reasoning.<n>This survey introduces a taxonomy centered on execution paradigms, shifting the focus from representational forms to computational strategies.
  arXiv  Detail & Related papers  (2025-09-02T14:16:02Z)
• A Survey on Latent Reasoning [100.54120559169735]
  Large Language Models (LLMs) have demonstrated impressive reasoning capabilities.<n>CoT reasoning that verbalizes intermediate steps limits the model's expressive bandwidth.<n>Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state.
  arXiv  Detail & Related papers  (2025-07-08T17:29:07Z)
• Reason from Future: Reverse Thought Chain Enhances LLM Reasoning [18.637191592875155]
  We propose a novel reasoning paradigm called Reason from Future (RFF)<n>RFF generates reasoning paths by bidirectional reasoning that combines top-down planning with bottom-up reasoning accumulation.<n>RFF outperforms conventional paradigms with higher accuracy and less searching space to solve complex tasks.
  arXiv  Detail & Related papers  (2025-06-04T08:03:17Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.