Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties

• URL: http://arxiv.org/abs/2506.05744v3
• Date: Wed, 01 Oct 2025 16:04:11 GMT
• Title: Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties
• Abstract summary: We introduce the notion of a reasoning graph, extracted by clustering hidden-state representations at each reasoning step.<n>We analyze three key graph-theoretic properties: cyclicity, diameter, and small-world index, across multiple tasks.
• Score: 48.56470855119906
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent large-scale reasoning models have achieved state-of-the-art performance on challenging mathematical benchmarks, yet the internal mechanisms underlying their success remain poorly understood. In this work, we introduce the notion of a reasoning graph, extracted by clustering hidden-state representations at each reasoning step, and systematically analyze three key graph-theoretic properties: cyclicity, diameter, and small-world index, across multiple tasks (GSM8K, MATH500, AIME 2024). Our findings reveal that distilled reasoning models (e.g., DeepSeek-R1-Distill-Qwen-32B) exhibit significantly more recurrent cycles (about 5 per sample), substantially larger graph diameters, and pronounced small-world characteristics (about 6x) compared to their base counterparts. Notably, these structural advantages grow with task difficulty and model capacity, with cycle detection peaking at the 14B scale and exploration diameter maximized in the 32B variant, correlating positively with accuracy. Furthermore, we show that supervised fine-tuning on an improved dataset systematically expands reasoning graph diameters in tandem with performance gains, offering concrete guidelines for dataset design aimed at boosting reasoning capabilities. By bridging theoretical insights into reasoning graph structures with practical recommendations for data construction, our work advances both the interpretability and the efficacy of large reasoning models.

Related papers

• Graph Hierarchical Recurrence for Long-Range Generalization [25.351165089986868]
  Graph Neural Networks (GNNs) and Graph Transformers (GTs) are now a fundamental paradigm for graph learning.<n>We introduce Graph Hierarchical Recurrence (GHR), a novel framework that operates jointly on the input graph and on a hierarchical abstraction obtained through pooling.<n>We show that GHR consistently outperforms existing graph models while using as little as 1% of the parameters of current state-of-the-art models.
  arXiv  Detail & Related papers  (2026-05-18T13:31:21Z)
• Where Larger Models Excel: The Primacy of Constraint-Guided Reasoning [15.328262855516689]
  We develop an automated framework that identifies questions where the larger model shows a stable advantage.<n>Our analysis yields a systematic taxonomy of larger model reasoning advantages, spanning both common advantages that recur across domains and specialized advantages associated with particular domains.<n>Across these patterns, a recurring theme is Constraint-Guided Reasoning: larger models are better at identifying explicit and implicit constraints, organizing them into structured reasoning, and using them to rule out infeasible paths and verify intermediate steps.
  arXiv  Detail & Related papers  (2026-05-09T09:54:59Z)
• Generalization Boundaries of Fine-Tuned Small Language Models for Graph Structural Inference [0.48302896549293584]
  We investigate the boundaries of structural inference in fine-tuned small language models.<n>We assess domain-learning capability on real-world graph benchmarks.
  arXiv  Detail & Related papers  (2026-04-20T11:09:12Z)
• Evaluating and Enhancing the Vulnerability Reasoning Capabilities of Large Language Models [15.849480549367684]
  We propose DAGVul, a novel framework that models vulnerability reasoning as a Directed Acyclic Graph (DAG) generation task.<n>By further introducing Reinforcement Learning with Verifiable Rewards (RLVR), we align model reasoning trace with program-intrinsic logic.<n>Our framework improves the reasoning F1-score by an average of 18.9% over all the baselines.
  arXiv  Detail & Related papers  (2026-02-06T13:19:45Z)
• The Underappreciated Power of Vision Models for Graph Structural Understanding [28.781392309798672]
  Graph Neural Networks operate through bottom-up message-passing, fundamentally differing from human visual perception.<n>We investigate the underappreciated potential of vision models for graph understanding, finding they achieve performance comparable to GNNs on established benchmarks.
  arXiv  Detail & Related papers  (2025-10-27T05:11:44Z)
• Causal Reasoning in Pieces: Modular In-Context Learning for Causal Discovery [6.72184534513047]
  Causal inference remains a fundamental challenge for large language models.<n>Recent advances in internal reasoning with large language models have sparked interest.<n>We study causal discovery on the Corr2Cause benchmark using the OpenAI's o-series and DeepSeek-R model families.
  arXiv  Detail & Related papers  (2025-07-31T12:10:27Z)
• GraphOracle: A Foundation Model for Knowledge Graph Reasoning [9.894106590443714]
  We introduce textbftextscGraphOracle, a relation-centric foundation model that unifies reasoning across knowledge graphs.<n>A query-dependent attention mechanism is developed to learn inductive representations for both relations and entities.<n>Pre-training on diverse knowledge graphs, followed by minutes-level fine-tuning, enables effective generalization to unseen entities, relations, and entire graphs.
  arXiv  Detail & Related papers  (2025-05-16T11:14:57Z)
• Phi-4-reasoning Technical Report [42.508165017775]
  We introduce Phi-4-reasoning, a 14-billion parameter reasoning model that achieves strong performance on complex reasoning tasks.<n>We develop Phi-4-reasoning-plus, a variant enhanced through a short phase of outcome-based reinforcement learning.<n>Both models outperform significantly larger open-weight models such as DeepSeek-R1-Distill-Llama-70B model and approach the performance levels of full DeepSeek-R1 model.
  arXiv  Detail & Related papers  (2025-04-30T05:05:09Z)
• Do Larger Language Models Imply Better Reasoning? A Pretraining Scaling Law for Reasoning [89.17086632436363]
  We introduce a synthetic multihop reasoning environment designed to replicate the structure and distribution of real-world large-scale knowledge graphs.<n>Our reasoning task involves completing missing edges in the graph, which requires advanced multi-hop reasoning and mimics real-world reasoning scenarios.<n>To predict the optimal model size for a specific knowledge graph, we find an empirical scaling that linearly maps the knowledge graph search entropy to the optimal model size.
  arXiv  Detail & Related papers  (2025-04-04T17:57:22Z)
• Revisiting Graph Neural Networks on Graph-level Tasks: Comprehensive Experiments, Analysis, and Improvements [54.006506479865344]
  We propose a unified evaluation framework for graph-level Graph Neural Networks (GNNs)<n>This framework provides a standardized setting to evaluate GNNs across diverse datasets.<n>We also propose a novel GNN model with enhanced expressivity and generalization capabilities.
  arXiv  Detail & Related papers  (2025-01-01T08:48:53Z)
• Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy [104.48511402784763]
  Performance Law for SR models aims to theoretically investigate and model the relationship between model performance and data quality.<n>We propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics.
  arXiv  Detail & Related papers  (2024-11-30T10:56:30Z)
• Deep Sparse Latent Feature Models for Knowledge Graph Completion [24.342670268545085]
  In this paper, we introduce a novel framework of sparse latent feature models for knowledge graphs. Our approach not only effectively completes missing triples but also provides clear interpretability of the latent structures. Our method significantly improves performance by revealing latent communities and producing interpretable representations.
  arXiv  Detail & Related papers  (2024-11-24T03:17:37Z)
• Introducing Diminutive Causal Structure into Graph Representation Learning [19.132025125620274]
  We introduce a novel method that enables Graph Neural Networks (GNNs) to glean insights from specialized diminutive causal structures. Our method specifically extracts causal knowledge from the model representation of these diminutive causal structures.
  arXiv  Detail & Related papers  (2024-06-13T00:18:20Z)
• Robust Causal Graph Representation Learning against Confounding Effects [21.380907101361643]
  We propose Robust Causal Graph Representation Learning (RCGRL) to learn robust graph representations against confounding effects. RCGRL introduces an active approach to generate instrumental variables under unconditional moment restrictions, which empowers the graph representation learning model to eliminate confounders.
  arXiv  Detail & Related papers  (2022-08-18T01:31:25Z)
• BCDAG: An R package for Bayesian structure and Causal learning of Gaussian DAGs [77.34726150561087]
  We introduce the R package for causal discovery and causal effect estimation from observational data. Our implementation scales efficiently with the number of observations and, whenever the DAGs are sufficiently sparse, the number of variables in the dataset. We then illustrate the main functions and algorithms on both real and simulated datasets.
  arXiv  Detail & Related papers  (2022-01-28T09:30:32Z)
• RelWalk A Latent Variable Model Approach to Knowledge Graph Embedding [50.010601631982425]
  This paper extends the random walk model (Arora et al., 2016a) of word embeddings to Knowledge Graph Embeddings (KGEs) We derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail) We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph.
  arXiv  Detail & Related papers  (2021-01-25T13:31:29Z)
• Towards Deeper Graph Neural Networks [63.46470695525957]
  Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Several recent studies attribute this performance deterioration to the over-smoothing issue. We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
  arXiv  Detail & Related papers  (2020-07-18T01:11:14Z)
• Block-Approximated Exponential Random Graphs [77.4792558024487]
  An important challenge in the field of exponential random graphs (ERGs) is the fitting of non-trivial ERGs on large graphs. We propose an approximative framework to such non-trivial ERGs that result in dyadic independence (i.e., edge independent) distributions. Our methods are scalable to sparse graphs consisting of millions of nodes.
  arXiv  Detail & Related papers  (2020-02-14T11:42:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.