Reasoning Topology Matters: Network-of-Thought for Complex Reasoning Tasks

• URL: http://arxiv.org/abs/2603.20730v1
• Date: Sat, 21 Mar 2026 09:32:28 GMT
• Title: Reasoning Topology Matters: Network-of-Thought for Complex Reasoning Tasks
• Abstract summary: Chain-of-Thought (CoT) produces linear traces, while Tree-of-Thought (ToT) performs branching search.<n>We propose Network-of-Thought (NoT), a framework that models reasoning as a directed graph with typed nodes and edges.
• Score: 5.523132953818281
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing prompting paradigms structure LLM reasoning in limited topologies: Chain-of-Thought (CoT) produces linear traces, while Tree-of-Thought (ToT) performs branching search. Yet complex reasoning often requires merging intermediate results, revisiting hypotheses, and integrating evidence from multiple sources. We propose Network-of-Thought (NoT), a framework that models reasoning as a directed graph with typed nodes and edges, guided by a heuristic-based controller policy. Across four benchmarks (GSM8K, Game of 24, HotpotQA, ProofWriter) and three models (GPT-4o-mini, Llama-3.3-70B-Instruct, Qwen2.5-72B-Instruct), we investigate when network topology outperforms chain or tree structures, whether LLM-generated heuristics can guide graph-based reasoning search, and the computation-accuracy tradeoff across topologies, evaluating each method on accuracy, topology simplicity, and token efficiency. Our results show that CoT remains effective for sequential tasks with GPT-4o-mini (89.5\% on GSM8K), while NoT surpasses ToT on multi-hop reasoning (91.0\% vs.\ 88.0\% on HotpotQA with LLM-as-Judge). With 72B open-source models, NoT achieves the highest accuracy on GSM8K (91.5\%), and Qwen2.5-72B achieves the best multi-hop QA result overall (91.7\% on HotpotQA). Self-generated controller heuristics outperform fixed and random strategies on logical reasoning, with uncertainty-only weighting achieving 57.0\% on ProofWriter. We also find that evaluation methodology significantly impacts method rankings: string-match underestimates all methods on open-ended QA, with the largest gap for NoT, a pattern consistent across all three models (14--18 percentage point gap on HotpotQA).

Related papers

• Generating Workflow DAGs from Natural Language with Non-Reasoning LLMs [0.08537603362277842]
  We show that neuro-symbolic decomposition enables lower-cost, non-reasoning large language models to generate complex workflow DAGs.<n>We show that the method improves judge validity by 24 percentage points and achieves statistical equivalence to a reasoning model's out-of-the-box quality.
  arXiv  Detail & Related papers  (2026-08-31T05:01:29Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Hierarchical Chain-of-Thought Prompting: Enhancing LLM Reasoning Performance and Efficiency [4.139013391483979]
  Chain-of-Thought (CoT) prompting has significantly improved the reasoning capabilities of large language models (LLMs)<n>In this work, we introduce Hierarchical Chain-of-Thought (Hi-CoT) prompting.<n>Hi-CoT decomposes the reasoning process into hierarchical substeps by alternating between instructional planning and step-by-step execution.
  arXiv  Detail & Related papers  (2026-03-31T18:35:22Z)
• CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning [17.364321308755667]
  CoT2-Meta is a training-free metacognitive reasoning framework.<n>It combines object-level chain-of-thought generation with meta-level control over partial reasoning trajectories.<n>It consistently outperforms strong single-path, sampling-based, and search-based baselines.
  arXiv  Detail & Related papers  (2026-03-30T07:59:47Z)
• TDA-RC: Task-Driven Alignment for Knowledge-Based Reasoning Chains in Large Language Models [54.20787536978013]
  Chain-of-Thought (CoT) paradigm dominates practical applications for its single-round efficiency, yet its reasoning chains often exhibit logical gaps.<n>This paper proposes a topology-based method for optimizing reasoning chains.<n>We show that our approach offers a superior balance between reasoning accuracy and efficiency, showcasing a practical solution to single-round generation with multi-round intelligence''
  arXiv  Detail & Related papers  (2026-03-13T13:01:01Z)
• Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning [11.179446105672461]
  We propose a multi-stage efficient reasoning method that combines supervised fine-tuning and reinforcement learning.<n>Our approach reduces response length by an average of 28% for 8B models and 40% for 32B models.<n>It achieves a superior trade-off compared to more complex state-of-the-art efficient reasoning methods.
  arXiv  Detail & Related papers  (2026-01-06T12:31:51Z)
• DeepPrune: Parallel Scaling without Inter-trace Redundancy [53.62015294143274]
  Over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation.<n>We propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning.<n>Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient.
  arXiv  Detail & Related papers  (2025-10-09T17:24:54Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• S-GRPO: Early Exit via Reinforcement Learning in Reasoning Models [2.9925837108958864]
  Test-Time Scaling emerges as an active research focus in the large language model community.<n>Recent studies reveal that reasoning models (even Qwen3) consistently exhibit excessive thought redundancy.<n>This paper introduces Serial-Group Decaying-Reward Policy Optimization (S-GRPO), a novel reinforcement learning paradigm.
  arXiv  Detail & Related papers  (2025-05-12T15:50:44Z)
• START: Self-taught Reasoner with Tools [51.38785489790888]
  We introduce START (Self-Taught Reasoner with Tools), a tool-integrated long Chain-of-thought (CoT) reasoning LLM.<n> START is capable of performing complex computations, self-checking, exploring diverse methods, and self-ging.<n>It significantly outperforms the base QwQ-32B and achieves performance comparable to the state-of-the-art open-weight model R1-Distill-Qwen-32B.
  arXiv  Detail & Related papers  (2025-03-06T17:11:51Z)
• Markovian Transformers for Informative Language Modeling [1.172865818448696]
  Chain-of-Thought (CoT) reasoning often fails to faithfully reflect a language model's underlying decision process.<n>We introduce a Markovian language model framework that can be understood as a reasoning autoencoder.
  arXiv  Detail & Related papers  (2024-04-29T17:36:58Z)
• Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models [74.40196814292426]
  We propose Graph-of-Thought (GoT) reasoning, which models human thought processes not only as a chain but also as a graph. GoT captures the non-sequential nature of human thinking and allows for a more realistic modeling of thought processes. We evaluate GoT's performance on a text-only reasoning task and a multimodal reasoning task.
  arXiv  Detail & Related papers  (2023-05-26T02:15:09Z)
• Faithful Chain-of-Thought Reasoning [51.21714389639417]
  Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of reasoning tasks. We propose Faithful CoT, a reasoning framework involving two stages: Translation and Problem Solving. This guarantees that the reasoning chain provides a faithful explanation of the final answer.
  arXiv  Detail & Related papers  (2023-01-31T03:04:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.