On Multi-Step Theorem Prediction via Non-Parametric Structural Priors

• URL: http://arxiv.org/abs/2603.04852v1
• Date: Thu, 05 Mar 2026 06:08:50 GMT
• Title: On Multi-Step Theorem Prediction via Non-Parametric Structural Priors
• Authors: Junbo Zhao, Ting Zhang, Can Li, Wei He, Jingdong Wang, Hua Huang,
• Abstract summary: In this work, we explore training-free theorem prediction through the lens of in-context learning (ICL)<n>We propose Theorem Precedence Graphs, which encode temporal dependencies from historical solution traces as directed graphs, and impose explicit topological constraints that effectively prune the search space during inference.<n>Experiments on the FormalGeo7k benchmark show that our method achieves 89.29% accuracy, substantially outperforming ICL baselines and matching state-of-the-art supervised models.
• Score: 50.16583672681106
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-step theorem prediction is a central challenge in automated reasoning. Existing neural-symbolic approaches rely heavily on supervised parametric models, which exhibit limited generalization to evolving theorem libraries. In this work, we explore training-free theorem prediction through the lens of in-context learning (ICL). We identify a critical scalability bottleneck, termed Structural Drift: as reasoning depth increases, the performance of vanilla ICL degrades sharply, often collapsing to near zero. We attribute this failure to the LLM's inability to recover latent topological dependencies, leading to unstructured exploration. To address this issue, we propose Theorem Precedence Graphs, which encode temporal dependencies from historical solution traces as directed graphs, and impose explicit topological constraints that effectively prune the search space during inference. Coupled with retrieval-augmented graph construction and a stepwise symbolic executor, our approach enables LLMs to act as structured planners without any gradient-based optimization. Experiments on the FormalGeo7k benchmark show that our method achieves 89.29% accuracy, substantially outperforming ICL baselines and matching state-of-the-art supervised models. These results indicate that explicit structural priors offer a promising direction for scaling LLM-based symbolic reasoning.

Related papers

• No Global Plan in Chain-of-Thought: Uncover the Latent Planning Horizon of LLMs [65.783709850324]
  This work stems from prior complementary observations on the dynamics of Chain-of-Thought (CoT): Large Language Models (LLMs)<n>LLMs are shown latent planning of subsequent reasoning prior to CoT emergence, thereby diminishing the significance of explicit CoT.<n>We investigate the latent planning strength of LLMs, through our probing method, Tele-Lens, applying to hidden states across diverse task domains.
  arXiv  Detail & Related papers  (2026-02-02T13:46:56Z)
• The Procrustean Bed of Time Series: The Optimization Bias of Point-wise Loss [53.542743390809356]
  This paper aims to provide a first-principles analysis of the Expectation of Optimization Bias (EOB)<n>Our analysis reveals a fundamental paradigm paradox: the more deterministic and structured the time series, the more severe the bias by point-wise loss function.<n>We present a concrete solution that simultaneously achieves both principles via DFT or DWT.
  arXiv  Detail & Related papers  (2025-12-21T06:08:22Z)
• GLOW: Graph-Language Co-Reasoning for Agentic Workflow Performance Prediction [51.83437071408662]
  We propose GLOW, a unified framework for AW performance prediction.<n>GLOW combines the graph-structure modeling capabilities of GNNs with the reasoning power of LLMs.<n>Experiments on FLORA-Bench show that GLOW outperforms state-of-the-art baselines in prediction accuracy and ranking utility.
  arXiv  Detail & Related papers  (2025-12-11T13:30:46Z)
• Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training [76.12556589212666]
  We show that curriculum post-training avoids the exponential complexity bottleneck.<n>Under outcome-only reward signals, reinforcement learning finetuning achieves high accuracy with sample complexity.<n>We establish guarantees for test-time scaling, where curriculum-aware querying reduces both reward oracle calls and sampling cost from exponential to order.
  arXiv  Detail & Related papers  (2025-11-10T18:29:54Z)
• Improving constraint-based discovery with robust propagation and reliable LLM priors [13.871152992680152]
  We propose MosaCD, a causal discovery method that propagates edges from a high-confidence set of seeds.<n>We then apply a novel confidence-down propagation strategy that orients the most reliable edges first, and can be integrated with any skeleton-based discovery method.
  arXiv  Detail & Related papers  (2025-09-28T02:00:20Z)
• Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning [8.248971739035746]
  Large Language Models (LLMs) display striking surface fluency yet systematically fail at tasks requiring symbolic reasoning, arithmetic accuracy, and logical consistency.<n>This paper offers a structural diagnosis of such failures, revealing a persistent gap between textitcomprehension and textitcompetence<n>We term this phenomenon the computational textitsplit-brain syndrome, where instruction and action pathways are geometrically and functionally dissociated.
  arXiv  Detail & Related papers  (2025-07-14T04:01:45Z)
• What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning [45.660562905010934]
  We present LCoT2Tree, an automated framework that converts sequential LCoTs into hierarchical tree structures.<n>Using graph neural networks (GNNs), we reveal that structural patterns extracted by LCoT2Tree serve as stronger predictors of final performance.<n>Our results underscore the critical role of internal structures of reasoning chains, positioning LCoT2Tree as a powerful tool for diagnosing, interpreting, and improving reasoning in LLMs.
  arXiv  Detail & Related papers  (2025-05-28T09:12:31Z)
• Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLM [11.181783720439563]
  Large Language Models (LLMs) display sophisticated reasoning abilities via extended Chain-of-Thought (CoT) generation.<n>RLMs often demonstrate counterintuitive and unstable behaviors, such as performance degradation under few-shot prompting.<n>We introduce a unified graph-based analytical framework for better modeling the reasoning processes of RLMs.
  arXiv  Detail & Related papers  (2025-05-20T03:54:57Z)
• The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning [56.574829311863446]
  Chain-of-Thought (CoT) prompting has been widely recognized for its ability to enhance reasoning capabilities in large language models (LLMs)<n>We demonstrate that CoT and its reasoning variants consistently underperform direct answering across varying model scales and benchmark complexities.<n>Our analysis uncovers a fundamental hybrid mechanism of explicit-implicit reasoning driving CoT's performance in pattern-based ICL.
  arXiv  Detail & Related papers  (2025-04-07T13:51:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.