SCPRM: A Schema-aware Cumulative Process Reward Model for Knowledge Graph Question Answering

• URL: http://arxiv.org/abs/2605.02819v1
• Date: Mon, 04 May 2026 16:56:01 GMT
• Title: SCPRM: A Schema-aware Cumulative Process Reward Model for Knowledge Graph Question Answering
• Abstract summary: Large language models excel at complex reasoning, yet evaluating their intermediate steps remains challenging.<n> SCPRM evaluates reasoning paths by conditioning on the reasoning prefix.<n> SCPRM-MCTS improves the performance of Hits@k by an average of 1.18% over strong baselines.
• Score: 22.83259436157509
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models excel at complex reasoning, yet evaluating their intermediate steps remains challenging. Although process reward models provide step-wise supervision, they often suffer from a risk compensation effect, where incorrect steps are offset by later correct ones, assigning high rewards to flawed reasoning paths. This issue is further exacerbated in knowledge graph (KG) reasoning, as there may exist multiple paths between the start and end entities in the KGs, and a risky step can make the reasoning path flawed. Those limitations are problematic in risk-sensitive tasks such as medical and legal KG reasoning. To address the issues, we propose a Schema-aware Cumulative Process Reward Model (SCPRM) that evaluates reasoning paths by conditioning on the reasoning prefix , and incorporating schema distance between current reasoning step and the implicit target parsed from the query, which provides cumulative and future rewards to guide the path explorations. We further integrate SCPRM into Monte Carlo Tree Search (MCTS) as SCPRM-MCTS to conduct multi-hop reasoning on KGs for question answering (QA) tasks. Across medical and legal KGQA and CWQ, SCPRM-MCTS improves the performance of Hits@k by an average of 1.18% over strong baselines, demonstrating more accurate and risk-sensitive reasoning evaluation.

Related papers

• SuCo: Sufficiency-guided Continuous Adaptive Reasoning [59.17222918775686]
  Large Reasoning Models (LRMs) often generate excessively long Chain-of-Thoughts (CoT)<n>Existing efforts to mitigate this inefficiency typically rely on discrete reasoning modes or fixed budget tiers.<n>We introduce Minimal Sufficient CoT (MSC), defined as the shortest prefix of a CoT trajectory which is adequate for producing the correct answer.
  arXiv  Detail & Related papers  (2026-06-16T08:52:15Z)
• DPRM: A Dual Implicit Process Reward Model in Multi-Hop Question Answering [62.11407895095892]
  In question answering tasks, Chain of Thought (CoT) guides large language models through multi-step reasoning, and Knowledge Graphs (KGs) reduce hallucinations via semantic matching.<n>Traditional Process Reward Models (PRMs) evaluate the reasoning process but require costly human annotations or rollout generation.<n>We propose the DPRM (Dual Implicit Process Reward Model), which trains two implicit PRMs for CoT and KG reasoning in MHQA tasks.
  arXiv  Detail & Related papers  (2025-11-11T15:41:22Z)
• What Defines Good Reasoning in LLMs? Dissecting Reasoning Steps with Multi-Aspect Evaluation [67.47463575774388]
  We decompose reasoning quality into two dimensions: relevance and coherence.<n>To measure these aspects reliably, we introduce causal stepwise evaluation (CaSE)<n>We show that curating training data with CaSE-evaluated relevance and coherence directly improves final task performance.
  arXiv  Detail & Related papers  (2025-10-23T14:30:37Z)
• Linking Process to Outcome: Conditional Reward Modeling for LLM Reasoning [30.302863491794543]
  Process Reward Models (PRMs) aim to guide their step-by-step reasoning toward a final answer.<n>Existing PRMs fail to capture inter-step dependencies, or struggle to align process rewards with the final outcome.<n>We propose Conditional Reward Modeling that frames reasoning as a temporal process leading to a correct answer.
  arXiv  Detail & Related papers  (2025-09-30T17:38:45Z)
• Stop Spinning Wheels: Mitigating LLM Overthinking via Mining Patterns for Early Reasoning Exit [114.83867400179354]
  Overthinking can degrade overall performance of large language models.<n>We categorize reasoning into three stages: insufficient exploration stage, compensatory reasoning stage, and reasoning convergence stage.<n>We develop a lightweight thresholding strategy based on rules to improve reasoning accuracy.
  arXiv  Detail & Related papers  (2025-08-25T03:17:17Z)
• Causal Reward Adjustment: Mitigating Reward Hacking in External Reasoning via Backdoor Correction [5.518813485456855]
  External reasoning systems combine language models with process reward models (PRMs) to select high-quality reasoning paths for complex tasks.<n>These systems are prone to reward hacking, where high-scoring but logically incorrect paths are assigned high scores by the PRMs, leading to incorrect answers.<n>We propose Causal Reward Adjustment (CRA), a method that mitigates reward hacking by estimating the true reward of a reasoning path.
  arXiv  Detail & Related papers  (2025-08-06T08:48:55Z)
• Socratic-PRMBench: Benchmarking Process Reward Models with Systematic Reasoning Patterns [79.42805969325036]
  Process Reward Models (PRMs) are crucial in complex reasoning and problem-solving tasks.<n>PRMs are required to identify errors under various reasoning patterns during the reasoning process.<n>Existing benchmarks mainly focus on evaluating PRMs with stepwise correctness.<n>We introduce Socratic-PRMBench, a new benchmark to evaluate PRMs systematically under six reasoning patterns.
  arXiv  Detail & Related papers  (2025-05-29T14:26:53Z)
• Reward Models Identify Consistency, Not Causality [54.987590763737145]
  State-of-the-art reward models prioritize structural consistency over causal correctness.<n>Removing the problem statement has minimal impact on reward scores.<n> altering numerical values or disrupting the reasoning flow significantly affects RM outputs.
  arXiv  Detail & Related papers  (2025-02-20T14:57:14Z)
• ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding [25.329712997545794]
  We propose Retrieval-Augmented Reasoning through Trustworthy Process Rewarding (ReARTeR)<n>ReARTeR enhances RAG systems' reasoning capabilities through post-training and test-time scaling.<n> Experimental results on multi-step reasoning benchmarks demonstrate significant improvements.
  arXiv  Detail & Related papers  (2025-01-14T05:56:26Z)
• Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Tasks [68.49251303172674]
  State-of-the-art large language models (LLMs) exhibit impressive problem-solving capabilities but may struggle with complex reasoning and factual correctness. Existing methods harness the strengths of chain-of-thought and retrieval-augmented generation (RAG) to decompose a complex problem into simpler steps and apply retrieval to improve factual correctness. We introduce Critic-guided planning with Retrieval-augmentation, CR-Planner, a novel framework that leverages fine-tuned critic models to guide both reasoning and retrieval processes through planning.
  arXiv  Detail & Related papers  (2024-10-02T11:26:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.