Exploring Generative Process Reward Modeling for Semi-Structured Data: A Case Study of Table Question Answering

• URL: http://arxiv.org/abs/2510.20304v1
• Date: Thu, 23 Oct 2025 07:49:39 GMT
• Title: Exploring Generative Process Reward Modeling for Semi-Structured Data: A Case Study of Table Question Answering
• Authors: Lei Tang, Wei Zhou, Mohsen Mesgar,
• Abstract summary: Process reward models (PRMs) improve complex reasoning in large language models (LLMs) by grading candidate solutions step-by-step and selecting answers via aggregated step scores.<n>This work presents the first systematic study of PRMs for table question answering (TQA)<n>We evaluate state-of-the-art generative PRMs on TQA from both answer and step perspectives.
• Score: 14.119525003137356
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Process reward models (PRMs) improve complex reasoning in large language models (LLMs) by grading candidate solutions step-by-step and selecting answers via aggregated step scores. While effective in domains such as mathematics, their applicability to tasks involving semi-structured data, like table question answering (TQA) remains unexplored. TQA poses unique challenges for PRMs, including abundant irrelevant information, loosely connected reasoning steps, and domain-specific reasoning. This work presents the first systematic study of PRMs for TQA. We evaluate state-of-the-art generative PRMs on TQA from both answer and step perspectives. Results show that PRMs that combine textual and code verification can aid solution selection but struggle to generalize to out-of-domain data. Analysis reveals a weak correlation between performance in step-level verification and answer accuracy, possibly stemming from weak step dependencies and loose causal links. Our findings highlight limitations of current PRMs on TQA and offer valuable insights for building more robust, process-aware verifiers.

Related papers

• KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering [64.62317305868264]
  We present textbfKBQA-R1, a framework that shifts the paradigm from text imitation to interaction optimization via Reinforcement Learning.<n>Treating KBQA as a multi-turn decision process, our model learns to navigate the knowledge base using a list of actions.<n>Experiments on WebQSP, GrailQA, and GraphQuestions demonstrate that KBQA-R1 achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-10T17:45:42Z)
• DPRM: A Dual Implicit Process Reward Model in Multi-Hop Question Answering [62.11407895095892]
  In question answering tasks, Chain of Thought (CoT) guides large language models through multi-step reasoning, and Knowledge Graphs (KGs) reduce hallucinations via semantic matching.<n>Traditional Process Reward Models (PRMs) evaluate the reasoning process but require costly human annotations or rollout generation.<n>We propose the DPRM (Dual Implicit Process Reward Model), which trains two implicit PRMs for CoT and KG reasoning in MHQA tasks.
  arXiv  Detail & Related papers  (2025-11-11T15:41:22Z)
• A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models [31.650962391182798]
  This survey provides a systematic overview of PRMs through the full loop.<n>We summarize applications across math, code, text, multimodal reasoning, robotics, and agents.<n>Our goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment.
  arXiv  Detail & Related papers  (2025-10-09T10:35:31Z)
• Rethinking Reward Models for Multi-Domain Test-Time Scaling [91.76069784586149]
  Prior work generally assumes that process reward models (PRMs) outperform outcome reward models (ORMs) that assess only the final answer.<n>We present the first unified evaluation of four reward model variants across 14 diverse domains.<n>We attribute this to PRM-style stepwise scoring, which inherits label noise from LLM auto-labeling and has difficulty evaluating long reasoning trajectories.
  arXiv  Detail & Related papers  (2025-10-01T04:21:14Z)
• Uncertainty-Based Methods for Automated Process Reward Data Construction and Output Aggregation in Mathematical Reasoning [10.227089771963943]
  We propose an uncertainty-driven framework for automated process reward data construction.<n>We introduce two generic uncertainty-aware output aggregation methods: Hybrid Majority Reward Vote and weighted Reward Frequency Vote.<n>Experiments on ProcessBench, MATH, and GSMPlus show the effectiveness and efficiency of the proposed PRM data construction framework.
  arXiv  Detail & Related papers  (2025-08-03T14:14:13Z)
• Socratic-PRMBench: Benchmarking Process Reward Models with Systematic Reasoning Patterns [79.42805969325036]
  Process Reward Models (PRMs) are crucial in complex reasoning and problem-solving tasks.<n>PRMs are required to identify errors under various reasoning patterns during the reasoning process.<n>Existing benchmarks mainly focus on evaluating PRMs with stepwise correctness.<n>We introduce Socratic-PRMBench, a new benchmark to evaluate PRMs systematically under six reasoning patterns.
  arXiv  Detail & Related papers  (2025-05-29T14:26:53Z)
• Discriminative Policy Optimization for Token-Level Reward Models [55.98642069903191]
  Process reward models (PRMs) provide more nuanced supervision compared to outcome reward models (ORMs)<n>Q-RM explicitly learns token-level Q-functions from preference data without relying on fine-grained annotations.<n>Reinforcement learning with Q-RM significantly enhances training efficiency, achieving convergence 12 times faster than ORM on GSM8K and 11 times faster than step-level PRM on MATH.
  arXiv  Detail & Related papers  (2025-05-29T11:40:34Z)
• Generalizable Process Reward Models via Formally Verified Training Data [13.781401358802462]
  FoVer is an approach to synthesize PRM training data with accurate step-level error labels automatically annotated by formal verification tools.<n>Experiments show that PRMs trained with FoVer exhibit cross-task generalization, enabling a single PRM to effectively perform verification across diverse reasoning tasks.
  arXiv  Detail & Related papers  (2025-05-21T19:23:45Z)
• Process Reward Model with Q-Value Rankings [18.907163177605607]
  Process Reward Modeling (PRM) is critical for complex reasoning and decision-making tasks.<n>We introduce the Process Q-value Model (PQM), a novel framework that redefines PRM in the context of a Markov Decision Process.<n>PQM optimize Q-value rankings based on a novel comparative loss function, enhancing the model's ability to capture the intricate dynamics among sequential decisions.
  arXiv  Detail & Related papers  (2024-10-15T05:10:34Z)
• An Empirical Comparison of LM-based Question and Answer Generation Methods [79.31199020420827]
  Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context. In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning. Experiments show that an end-to-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.
  arXiv  Detail & Related papers  (2023-05-26T14:59:53Z)
• RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering [87.18962441714976]
  We introduce RoMQA, the first benchmark for robust, multi-evidence, multi-answer question answering (QA) We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, and find that RoMQA is challenging. Our results show that RoMQA is a challenging benchmark for large language models, and provides a quantifiable test to build more robust QA methods.
  arXiv  Detail & Related papers  (2022-10-25T21:39:36Z)
• Automatically Generating Counterfactuals for Relation Exaction [18.740447044960796]
  relation extraction (RE) is a fundamental task in natural language processing. Current deep neural models have achieved high accuracy but are easily affected by spurious correlations. We develop a novel approach to derive contextual counterfactuals for entities.
  arXiv  Detail & Related papers  (2022-02-22T04:46:10Z)
• Abstractive Query Focused Summarization with Query-Free Resources [60.468323530248945]
  In this work, we consider the problem of leveraging only generic summarization resources to build an abstractive QFS system. We propose Marge, a Masked ROUGE Regression framework composed of a novel unified representation for summaries and queries. Despite learning from minimal supervision, our system achieves state-of-the-art results in the distantly supervised setting.
  arXiv  Detail & Related papers  (2020-12-29T14:39:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.