Reasoning over mathematical objects: on-policy reward modeling and test time aggregation

• URL: http://arxiv.org/abs/2603.18886v1
• Date: Thu, 19 Mar 2026 13:27:12 GMT
• Title: Reasoning over mathematical objects: on-policy reward modeling and test time aggregation
• Abstract summary: We build and release training data and benchmarks for deriving mathematical objects, the Principia suite.<n>We show how on-policy training can also be used to scale test-time compute via aggregation.<n>We find that strong LMs such as Qwen3-235B and o3 struggle on Principia, while our training recipes can bring significant improvements.
• Score: 120.92237765735813
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability to precisely derive mathematical objects is a core requirement for downstream STEM applications, including mathematics, physics, and chemistry, where reasoning must culminate in formally structured expressions. Yet, current LM evaluations of mathematical and scientific reasoning rely heavily on simplified answer formats such as numerical values or multiple choice options due to the convenience of automated assessment. In this paper we provide three contributions for improving reasoning over mathematical objects: (i) we build and release training data and benchmarks for deriving mathematical objects, the Principia suite; (ii) we provide training recipes with strong LLM-judges and verifiers, where we show that on-policy judge training boosts performance; (iii) we show how on-policy training can also be used to scale test-time compute via aggregation. We find that strong LMs such as Qwen3-235B and o3 struggle on Principia, while our training recipes can bring significant improvements over different LLM backbones, while simultaneously improving results on existing numerical and MCQA tasks, demonstrating cross-format generalization of reasoning abilities.

Related papers

• From Atomic to Agentic: Towards Interpretable Evaluation of LLMs' Agentic Mathematical Capabilities [103.60384544335072]
  Large Language Models (LLMs) are evolving from performing end-to-end mathematical reasoning to integrating agentic intelligence.<n>Our framework aligns problem-solving agentic behaviors with a structured taxonomy of reusable mathematical atomic capabilities.<n> Experiments reveal that models with similar end-to-end accuracy can exhibit markedly different agentic capability profiles.
  arXiv  Detail & Related papers  (2026-08-27T10:52:06Z)
• PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models [31.37252086950609]
  PyraMathBench is a comprehensive benchmark with 32,505 questions derived from 7,404 math word problems.<n>Experiments reveal that LLMs' performance is severely compromised by inadequate numerical computation and weak handling of abstract numerical questions.<n>We propose the Smart Optimization & Learning-based VErsatile module (SOLVE) and Interactive Relative Policy Optimization (IRPO) to enhance LLMs' numerical-mathematical synergy.
  arXiv  Detail & Related papers  (2026-06-02T16:32:53Z)
• AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent [80.83250816918861]
  Large Reasoning Models (LRMs) like o3 and DeepSeek-R1 have achieved remarkable progress in natural language reasoning with long chain-of-thought.<n>However, they remain computationally inefficient and struggle with accuracy when solving problems requiring complex mathematical operations.<n>We present AgentMath, an agent framework that seamlessly integrates language models' reasoning capabilities with code interpreters' computational precision.
  arXiv  Detail & Related papers  (2025-12-23T19:57:49Z)
• FEM-Bench: A Structured Scientific Reasoning Benchmark for Evaluating Code-Generating LLMs [2.3052479658146323]
  We introduce FEM-Bench, a benchmark to evaluate the ability of LLMs to generate correct finite element method (FEM) and related code.<n>These tasks capture essential numerical and physical modeling challenges while representing only a small fraction of the complexity present in the discipline.<n>The best performing model at function writing, Gemini 3 Pro, completed 30/33 tasks at least once and 26/33 tasks all five times.
  arXiv  Detail & Related papers  (2025-12-23T19:40:51Z)
• Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimization [1.246870021158888]
  This paper investigates the capabilities of large language models (LLMs) in formulating and solving decision-making problems using mathematical programming.<n>We first conduct a systematic review and meta-analysis of recent literature to assess how well LLMs understand, structure, and solve optimization problems across domains.<n>Our systematic evidence is complemented by targeted experiments designed to evaluate the performance of state-of-the-art LLMs in automatically generating optimization models for problems in computer networks.
  arXiv  Detail & Related papers  (2025-08-25T14:52:56Z)
• Computational Reasoning of Large Language Models [51.629694188014064]
  We introduce textbfTuring Machine Bench, a benchmark to assess the ability of Large Language Models (LLMs) to execute reasoning processes.<n> TMBench incorporates four key features: self-contained and knowledge-agnostic reasoning, a minimalistic multi-step structure, controllable difficulty, and a theoretical foundation based on Turing machine.
  arXiv  Detail & Related papers  (2025-04-29T13:52:47Z)
• ML4PhySim : Machine Learning for Physical Simulations Challenge (The airfoil design) [16.140736542578562]
  The aim of this competition is to encourage the development of new ML techniques to solve physical problems. We propose learning a task representing the airfoil design simulation, using a dataset called AirfRANS. To the best of our knowledge, this is the first competition addressing the use of ML-based surrogate approaches to improve the trade-off computational cost/accuracy of physical simulation.
  arXiv  Detail & Related papers  (2024-03-03T22:10:21Z)
• JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding [74.12405417718054]
  This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model(PLM) Unlike other standard NLP tasks, mathematical texts are difficult to understand, since they involve mathematical terminology, symbols and formulas in the problem statement. We design a novel curriculum pre-training approach for improving the learning of mathematical PLMs, consisting of both basic and advanced courses.
  arXiv  Detail & Related papers  (2022-06-13T17:03:52Z)
• BenchML: an extensible pipelining framework for benchmarking representations of materials and molecules at scale [0.0]
  We introduce a machine-learning framework for benchmarking representations of chemical systems against datasets of materials and molecules. The guiding principle is to evaluate raw descriptor performance by limiting model complexity to simple regression schemes. The resulting models are intended as baselines that can inform future method development.
  arXiv  Detail & Related papers  (2021-12-04T09:07:16Z)
• An Extensible Benchmark Suite for Learning to Simulate Physical Systems [60.249111272844374]
  We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
  arXiv  Detail & Related papers  (2021-08-09T17:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.