OptProver: Bridging Olympiad and Optimization through Continual Training in Formal Theorem Proving

• URL: http://arxiv.org/abs/2604.23712v2
• Date: Tue, 28 Apr 2026 04:48:08 GMT
• Title: OptProver: Bridging Olympiad and Optimization through Continual Training in Formal Theorem Proving
• Abstract summary: Recent advances in formal Olympiad-level mathematics have left largely naive domain domains.<n>We introduce specialized perity-weighted optimization with a mechanism to valid but non-progressed proof proof steps.<n>On this benchmark, demonstrating state-of-the-art domain knowledge achieves state-of-one and comparably sized domains.
• Score: 11.603331256500033
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in formal theorem proving have focused on Olympiad-level mathematics, leaving undergraduate domains largely unexplored. Optimization, fundamental to machine learning, operations research, and scientific computing, remains underserved by existing provers. Its reliance on domain-specific formalisms (convexity, optimality conditions, and algorithmic analysis) creates significant distribution shift, making naive domain transfer ineffective. We present OptProver, a trained model that achieves robust transfer from Olympiad to undergraduate optimization. Starting from a strong Olympiad-level prover, our pipeline mitigates distribution shift through two key innovations. First, we employ large-scale optimization-focused data curation via expert iteration. Second, we introduce a specialized preference learning objective that integrates perplexity-weighted optimization with a mechanism to penalize valid but non-progressing proof steps. This not only addresses distribution shifts but also guides the search toward efficient trajectories. To enable rigorous evaluation, we construct a novel benchmark in Lean 4 focused on optimization. On this benchmark, OptProver achieves state-of-the-art Pass@1 and Pass@32 among comparably sized models while maintaining competitive performance on general theorem-proving tasks, demonstrating effective domain transfer without catastrophic forgetting.

Related papers

• Rethinking Learnability in Offline Data-driven Optimization [20.146021332129703]
  Black-Box Optimization (BBO) faces efficiency challenges as real-world problems grow increasingly complex.<n>Data-driven optimization has been the most popular paradigm to improve the efficiency of BBO, by learning from data.<n>Many offline optimization methods have been proposed, but a fundamental question remains unanswered: what learnability is sufficient for offline optimization?
  arXiv  Detail & Related papers  (2026-09-01T16:22:28Z)
• Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization [9.054156002180028]
  This thesis addresses seven challenges at the intersection of theory and practice.<n>First, we introduce ProxSkip and prove that local gradient steps can accelerate communication.<n>Third, we develop Variance Reduced ProxSkip, which eliminates the neighborhood error of local updates.
  arXiv  Detail & Related papers  (2026-08-06T20:20:52Z)
• A Defense of the Quadratic Model [71.57026347416729]
  We stress test the simplest possible model of optimization -- the quadratic model.<n>We show that it can be predictive surprisingly in an LLM setting with 150M parameters and 3B training tokens.
  arXiv  Detail & Related papers  (2026-07-23T18:02:03Z)
• A Novel Unified Parametric Assumption for Nonconvex Optimization [53.943470475510196]
  Non optimization is central to machine learning, but the general framework non convexity enables weak convergence guarantees too pessimistic compared to the other hand.<n>We introduce a novel unified assumption in non convex algorithms.
  arXiv  Detail & Related papers  (2025-02-17T21:25:31Z)
• Alchemy: Amplifying Theorem-Proving Capability through Symbolic Mutation [71.32761934724867]
  This work proposes Alchemy, a framework for data synthesis that constructs formal theorems through symbolic mutation.<n>For each candidate theorem in Mathlib, we identify all invocable theorems that can be used to rewrite or apply to it.<n>As a result, our method increases the number of theorems in Mathlib by an order of magnitude, from 110k to 6M.
  arXiv  Detail & Related papers  (2024-10-21T08:04:21Z)
• Discovering Preference Optimization Algorithms with and for Large Language Models [50.843710797024805]
  offline preference optimization is a key method for enhancing and controlling the quality of Large Language Model (LLM) outputs. We perform objective discovery to automatically discover new state-of-the-art preference optimization algorithms without (expert) human intervention. Experiments demonstrate the state-of-the-art performance of DiscoPOP, a novel algorithm that adaptively blends logistic and exponential losses.
  arXiv  Detail & Related papers  (2024-06-12T16:58:41Z)
• ENOT: Expectile Regularization for Fast and Accurate Training of Neural Optimal Transport [3.0237149871998095]
  We present a new approach to accurately and efficiently estimating optimal transportation plan. It is called ExpectileRegularised Neural Transport Optimal (ENOT) ENOT enforces binding conditions on the learning process of dual potentials.
  arXiv  Detail & Related papers  (2024-03-06T15:15:42Z)
• Pseudo-Bayesian Optimization [7.556071491014536]
  We study an axiomatic framework that elicits the minimal requirements to guarantee black-box optimization convergence.<n>We show how using simple local regression, and a suitable "randomized prior" construction to quantify uncertainty, not only guarantees convergence but also consistently outperforms state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2023-10-15T07:55:28Z)
• Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers [109.52244418498974]
  We propose a novel textscAdmeta (textbfADouble exponential textbfMov averagtextbfE textbfAdaptive and non-adaptive momentum) framework. We provide two implementations, textscAdmetaR and textscAdmetaS, the former based on RAdam and the latter based on SGDM.
  arXiv  Detail & Related papers  (2023-07-02T18:16:06Z)
• Domain Adversarial Training: A Game Perspective [80.3821370633883]
  This paper defines optimal solutions in domain-adversarial training from a game theoretical perspective. We show that descent in domain-adversarial training can violate the convergence guarantees of the gradient, oftentimes hindering the transfer performance. Ours are easy to implement, free of additional parameters, and can be plugged into any domain-adversarial framework.
  arXiv  Detail & Related papers  (2022-02-10T22:17:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.