Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards

• URL: http://arxiv.org/abs/2607.10474v1
• Date: Sat, 11 Jul 2026 20:53:04 GMT
• Title: Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards
• Abstract summary: We introduce RLVP: Reinforcement Learning with Verifiable Physics, an RL post-training framework for multi-PDE solver code generation.<n>A single policy is post-trained across diverse PDE families spanning hyperbolic, parabolic, elliptic, and incompressible-flow systems.<n> RLVP improves over both pre-trained and supervised-only baselines on PDE benchmarks, and shows zero-shot improvement transfer to held-out PDEs.
• Score: 3.593890190223936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Partial differential equations (PDEs) are foundational to modeling in science and engineering, but constructing reliable numerical solvers remains labor-intensive, demanding expert knowledge of discretization schemes, stability conditions, and boundary treatments. Recent work has begun to frame PDE solving as a code-generation task for large language models (LLMs), yet existing approaches operate primarily at inference time: relying on prompting, debugging, self-refinement, and test-time scaling rather than adapting the model itself. In parallel, reinforcement learning with verifiable rewards has emerged as a post-training paradigm for code and math reasoning, but its verifiers are typically binary: a compiler runs, or a test passes. Such signals discard the graded structure of scientific correctness, where two solvers may both execute and yet differ in solution accuracy by orders of magnitude. In this work, we introduce RLVP: Reinforcement Learning with Verifiable Physics, an RL post-training framework for multi-PDE solver code generation. RLVP addresses this verifiability gap with a hybrid verifier: hard program-validity checks ensure executability, while continuous physics rewards score function-space accuracy and PDE-residual consistency. A single policy is post-trained across diverse PDE families spanning hyperbolic, parabolic, elliptic, and incompressible-flow systems. RLVP improves over both pre-trained and supervised-only baselines on PDE benchmarks, and shows zero-shot improvement transfer to held-out PDEs. We show that a smaller LLM post-trained with RLVP can outperform prompting a frontier model on in-distribution PDE solver generation. The trained policy shows evidence of compositionality in numerical motifs: it recombines stencils, time-stepping schemes, and boundary-handling primitives learned from the PDEs used in training into generated solvers for unseen PDE problems.

Related papers

• AutoPDE: Reliable Agentic PDE Solving via Explicitly Represented Solver Strategies [13.63605095777436]
  We propose AutoPDE, a code agent that maintains the solver strategy as an explicitly represented object throughout the solving process.<n>AutoPDE builds and maintains this object in three stages, all drawing from a library of reusable PDE-solving skills.<n>Experiments show that AutoPDE achieves a pass rate of $54.5%$, improving over the strongest baseline by $14.2$ percentage points.
  arXiv  Detail & Related papers  (2026-06-09T12:02:58Z)
• DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models [55.01951088768769]
  DiffusionOPD is a new multi-task training paradigm for diffusion models based on Online Policy Distillation (OPD)<n>We show that DiffusionOPD consistently surpasses both multi-reward RL and cascade RL baselines in training efficiency and final performance.
  arXiv  Detail & Related papers  (2026-05-14T16:49:09Z)
• PDEAgent-Bench: A Multi-Metric, Multi-Library Benchmark for PDE Solver Generation [31.813357785544408]
  We introduce PDEAgent-Bench, the first multi-metric, multi-library benchmark for PDE-to-solver code generation.<n>PDEAgent-Bench contains 645 instances across 6 mathematical categories and 11 PDE families, with common FEM libraries for DOLFINx, Firedrake, and deal.II.<n>Experiments show that models can often produce runnable code, but their pass rate drops substantially once accuracy and efficiency requirements are enforced.
  arXiv  Detail & Related papers  (2026-05-10T16:25:43Z)
• PDE-Agent: A toolchain-augmented multi-agent framework for PDE solving [50.54386756067269]
  Solving Partial Differential Equations (PDEs) is a cornerstone of engineering and scientific research.<n>Traditional methods for PDE solving are cumbersome, relying on manual setup and domain expertise.<n>In this work, we frame PDE solving as tool invocation via LLM-driven agents.<n>We introduce PDE-Agent, the first toolchain-augmented multi-agent collaboration framework.
  arXiv  Detail & Related papers  (2025-12-18T06:02:50Z)
• CodePDE: An Inference Framework for LLM-driven PDE Solver Generation [57.15474515982337]
  Partial differential equations (PDEs) are fundamental to modeling physical systems.<n>Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive.<n>We introduce CodePDE, the first inference framework for generating PDE solvers using large language models.
  arXiv  Detail & Related papers  (2025-05-13T17:58:08Z)
• Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers [53.79279286773326]
  We present Unisolver, a novel Transformer model trained on diverse data and conditioned on diverse PDEs.<n>Unisolver achieves consistent state-of-the-art on three challenging large-scale benchmarks, showing impressive performance and generalizability.
  arXiv  Detail & Related papers  (2024-05-27T15:34:35Z)
• Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
  We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
  arXiv  Detail & Related papers  (2023-11-30T22:34:57Z)
• Physics-constrained robust learning of open-form partial differential equations from limited and noisy data [1.50528618730365]
  This study proposes a framework to robustly uncover open-form partial differential equations (PDEs) from limited and noisy data. A neural network-based predictive model fits the system response and serves as the reward evaluator for the generated PDEs. Numerical experiments demonstrate our framework's capability to uncover governing equations from nonlinear dynamic systems with limited and highly noisy data.
  arXiv  Detail & Related papers  (2023-09-14T12:34:42Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers [26.444103444634994]
  We show that machine learning can improve the solution accuracy by correcting for effects not captured by the discretized PDE. We find that previously used learning approaches are significantly outperformed by methods that integrate the solver into the training loop. This provides the model with realistic input distributions that take previous corrections into account.
  arXiv  Detail & Related papers  (2020-06-30T18:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.