PDEAgent-Bench: A Multi-Metric, Multi-Library Benchmark for PDE Solver Generation

• URL: http://arxiv.org/abs/2605.09636v1
• Date: Sun, 10 May 2026 16:25:43 GMT
• Title: PDEAgent-Bench: A Multi-Metric, Multi-Library Benchmark for PDE Solver Generation
• Abstract summary: We introduce PDEAgent-Bench, the first multi-metric, multi-library benchmark for PDE-to-solver code generation.<n>PDEAgent-Bench contains 645 instances across 6 mathematical categories and 11 PDE families, with common FEM libraries for DOLFINx, Firedrake, and deal.II.<n>Experiments show that models can often produce runnable code, but their pass rate drops substantially once accuracy and efficiency requirements are enforced.
• Score: 31.813357785544408
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: PDE-to-solver code generation aims to automatically synthesize executable numerical solvers from partial differential equation (PDE) specifications. This task requires not only understanding the mathematical structure of PDEs, but also selecting appropriate discretization schemes and solver configurations, and correctly implementing the resulting formulations in finite-element method (FEM) libraries. Existing code generation benchmarks mainly evaluate syntactic correctness, or success on predefined test cases. To our knowledge, there is currently no publicly available benchmark specifically for PDE-to-solver code generation, and general-purpose code benchmarks do not fully capture the unique challenges of numerical PDE solution, such as ensuring solver accuracy, efficiency, and compatibility with professional FEM libraries. We introduce PDEAgent-Bench, to the best of our knowledge, the first multi-metric, multi-library benchmark for PDE-to-solver code generation. PDEAgent-Bench contains 645 instances across 6 mathematical categories and 11 PDE families, with common FEM libraries for DOLFINx, Firedrake, and deal.II. Each instance provides an agent-facing problem specification, a reference solution on a prescribed evaluation grid, and case-specific accuracy and runtime targets. PDEAgent-Bench adopts a staged evaluation framework in which generated solvers must sequentially pass executability, numerical accuracy, and computational efficiency checks. Experiments with representative LLMs and code agents show that models can often produce runnable code, but their pass rate drops substantially once accuracy and efficiency requirements are enforced. These results indicate that current agents remain limited in producing numerically reliable and efficient PDE solvers, and that PDEAgent-Bench provides a reproducible testbed grounded in the practical requirements of numerical PDE solving.

Related papers

• DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows [69.12339622053588]
  We introduce DuMateBench, a real-session benchmark reconstructed from anonymized and privacy-screened user sessions.<n>The resulting benchmark comprises 200 tasks spanning 8 broad scenarios and 17 fine-grained capability categories.<n>We execute these tasks in isolated Docker containers injected with three forms of real-world environmental complexity: Insufficient, Unstable, and Noisy.
  arXiv  Detail & Related papers  (2026-08-27T02:36:23Z)
• Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards [3.593890190223936]
  We introduce RLVP: Reinforcement Learning with Verifiable Physics, an RL post-training framework for multi-PDE solver code generation.<n>A single policy is post-trained across diverse PDE families spanning hyperbolic, parabolic, elliptic, and incompressible-flow systems.<n> RLVP improves over both pre-trained and supervised-only baselines on PDE benchmarks, and shows zero-shot improvement transfer to held-out PDEs.
  arXiv  Detail & Related papers  (2026-07-11T20:53:04Z)
• AutoPDE: Reliable Agentic PDE Solving via Explicitly Represented Solver Strategies [13.63605095777436]
  We propose AutoPDE, a code agent that maintains the solver strategy as an explicitly represented object throughout the solving process.<n>AutoPDE builds and maintains this object in three stages, all drawing from a library of reusable PDE-solving skills.<n>Experiments show that AutoPDE achieves a pass rate of $54.5%$, improving over the strongest baseline by $14.2$ percentage points.
  arXiv  Detail & Related papers  (2026-06-09T12:02:58Z)
• OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inference [8.112335572297928]
  Large language models (LLMs) have demonstrated strong capabilities in code generation, symbolic reasoning, and tool use.<n>We propose OpInf-LLM, an LLM PDE solving framework based on operator inference.
  arXiv  Detail & Related papers  (2026-02-02T00:04:50Z)
• PDE-Agent: A toolchain-augmented multi-agent framework for PDE solving [50.54386756067269]
  Solving Partial Differential Equations (PDEs) is a cornerstone of engineering and scientific research.<n>Traditional methods for PDE solving are cumbersome, relying on manual setup and domain expertise.<n>In this work, we frame PDE solving as tool invocation via LLM-driven agents.<n>We introduce PDE-Agent, the first toolchain-augmented multi-agent collaboration framework.
  arXiv  Detail & Related papers  (2025-12-18T06:02:50Z)
• Generalizing PDE Emulation with Equation-Aware Neural Operators [1.9410267062889204]
  Solving partial differential equations (PDEs) can be prohibitively expensive using traditional numerical methods.<n>Deep learning-based surrogate models typically specialize in a single PDE with fixed parameters.<n>We present a framework for equation-aware emulation that generalizes to unseen PDEs.
  arXiv  Detail & Related papers  (2025-11-12T20:40:05Z)
• PDE-SHARP: PDE Solver Hybrids through Analysis and Refinement Passes [12.096472648029204]
  We introduce PDE-SHARP, a framework to reduce computational costs by replacing expensive scientific computation by cheaper LLM inference.<n>PDE-SHARP achieves superior solver accuracy with 60-75% fewer computational evaluations.<n>To generate high-quality solvers, PDE-SHARP requires fewer than 13 solver evaluations on average compared to 30+ for baseline methods.
  arXiv  Detail & Related papers  (2025-10-31T18:38:05Z)
• Lang-PINN: From Language to Physics-Informed Neural Networks via a Multi-Agent Framework [54.447408954009454]
  Physics-informed neural networks (PINNs) provide a powerful approach for solving partial differential equations (PDEs)<n>We present Lang-PINN, an LLM-driven multi-agent system that builds trainable PINNs directly from natural language task descriptions.<n>Experiments show that Lang-PINN achieves substantially lower errors and greater robustness than competitive baselines.
  arXiv  Detail & Related papers  (2025-10-03T08:20:02Z)
• CodePDE: An Inference Framework for LLM-driven PDE Solver Generation [57.15474515982337]
  Partial differential equations (PDEs) are fundamental to modeling physical systems.<n>Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive.<n>We introduce CodePDE, the first inference framework for generating PDE solvers using large language models.
  arXiv  Detail & Related papers  (2025-05-13T17:58:08Z)
• Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers [53.79279286773326]
  We present Unisolver, a novel Transformer model trained on diverse data and conditioned on diverse PDEs.<n>Unisolver achieves consistent state-of-the-art on three challenging large-scale benchmarks, showing impressive performance and generalizability.
  arXiv  Detail & Related papers  (2024-05-27T15:34:35Z)
• Lie Point Symmetry Data Augmentation for Neural PDE Solvers [69.72427135610106]
  We present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity. In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations. We show how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
  arXiv  Detail & Related papers  (2022-02-15T18:43:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.