PDE-Controller: LLMs for Autoformalization and Reasoning of PDEs

• URL: http://arxiv.org/abs/2502.00963v1
• Date: Mon, 03 Feb 2025 00:03:41 GMT
• Title: PDE-Controller: LLMs for Autoformalization and Reasoning of PDEs
• Authors: Mauricio Soroco, Jialin Song, Mengzhou Xia, Kye Emond, Weiran Sun, Wuyang Chen,
• Abstract summary: We present PDE-Controller, a framework that enables large language models to control systems governed by partial differential equations (PDEs)<n>Our approach enables LLMs to transform informal natural language instructions into formal specifications, and then execute reasoning and planning steps to improve the utility of PDE control.<n>Our PDE-Controller significantly outperforms prompting the latest open-source and GPT models in reasoning, autoformalization, and program synthesis.
• Score: 16.01754287623487
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While recent AI-for-math has made strides in pure mathematics, areas of applied mathematics, particularly PDEs, remain underexplored despite their significant real-world applications. We present PDE-Controller, a framework that enables large language models (LLMs) to control systems governed by partial differential equations (PDEs). Our approach enables LLMs to transform informal natural language instructions into formal specifications, and then execute reasoning and planning steps to improve the utility of PDE control. We build a holistic solution comprising datasets (both human-written cases and 2 million synthetic samples), math-reasoning models, and novel evaluation metrics, all of which require significant effort. Our PDE-Controller significantly outperforms prompting the latest open-source and GPT models in reasoning, autoformalization, and program synthesis, achieving up to a 62% improvement in utility gain for PDE control. By bridging the gap between language generation and PDE systems, we demonstrate the potential of LLMs in addressing complex scientific and engineering challenges. We will release all data, model checkpoints, and code at https://pde-controller.github.io/.

Related papers

• CodePDE: An Inference Framework for LLM-driven PDE Solver Generation [57.15474515982337]
  Partial differential equations (PDEs) are fundamental to modeling physical systems.<n>Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive.<n>We introduce CodePDE, the first inference framework for generating PDE solvers using large language models.
  arXiv  Detail & Related papers  (2025-05-13T17:58:08Z)
• Generative Discovery of Partial Differential Equations by Learning from Math Handbooks [15.151135612091306]
  This study introduces a knowledge guided approach that incorporates existing PDEs documented in a mathematical handbook to facilitate the discovery process.<n>The framework can recover a variety of PDE forms with high accuracy and computational efficiency, particularly in cases involving complex temporal derivatives or intricate spatial terms.<n> Notably, it succeeds in discovering a previously unreported PDE governing strongly nonlinear surface gravity waves propagating toward breaking.
  arXiv  Detail & Related papers  (2025-05-09T08:09:21Z)
• Paving the way for scientific foundation models: enhancing generalization and robustness in PDEs with constraint-aware pre-training [49.8035317670223]
  A scientific foundation model (SciFM) is emerging as a promising tool for learning transferable representations across diverse domains. We propose incorporating PDE residuals into pre-training either as the sole learning signal or in combination with data loss to compensate for limited or infeasible training data. Our results show that pre-training with PDE constraints significantly enhances generalization, outperforming models trained solely on solution data.
  arXiv  Detail & Related papers  (2025-03-24T19:12:39Z)
• Mechanistic PDE Networks for Discovery of Governing Equations [52.492158106791365]
  We present Mechanistic PDE Networks, a model for discovery of partial differential equations from data. The represented PDEs are then solved and decoded for specific tasks. We develop a native, GPU-capable, parallel, sparse, and differentiable multigrid solver specialized for linear partial differential equations.
  arXiv  Detail & Related papers  (2025-02-25T17:21:44Z)
• Explain Like I'm Five: Using LLMs to Improve PDE Surrogate Models with Text [7.136205674624813]
  We use pretrained Large Language Models (LLMs) to integrate various amounts known system information into PDE learning. Our approach significantly outperforms our baseline model, FactFormer, in both next-step prediction and autore rollout performance. Further analysis shows that pretrained LLMs provide highly structured latent space that is consistent with the amount of system information provided through text.
  arXiv  Detail & Related papers  (2024-10-02T00:19:20Z)
• Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers [55.0876373185983]
  We present the Universal PDE solver (Unisolver) capable of solving a wide scope of PDEs. Our key finding is that a PDE solution is fundamentally under the control of a series of PDE components. Unisolver achieves consistent state-of-the-art results on three challenging large-scale benchmarks.
  arXiv  Detail & Related papers  (2024-05-27T15:34:35Z)
• PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equations [3.0248879829045388]
  We present the first learning-based environment for boundary control of PDEs. We present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems.
  arXiv  Detail & Related papers  (2024-05-18T22:01:55Z)
• Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies [0.5919433278490629]
  Optimal control of parametric partial differential equations (PDEs) is crucial in many applications in engineering and science. Deep reinforcement learning (DRL) has the potential to solve high-dimensional and complex control problems. In this work, we leverage dictionary learning and differentiable L$_0$ regularization to learn sparse, robust, and interpretable control policies for PDEs.
  arXiv  Detail & Related papers  (2024-03-22T15:06:31Z)
• Self-Supervised Learning with Lie Symmetries for Partial Differential Equations [25.584036829191902]
  We learn general-purpose representations of PDEs by implementing joint embedding methods for self-supervised learning (SSL) Our representation outperforms baseline approaches to invariant tasks, such as regressing the coefficients of a PDE, while also improving the time-stepping performance of neural solvers. We hope that our proposed methodology will prove useful in the eventual development of general-purpose foundation models for PDEs.
  arXiv  Detail & Related papers  (2023-07-11T16:52:22Z)
• Training Deep Surrogate Models with Large Scale Online Learning [48.7576911714538]
  Deep learning algorithms have emerged as a viable alternative for obtaining fast solutions for PDEs. Models are usually trained on synthetic data generated by solvers, stored on disk and read back for training. It proposes an open source online training framework for deep surrogate models.
  arXiv  Detail & Related papers  (2023-06-28T12:02:27Z)
• Learning Neural PDE Solvers with Parameter-Guided Channel Attention [17.004380150146268]
  In application domains such as weather forecasting, molecular dynamics, and inverse design, ML-based surrogate models are increasingly used. We propose a Channel Attention Embeddings (CAPE) component for neural surrogate models and a simple yet effective curriculum learning strategy. The CAPE module can be combined with neural PDE solvers allowing them to adapt to unseen PDE parameters.
  arXiv  Detail & Related papers  (2023-04-27T12:05:34Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• Lie Point Symmetry Data Augmentation for Neural PDE Solvers [69.72427135610106]
  We present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity. In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations. We show how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
  arXiv  Detail & Related papers  (2022-02-15T18:43:17Z)
• A composable autoencoder-based iterative algorithm for accelerating numerical simulations [0.0]
  CoAE-MLSim is an unsupervised, lower-dimensional, local method that is motivated from key ideas used in commercial PDE solvers. It is tested for a variety of complex engineering cases to demonstrate its computational speed, accuracy, scalability, and generalization across different PDE conditions.
  arXiv  Detail & Related papers  (2021-10-07T20:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.