SPUS: A Lightweight and Parameter-Efficient Foundation Model for PDEs

• URL: http://arxiv.org/abs/2510.01370v1
• Date: Wed, 01 Oct 2025 18:54:59 GMT
• Title: SPUS: A Lightweight and Parameter-Efficient Foundation Model for PDEs
• Authors: Abu Bucker Siddik, Diane Oyen, Alexander Most, Michal Kucer, Ayan Biswas,
• Abstract summary: We introduce SPUS, a compact and efficient foundation model (FM) designed as a unified neural operator for solving a wide range of partial differential equations (PDEs)<n>SPUS is pretrained on a diverse set of fluid dynamics PDEs and evaluated across 6 challenging unseen downstream PDEs spanning various physical systems.<n> Experimental results demonstrate that SPUS using residual U-Net based architecture achieves state-of-the-art generalization on these downstream tasks.
• Score: 40.11476265839176
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Small PDE U-Net Solver (SPUS), a compact and efficient foundation model (FM) designed as a unified neural operator for solving a wide range of partial differential equations (PDEs). Unlike existing state-of-the-art PDE FMs-primarily based on large complex transformer architectures with high computational and parameter overhead-SPUS leverages a lightweight residual U-Net-based architecture that has been largely underexplored as a foundation model architecture in this domain. To enable effective learning in this minimalist framework, we utilize a simple yet powerful auto-regressive pretraining strategy which closely replicates the behavior of numerical solvers to learn the underlying physics. SPUS is pretrained on a diverse set of fluid dynamics PDEs and evaluated across 6 challenging unseen downstream PDEs spanning various physical systems. Experimental results demonstrate that SPUS using residual U-Net based architecture achieves state-of-the-art generalization on these downstream tasks while requiring significantly fewer parameters and minimal fine-tuning data, highlighting its potential as a highly parameter-efficient FM for solving diverse PDE systems.

Related papers

• PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations [6.876642270107136]
  We propose a physics-informed multimodal foundation model (PI-MFM) framework that directly enforces governing equations during pretraining and adaptation.<n>PI-MFM takes symbolic representations of PDEs as the input, and automatically assembles PDE residual losses from the input expression.<n>On a benchmark of 13 parametric one-dimensional time-dependent PDE families, PI-MFM consistently outperforms purely data-driven counterparts.
  arXiv  Detail & Related papers  (2025-12-28T19:43:57Z)
• Physics-informed Reduced Order Modeling of Time-dependent PDEs via Differentiable Solvers [1.224954637705144]
  We propose Physics-informed ROM ($Phi$-ROM) by incorporating differentiable PDE solvers into the training procedure.<n>Our model outperforms state-of-the-art data-driven ROMs and other physics-informed strategies.<n>$Phi$-ROM learns to recover and forecast the solution fields even when trained or evaluated with sparse and irregular observations of the fields.
  arXiv  Detail & Related papers  (2025-05-20T16:47:04Z)
• Mechanistic PDE Networks for Discovery of Governing Equations [52.492158106791365]
  We present Mechanistic PDE Networks, a model for discovery of partial differential equations from data.<n>The represented PDEs are then solved and decoded for specific tasks.<n>We develop a native, GPU-capable, parallel, sparse, and differentiable multigrid solver specialized for linear partial differential equations.
  arXiv  Detail & Related papers  (2025-02-25T17:21:44Z)
• Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
  Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations.<n>We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
  arXiv  Detail & Related papers  (2024-11-28T13:16:20Z)
• Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers [53.79279286773326]
  We present Unisolver, a novel Transformer model trained on diverse data and conditioned on diverse PDEs.<n>Unisolver achieves consistent state-of-the-art on three challenging large-scale benchmarks, showing impressive performance and generalizability.
  arXiv  Detail & Related papers  (2024-05-27T15:34:35Z)
• Physics-informed Discretization-independent Deep Compositional Operator Network [1.2430809884830318]
  We introduce a novel physics-informed model architecture which can generalize to various discrete representations of PDE parameters and irregular domain shapes. Inspired by deep operator neural networks, our model involves a discretization-independent learning of parameter embedding repeatedly. Numerical results demonstrate the accuracy and efficiency of the proposed method.
  arXiv  Detail & Related papers  (2024-04-21T12:41:30Z)
• DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training [87.90342423839876]
  We present a new auto-regressive denoising pre-training strategy, which allows for more stable and efficient pre-training on PDE data. We train our PDE foundation model with up to 0.5B parameters on 10+ PDE datasets with more than 100k trajectories.
  arXiv  Detail & Related papers  (2024-03-06T08:38:34Z)
• Physics-driven machine learning models coupling PyTorch and Firedrake [0.0]
  Partial differential equations (PDEs) are central to describing and modelling complex physical systems. PDE-based machine learning techniques are designed to address this limitation. We present a simple yet effective coupling between the machine learning framework PyTorch and the PDE system Firedrake.
  arXiv  Detail & Related papers  (2023-03-13T05:42:58Z)
• Solving Coupled Differential Equation Groups Using PINO-CDE [42.363646159367946]
  PINO-CDE is a deep learning framework for solving coupled differential equation groups (CDEs) Based on the theory of physics-informed neural operator (PINO), PINO-CDE uses a single network for all quantities in a CDEs. This framework integrates engineering dynamics and deep learning technologies and may reveal a new concept for CDEs solving and uncertainty propagation.
  arXiv  Detail & Related papers  (2022-10-01T08:39:24Z)
• LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data [47.49194807524502]
  We propose LordNet, a tunable and efficient neural network for modeling entanglements. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements can be well modeled by the LordNet.
  arXiv  Detail & Related papers  (2022-06-19T14:41:08Z)
• Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs [70.51212431290611]
  Partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. We propose the Neural Operator with Regularity Structure (NORS) which incorporates the feature vectors for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic Phi41 model and the 2d Navier-Stokes equation.
  arXiv  Detail & Related papers  (2022-04-13T08:53:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.