Related papers: Unsupervised Adaptation of PDE Foundation Models

Unsupervised Adaptation of PDE Foundation Models

URL: http://arxiv.org/abs/2608.07053v1
Date: Fri, 07 Aug 2026 10:01:50 GMT
Title: Unsupervised Adaptation of PDE Foundation Models
Abstract summary: We propose an unsupervised PDE-based finetuning framework that eliminates the need for ground-truth solutions.<n>Our method achieves performance comparable to supervised LoRA finetuning without requiring any ground-truth solutions.
Score: 17.002625091063937
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Pretrained partial differential equation (PDE) foundation models can generalize across different equations, but adapting them to unseen PDE systems typically requires dense solution data, which is often expensive or unavailable. To address this limitation, we propose an unsupervised PDE-based finetuning framework that eliminates the need for ground-truth solutions. We first pretrain a neighborhood attention Transformer on diverse time-dependent PDEs spanning varying spatial scales, yielding transferable representations across heterogeneous equations. In the adaptation stage, we construct a physics-based objective using the PDE residual and boundary conditions, and finetune the model on unseen equations via low-rank adaptation (LoRA). To address the uneven learning across physical quantities in standard LoRA, we introduce NSLoRA, a Newton-Schulz orthogonalized variant that rebalances adaptation. Our method achieves performance comparable to supervised LoRA finetuning without requiring any ground-truth solutions, while consistently outperforming competitive neural operator baselines and recent PDE foundation models across heterogeneous PDE benchmarks spanning multiple spatial dimensions.

Related papers

AOT-POT: Adaptive Operator Transformation for Large-Scale PDE Pre-training [50.28798360093496]
We propose to transform complex and diverse solution operators into simpler, better-aligned forms that are easier to model jointly.<n>We instantiate this idea as AOT-POT (adaptive operator-transformation for pre-training operator transformer), which expands hidden representations into multiple parallel streams.<n>AOT-POT achieves state-of-the-art performance on 12 PDE benchmarks with only 3% additional parameters, reducing relative L2 error by up to 77.6% (40.9% on average)
arXiv Detail & Related papers (2026-05-15T09:50:34Z)
A Randomized PDE Energy driven Iterative Framework for Efficient and Stable PDE Solutions [3.005135324395161]
Partial differential equations (PDEs) are central to scientific and engineering applications.<n>Existing numerical solvers rely heavily on matrix based discretizations.<n>We propose a PDE energy driven framework that solves PDEs through physically constrained diffusion iterations.
arXiv Detail & Related papers (2026-04-17T06:53:03Z)
Scale-Consistent Learning for Partial Differential Equations [79.48661503591943]
We propose a data augmentation scheme based on scale-consistency properties of PDEs.<n>We then design a scale-informed neural operator that can model a wide range of scales.<n>With scale-consistency, the model trained on $Re$ of 1000 can generalize to $Re$ ranging from 250 to 10000.
arXiv Detail & Related papers (2025-07-24T21:29:52Z)
Mechanistic PDE Networks for Discovery of Governing Equations [52.492158106791365]
We present Mechanistic PDE Networks, a model for discovery of partial differential equations from data.<n>The represented PDEs are then solved and decoded for specific tasks.<n>We develop a native, GPU-capable, parallel, sparse, and differentiable multigrid solver specialized for linear partial differential equations.
arXiv Detail & Related papers (2025-02-25T17:21:44Z)
Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations.<n>We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
arXiv Detail & Related papers (2024-11-28T13:16:20Z)
DiffusionPDE: Generative PDE-Solving Under Partial Observation [10.87702379899977]
We introduce a general framework for solving partial differential equations (PDEs) using generative diffusion models. We show that the learned generative priors lead to a versatile framework for accurately solving a wide range of PDEs under partial observation.
arXiv Detail & Related papers (2024-06-25T17:48:24Z)
Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers [53.79279286773326]
We present Unisolver, a novel Transformer model trained on diverse data and conditioned on diverse PDEs.<n>Unisolver achieves consistent state-of-the-art on three challenging large-scale benchmarks, showing impressive performance and generalizability.
arXiv Detail & Related papers (2024-05-27T15:34:35Z)
Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
arXiv Detail & Related papers (2023-11-30T22:34:57Z)
Neural Control of Parametric Solutions for High-dimensional Evolution PDEs [6.649496716171139]
We develop a novel computational framework to approximate solution operators of evolution partial differential equations (PDEs) We propose to approximate the solution operator of the PDE by learning the control vector field in the parameter space. This allows for substantially reduced computational cost to solve the evolution PDE with arbitrary initial conditions.
arXiv Detail & Related papers (2023-01-31T19:26:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.