Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models

• URL: http://arxiv.org/abs/2505.10930v2
• Date: Sat, 31 May 2025 03:05:23 GMT
• Title: Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models
• Authors: Congcong Zhu, Xiaoyan Xu, Jiayue Han, Jingrun Chen,
• Abstract summary: We propose physics-informed temporal alignment (PITA), a self-supervised learning framework inspired by inverse problem solving.<n>PITA aligns the physical dynamics discovered at different time steps on each given PDE trajectory by integrating physics-informed constraints into the self-supervision signal.<n>Experiments show that PITA significantly enhances the accuracy and robustness of existing foundation models on diverse time-dependent PDE data.
• Score: 2.57401182252473
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Auto-regressive partial differential equation (PDE) foundation models have shown great potential in handling time-dependent data. However, these models suffer from the shortcut problem deeply rooted in auto-regressive prediction, causing error accumulation. The challenge becomes particularly evident for out-of-distribution data, as the pretraining performance may approach random model initialization for downstream tasks with long-term dynamics. To deal with this problem, we propose physics-informed temporal alignment (PITA), a self-supervised learning framework inspired by inverse problem solving. Specifically, PITA aligns the physical dynamics discovered at different time steps on each given PDE trajectory by integrating physics-informed constraints into the self-supervision signal. The alignment is derived from observation data without relying on known physics priors, indicating strong generalization ability to the out-of-distribution data. Extensive experiments show that PITA significantly enhances the accuracy and robustness of existing foundation models on diverse time-dependent PDE data. The code is available at https://github.com/SCAILab-USTC/PITA.

Related papers

• Hybrid Generative Modeling for Incomplete Physics: Deep Grey-Box Meets Optimal Transport [48.06072022424773]
  Many real-world systems are described only approximately with missing or unknown terms in the equations.<n>This makes the distribution of the physics model differ from the true data-generating process (DGP)<n>We present a novel hybrid generative model approach combining deep grey-box modelling with Optimal Transport (OT) methods to enhance incomplete physics models.
  arXiv  Detail & Related papers  (2025-06-27T13:23:27Z)
• Physics-informed Reduced Order Modeling of Time-dependent PDEs via Differentiable Solvers [0.0]
  We propose Physics-informed ROM ($Phi$-ROM) by incorporating differentiable PDE solvers into the training procedure.<n>Specifically, the latent space dynamics and its dependence on PDE parameters are shaped directly by the governing physics encoded in the solver.<n>Our model outperforms state-of-the-art data-driven ROMs and other physics-informed strategies by accurately generalizing to new dynamics arising from unseen parameters.
  arXiv  Detail & Related papers  (2025-05-20T16:47:04Z)
• PINT: Physics-Informed Neural Time Series Models with Applications to Long-term Inference on WeatherBench 2m-Temperature Data [2.6842667886955196]
  PINT (Physics-Informed Neural Time Series Models) is a framework that integrates physical constraints into neural time series models.<n>We apply PINT to the ERA5 WeatherBench dataset, focusing on long-term forecasting of 2m-temperature data.
  arXiv  Detail & Related papers  (2025-02-06T12:19:34Z)
• Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
  Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations.<n>We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
  arXiv  Detail & Related papers  (2024-11-28T13:16:20Z)
• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which generalizes weather forecasts to finer-grained temporal scales beyond training dataset.<n>Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale.<n>We also introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers [40.097474800631]
  Time-dependent partial differential equations (PDEs) are ubiquitous in science and engineering. Deep neural network based surrogates have gained increased interest.
  arXiv  Detail & Related papers  (2023-08-10T17:53:05Z)
• A Neural PDE Solver with Temporal Stencil Modeling [44.97241931708181]
  Recent Machine Learning (ML) models have shown new promises in capturing important dynamics in high-resolution signals. This study shows that significant information is often lost in the low-resolution down-sampled features. We propose a new approach, which combines the strengths of advanced time-series sequence modeling and state-of-the-art neural PDE solvers.
  arXiv  Detail & Related papers  (2023-02-16T06:13:01Z)
• Deep Physics Corrector: A physics enhanced deep learning architecture for solving stochastic differential equations [0.0]
  We propose a novel gray-box modeling algorithm for physical systems governed by differential equations (SDE) The proposed approach, referred to as the Deep Physics Corrector (DPC), blends approximate physics represented in terms of SDE with deep neural network (DNN) We illustrate the performance of the proposed DPC on four benchmark examples from the literature.
  arXiv  Detail & Related papers  (2022-09-20T14:30:07Z)
• Deep Convolutional Architectures for Extrapolative Forecast in Time-dependent Flow Problems [0.0]
  Deep learning techniques are employed to model the system dynamics for advection dominated problems. These models take as input a sequence of high-fidelity vector solutions for consecutive time-steps obtained from the PDEs. Non-intrusive reduced-order modelling techniques such as deep auto-encoder networks are utilized to compress the high-fidelity snapshots.
  arXiv  Detail & Related papers  (2022-09-18T03:45:56Z)
• Physics Informed RNN-DCT Networks for Time-Dependent Partial Differential Equations [62.81701992551728]
  We present a physics-informed framework for solving time-dependent partial differential equations. Our model utilizes discrete cosine transforms to encode spatial and recurrent neural networks. We show experimental results on the Taylor-Green vortex solution to the Navier-Stokes equations.
  arXiv  Detail & Related papers  (2022-02-24T20:46:52Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.