A Residual Guided strategy with Generative Adversarial Networks in training Physics-Informed Transformer Networks

• URL: http://arxiv.org/abs/2508.00855v1
• Date: Tue, 15 Jul 2025 03:45:42 GMT
• Title: A Residual Guided strategy with Generative Adversarial Networks in training Physics-Informed Transformer Networks
• Authors: Ziyang Zhang, Feifan Zhang, Weidong Tang, Lei Shi, Tailai Chen,
• Abstract summary: We propose a novel Residual Guided Training strategy for Physics-In Transformer via Generative Adrative Network (GAN)<n>Our framework integrates a Transformer to inherently capture temporal correlations through autoregressive processing, coupled with a residual-aware GAN.<n>Experiments on the Allen-Cahn-Gordon, and Navier-Stokes equations demonstrate significant improvements, relative relative MSE reductions of up three orders of magnitude compared to baseline methods.
• Score: 8.614387766858496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nonlinear partial differential equations (PDEs) are pivotal in modeling complex physical systems, yet traditional Physics-Informed Neural Networks (PINNs) often struggle with unresolved residuals in critical spatiotemporal regions and violations of temporal causality. To address these limitations, we propose a novel Residual Guided Training strategy for Physics-Informed Transformer via Generative Adversarial Networks (GAN). Our framework integrates a decoder-only Transformer to inherently capture temporal correlations through autoregressive processing, coupled with a residual-aware GAN that dynamically identifies and prioritizes high-residual regions. By introducing a causal penalty term and an adaptive sampling mechanism, the method enforces temporal causality while refining accuracy in problematic domains. Extensive numerical experiments on the Allen-Cahn, Klein-Gordon, and Navier-Stokes equations demonstrate significant improvements, achieving relative MSE reductions of up to three orders of magnitude compared to baseline methods. This work bridges the gap between deep learning and physics-driven modeling, offering a robust solution for multiscale and time-dependent PDE systems.

Related papers

• PhysicsCorrect: A Training-Free Approach for Stable Neural PDE Simulations [4.7903561901859355]
  We present PhysicsCorrect, a training-free correction framework that enforces PDE consistency at each prediction step.<n>Our key innovation is an efficient caching strategy that precomputes the Jacobian and its pseudoinverse during an offline warm-up phase.<n>Across three representative PDE systems, PhysicsCorrect reduces prediction errors by up to 100x while adding negligible inference time.
  arXiv  Detail & Related papers  (2025-07-03T01:22:57Z)
• Accelerating Multiscale Modeling with Hybrid Solvers: Coupling FEM and Neural Operators with Domain Decomposition [3.0635300721402228]
  This work introduces a novel hybrid framework that integrates PI-NO with finite element method (FE) through domain decomposition.<n>The framework efficacy has been validated across a range of problems, spanning static, quasi-static, and dynamic regimes.<n>Our study shows that our hybrid solver: (1) maintains solution continuity across subdomain interfaces, (2) reduces computational costs by eliminating fine mesh requirements, (3) mitigates error accumulation in time dependent simulations, and (4) enables automatic adaptation to evolving physical phenomena.
  arXiv  Detail & Related papers  (2025-04-15T16:54:04Z)
• Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
  Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations.<n>We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
  arXiv  Detail & Related papers  (2024-11-28T13:16:20Z)
• A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks [52.5899851000193]
  We develop an ODE based IVP solver which prevents the network from getting ill-conditioned and runs in time linear in the number of parameters. We show that current methods based on this approach suffer from two key issues. First, following the ODE produces an uncontrolled growth in the conditioning of the problem, ultimately leading to unacceptably large numerical errors.
  arXiv  Detail & Related papers  (2023-04-28T17:28:18Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equations [64.78260098263489]
  In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs) We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases. We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
  arXiv  Detail & Related papers  (2022-10-14T15:01:32Z)
• LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data [47.49194807524502]
  We propose LordNet, a tunable and efficient neural network for modeling entanglements. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements can be well modeled by the LordNet.
  arXiv  Detail & Related papers  (2022-06-19T14:41:08Z)
• Multilayer Perceptron Based Stress Evolution Analysis under DC Current Stressing for Multi-segment Wires [8.115870370527324]
  Electromigration (EM) is one of the major concerns in the reliability analysis of very large scale integration (VLSI) systems. Traditional methods are often not sufficiently accurate, leading to undesirable over-design especially in advanced technology nodes. We propose an approach using multilayer perceptrons (MLP) to compute stress evolution in the interconnect trees during the void nucleation phase.
  arXiv  Detail & Related papers  (2022-05-17T07:38:20Z)
• Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics [11.981731023317945]
  Physics-informed deep learning (PiDL) addresses these challenges by incorporating physical principles into the model. We propose to leverage physics prior knowledge by baking'' the discretized governing equations into the neural network architecture. This method, embedding discretized PDEs through convolutional residual networks in a multi-resolution setting, largely improves the generalizability and long-term prediction.
  arXiv  Detail & Related papers  (2022-05-09T01:27:58Z)
• Physics Informed RNN-DCT Networks for Time-Dependent Partial Differential Equations [62.81701992551728]
  We present a physics-informed framework for solving time-dependent partial differential equations. Our model utilizes discrete cosine transforms to encode spatial and recurrent neural networks. We show experimental results on the Taylor-Green vortex solution to the Navier-Stokes equations.
  arXiv  Detail & Related papers  (2022-02-24T20:46:52Z)
• Physics informed neural networks for continuum micromechanics [68.8204255655161]
  Recently, physics informed neural networks have successfully been applied to a broad variety of problems in applied mathematics and engineering. Due to the global approximation, physics informed neural networks have difficulties in displaying localized effects and strong non-linear solutions by optimization. It is shown, that the domain decomposition approach is able to accurately resolve nonlinear stress, displacement and energy fields in heterogeneous microstructures obtained from real-world $mu$CT-scans.
  arXiv  Detail & Related papers  (2021-10-14T14:05:19Z)
• Physics-informed attention-based neural network for solving non-linear partial differential equations [6.103365780339364]
  Physics-Informed Neural Networks (PINNs) have enabled significant improvements in modelling physical processes. PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Here, we address the question of which network architectures are best suited to learn the complex behavior of non-linear PDEs.
  arXiv  Detail & Related papers  (2021-05-17T14:29:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.