Physics-Guided Transformer (PGT): Physics-Aware Attention Mechanism for PINNs

• URL: http://arxiv.org/abs/2603.27929v1
• Date: Mon, 30 Mar 2026 00:55:32 GMT
• Title: Physics-Guided Transformer (PGT): Physics-Aware Attention Mechanism for PINNs
• Abstract summary: We introduce the Physics-Guided Transformer (PGT), a neural architecture that embeds physical structure directly into the self-attention mechanism.<n>We evaluate PGT on the one-dimensional heat equation and two-dimensional incompressible Navier-Stokes systems.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing continuous physical fields from sparse, irregular observations is a central challenge in scientific machine learning, particularly for systems governed by partial differential equations (PDEs). Existing physics-informed methods typically enforce governing equations as soft penalty terms during optimization, often leading to gradient imbalance, instability, and degraded physical consistency under limited data. We introduce the Physics-Guided Transformer (PGT), a neural architecture that embeds physical structure directly into the self-attention mechanism. Specifically, PGT incorporates a heat-kernel-derived additive bias into attention logits, encoding diffusion dynamics and temporal causality within the representation. Query coordinates attend to these physics-conditioned context tokens, and the resulting features are decoded using a FiLM-modulated sinusoidal implicit network that adaptively controls spectral response. We evaluate PGT on the one-dimensional heat equation and two-dimensional incompressible Navier-Stokes systems. In sparse 1D reconstruction with 100 observations, PGT achieves a relative L2 error of 5.9e-3, significantly outperforming both PINNs and sinusoidal representations. In the 2D cylinder wake problem, PGT uniquely achieves both low PDE residual (8.3e-4) and competitive relative error (0.034), outperforming methods that optimize only one objective. These results demonstrate that embedding physics within attention improves stability, generalization, and physical fidelity under data-scarce conditions.

Related papers

• Physics-Integrated Operator Learning via Gaussian Splatting Representations [1.40323503250909]
  We introduce a representation-level approach to physics integration in which a feed-forward Gaussian splatting representation serves as a continuous interface between discretized solution fields and governing operators.<n>We evaluate the framework across two- and three-dimensional PDE systems, including advection, diffusion, nonlinear self-advection rollout, and reaction dynamics.
  arXiv  Detail & Related papers  (2026-08-25T04:18:08Z)
• Adjoint Method versus Physics-Informed Neural Networks in PDE-Constrained Inverse Problems [48.470700002941136]
  Inverse problems governed by partial differential equations (PDEs) are central to computational mechanics and are commonly solved by adjoint-based optimization.<n> physics-informed neural networks (PINNs) have emerged as a flexible alternative.<n>We present a fair comparison of adjoint optimization and PINNs for PDE-constrained inverse problems.
  arXiv  Detail & Related papers  (2026-06-10T17:07:09Z)
• Physics from Video: Identifiability of Time-Invariant Second-Order ODEs under Minimal Trajectory Conditions [60.2784641643737]
  Bridging the gap between visual realism and physical understanding is a core challenge for video-based world models.<n>We study the structural identifiability of continuous-time physical laws from raw pixels.<n>We prove that a level-set slope-coverage condition ensures the learned latent space is locally affine to the true physical state.
  arXiv  Detail & Related papers  (2026-05-27T13:22:29Z)
• Manifold-Orthogonal Dual-spectrum Extrapolation for Parameterized Physics-Informed Neural Networks [5.601252185916962]
  Physics-informed neural networks (PINNs) have achieved notable success in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose Manifold-Orthogonal Dual-spectrum Extrapolation (MODE), a lightweight micro-architecture designed for physics operator adaptation.
  arXiv  Detail & Related papers  (2026-03-14T04:52:50Z)
• Neural Field Thermal Tomography: A Differentiable Physics Framework for Non-Destructive Evaluation [3.697746606429496]
  We propose a differentiable physics framework for the quantitative 3D reconstruction of material properties from transient surface temperature measurements.<n>NeFTY parameterizes the 3D diffusivity field as a continuous neural field optimized through a rigorous numerical solver.<n>Our approach enforces thermodynamic laws as hard constraints while maintaining the memory efficiency required for high-resolution 3D tomography.
  arXiv  Detail & Related papers  (2026-03-11T17:59:42Z)
• Conditional Denoising Model as a Physical Surrogate Model [1.0616273526777913]
  We introduce a generative model designed to learn the geometry of the physical manifold itself.<n>By training the network to restore clean states from noisy ones, the model learns a vector field that points continuously towards the valid solution subspace.
  arXiv  Detail & Related papers  (2026-01-28T20:32:20Z)
• PEGNet: A Physics-Embedded Graph Network for Long-Term Stable Multiphysics Simulation [8.95344024479836]
  Physical phenomena governed by partial differential equations (PDEs) are important for scientific and engineering progress.<n>PEGNet is a physics-Embedded Graph Network that incorporates PDE-guided message passing to the redesign graph neural network architecture.<n>We show significant improvements in long-term prediction accuracy and physical consistency over existing methods.
  arXiv  Detail & Related papers  (2025-11-11T19:02:16Z)
• PHASE-Net: Physics-Grounded Harmonic Attention System for Efficient Remote Photoplethysmography Measurement [63.007237197267834]
  Existing deep learning methods are mostly physiological monitoring and lack theoretical robustness.<n>We propose a physics-informed r paradigm derived from the Navier-Stokes equations of hemodynamics, showing that the pulse signal follows a second-order system.<n>This provides a theoretical justification for using a Temporal Conal Network (TCN)<n>Phase-Net achieves state-of-the-art performance with strong efficiency, offering a theoretically grounded and deployment-ready r solution.
  arXiv  Detail & Related papers  (2025-09-29T14:36:45Z)
• KITINet: Kinetics Theory Inspired Network Architectures with PDE Simulation Approaches [43.872190335490515]
  This paper introduces KITINet, a novel architecture that reinterprets feature propagation through the lens of non-equilibrium particle dynamics.<n>At its core, we propose a residual module that models update as the evolution of a particle system.<n>This formulation mimics particle collisions and energy exchange, enabling adaptive feature refinement via physics-informed interactions.
  arXiv  Detail & Related papers  (2025-05-23T13:58:29Z)
• Enabling Automatic Differentiation with Mollified Graph Neural Operators [73.52999622724101]
  We propose the mollified graph neural operator ($m$GNO), the first method to leverage automatic differentiation and compute exact gradients on arbitrary geometries.<n>For a PDE example on regular grids, $m$GNO paired with autograd reduced the L2 relative data error by 20x compared to finite differences.<n>It can also solve PDEs on unstructured point clouds seamlessly, using physics losses only, at resolutions vastly lower than those needed for finite differences to be accurate enough.
  arXiv  Detail & Related papers  (2025-04-11T06:16:30Z)
• Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
  Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations.<n>We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
  arXiv  Detail & Related papers  (2024-11-28T13:16:20Z)
• Equivariant Graph Mechanics Networks with Constraints [83.38709956935095]
  We propose Graph Mechanics Network (GMN) which is efficient, equivariant and constraint-aware. GMN represents, by generalized coordinates, the forward kinematics information (positions and velocities) of a structural object. Extensive experiments support the advantages of GMN compared to the state-of-the-art GNNs in terms of prediction accuracy, constraint satisfaction and data efficiency.
  arXiv  Detail & Related papers  (2022-03-12T14:22:14Z)
• Robust Learning of Physics Informed Neural Networks [2.86989372262348]
  Physics-informed Neural Networks (PINNs) have been shown to be effective in solving partial differential equations. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE.
  arXiv  Detail & Related papers  (2021-10-26T00:10:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.