Regime-Aware Time Weighting for Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2407.21642v2
• Date: Sat, 16 Aug 2025 13:51:28 GMT
• Title: Regime-Aware Time Weighting for Physics-Informed Neural Networks
• Authors: Gabriel Turinici,
• Abstract summary: We introduce a novel method to handle the time dimension when PINNs are used to solve time-dependent differential equations.<n>Our approach is grounded in theoretical insights derived from the Lyapunovs, which quantify the sensitivity of solutions to perturbations over time.<n> Numerical experiments on challenging benchmarks, including the chaotic Lorenz system and the Burgers' equation, demonstrate the effectiveness and robustness of the proposed method.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce a novel method to handle the time dimension when Physics-Informed Neural Networks (PINNs) are used to solve time-dependent differential equations; our proposal focuses on how time sampling and weighting strategies affect solution quality. While previous methods proposed heuristic time-weighting schemes, our approach is grounded in theoretical insights derived from the Lyapunov exponents, which quantify the sensitivity of solutions to perturbations over time. This principled methodology automatically adjusts weights based on the stability regime of the system -- whether chaotic, periodic, or stable. Numerical experiments on challenging benchmarks, including the chaotic Lorenz system and the Burgers' equation, demonstrate the effectiveness and robustness of the proposed method. Compared to existing techniques, our approach offers improved convergence and accuracy without requiring additional hyperparameter tuning. The findings underline the importance of incorporating causality and dynamical system behavior into PINN training strategies, providing a robust framework for solving time-dependent problems with enhanced reliability.

Related papers

• MAD-NG: Meta-Auto-Decoder Neural Galerkin Method for Solving Parametric Partial Differential Equations [5.767740428776141]
  Parametric partial differential equations (PDEs) are fundamental for modeling a wide range of physical and engineering systems.<n>Traditional neural network-based solvers, such as Physics-Informed Neural Networks (PINNs) and Deep Galerkin Methods, often face challenges in generalization and long-time prediction efficiency.<n>We propose a novel and scalable framework that significantly enhances the Neural Galerkin Method (NGM) by incorporating the Meta-Auto-Decoder (MAD) paradigm.
  arXiv  Detail & Related papers  (2025-12-25T11:27:40Z)
• On-line learning of dynamic systems: sparse regression meets Kalman filtering [41.99844472131922]
  We extend sparsity-driven approaches to real-time learning by integrating a cornerstone algorithm from control theory -- the Kalman filter (KF)<n>The resulting Sindy Kalman Filter (KF) unifies both frameworks by treating unknown system parameters as state variables.<n>We demonstrate KF's effectiveness in the real-time identification of a sparse nonlinear aircraft model built from real flight data.
  arXiv  Detail & Related papers  (2025-11-14T11:24:05Z)
• Multiresolution Analysis and Statistical Thresholding on Dynamic Networks [49.09073800467438]
  ANIE (Adaptive Network Intensity Estimation) is a multi-resolution framework designed to automatically identify the time scales at which network structure evolves.<n>We show that ANIE adapts to the appropriate time resolution and is able to capture sharp structural changes while remaining robust to noise.
  arXiv  Detail & Related papers  (2025-06-01T22:55:55Z)
• Generative System Dynamics in Recurrent Neural Networks [56.958984970518564]
  We investigate the continuous time dynamics of Recurrent Neural Networks (RNNs) We show that skew-symmetric weight matrices are fundamental to enable stable limit cycles in both linear and nonlinear configurations. Numerical simulations showcase how nonlinear activation functions not only maintain limit cycles, but also enhance the numerical stability of the system integration process.
  arXiv  Detail & Related papers  (2025-04-16T10:39:43Z)
• Hard constraint learning approaches with trainable influence functions for evolutionary equations [8.812375888020398]
  This paper develops a novel deep learning approach for solving evolutionary equations.<n>Sequential learning strategies divide a large temporal domain into multiple subintervals and solve them one by one in a chronological order.<n>The improved hard constraint strategy strictly ensures the continuity and smoothness of the PINN solution at time interval nodes.
  arXiv  Detail & Related papers  (2025-02-21T07:54:01Z)
• Neural equilibria for long-term prediction of nonlinear conservation laws [38.88412478541979]
  We introduce Neural Discrete Equilibrium (NeurDE), a machine learning (ML) approach for long-term forecasting of flow phenomena. We show that NeurDE enables accurate prediction of compressible flows, including supersonic flows, while tracking shocks over hundreds of time steps.
  arXiv  Detail & Related papers  (2025-01-12T21:02:20Z)
• Random-matrix approach to time-dependent forcing in many-body quantum systems [0.36832029288386137]
  We develop a recently proposed nonlinear response theory based on typicality and random-matrix methods. We derive analytical approximations of the characteristic response function for the two limiting cases of fast driving and of strong and short-ranged-in-energy driving. We verify all predictions by numerical examples and discuss the theory's scope and limitations.
  arXiv  Detail & Related papers  (2024-09-16T07:29:30Z)
• Enriched Physics-informed Neural Networks for Dynamic Poisson-Nernst-Planck Systems [0.8192907805418583]
  This paper proposes a meshless deep learning algorithm, enriched physics-informed neural networks (EPINNs) to solve dynamic Poisson-Nernst-Planck (PNP) equations. The EPINNs takes the traditional physics-informed neural networks as the foundation framework, and adds the adaptive loss weight to balance the loss functions. Numerical results indicate that the new method has better applicability than traditional numerical methods in solving such coupled nonlinear systems.
  arXiv  Detail & Related papers  (2024-02-01T02:57:07Z)
• PINNs-Based Uncertainty Quantification for Transient Stability Analysis [22.116325319900973]
  We introduce a novel application of Physics-Informed Neural Networks (PINNs), specifically an Ensemble of PINNs (E-PINNs) to estimate critical parameters. E-PINNs capitalize on the underlying physical principles of swing equations to provide a robust solution. The study advances the application of machine learning in power system stability, paving the way for reliable and computationally efficient transient stability analysis.
  arXiv  Detail & Related papers  (2023-11-21T19:21:49Z)
• TANGO: Time-Reversal Latent GraphODE for Multi-Agent Dynamical Systems [43.39754726042369]
  We propose a simple-yet-effective self-supervised regularization term as a soft constraint that aligns the forward and backward trajectories predicted by a continuous graph neural network-based ordinary differential equation (GraphODE) It effectively imposes time-reversal symmetry to enable more accurate model predictions across a wider range of dynamical systems under classical mechanics. Experimental results on a variety of physical systems demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2023-10-10T08:52:16Z)
• Tipping Point Forecasting in Non-Stationary Dynamics on Function Spaces [78.08947381962658]
  Tipping points are abrupt, drastic, and often irreversible changes in the evolution of non-stationary dynamical systems. We learn the evolution of such non-stationary systems using a novel recurrent neural operator (RNO), which learns mappings between function spaces. We propose a conformal prediction framework to forecast tipping points by monitoring deviations from physics constraints.
  arXiv  Detail & Related papers  (2023-08-17T05:42:27Z)
• A Neural RDE approach for continuous-time non-Markovian stochastic control problems [4.155942878350882]
  We propose a novel framework for continuous-time non-Markovian control problems by means of neural rough differential equations (Neural RDEs) Non-Markovianity naturally arises in control problems due to the time delay effects in the system coefficients or the driving noises. By modelling the control process as the solution of a Neural RDE driven by the state process, we show that the control-state joint dynamics are governed by an uncontrolled, augmented Neural RDE.
  arXiv  Detail & Related papers  (2023-06-25T14:30:33Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics [97.38308257547186]
  Many NN approaches learn an end-to-end model that implicitly models both the governing PDE and material models. We argue that the governing PDEs are often well-known and should be explicitly enforced rather than learned. We introduce a new framework termed "Neural Constitutive Laws" (NCLaw) which utilizes a network architecture that strictly guarantees standard priors.
  arXiv  Detail & Related papers  (2023-04-27T17:42:24Z)
• Dynamics with autoregressive neural quantum states: application to critical quench dynamics [41.94295877935867]
  We present an alternative general scheme that enables one to capture long-time dynamics of quantum systems in a stable fashion. We apply the scheme to time-dependent quench dynamics by investigating the Kibble-Zurek mechanism in the two-dimensional quantum Ising model.
  arXiv  Detail & Related papers  (2022-09-07T15:50:00Z)
• Extended critical regimes of deep neural networks [0.0]
  We show that heavy-tailed weights enable the emergence of an extended critical regime without fine-tuning parameters. In this extended critical regime, DNNs exhibit rich and complex propagation dynamics across layers. We provide a theoretical guide for the design of efficient neural architectures.
  arXiv  Detail & Related papers  (2022-03-24T10:15:50Z)
• Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
  Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design. We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
  arXiv  Detail & Related papers  (2021-05-17T08:36:18Z)
• Role of stochastic noise and generalization error in the time propagation of neural-network quantum states [0.0]
  Neural-network quantum states (NQS) have been shown to be a suitable variational ansatz to simulate out-of-equilibrium dynamics. We show that stable and accurate time propagation can be achieved in regimes of sufficiently regularized variational dynamics.
  arXiv  Detail & Related papers  (2021-05-03T17:55:09Z)
• Neural ODE Processes [64.10282200111983]
  We introduce Neural ODE Processes (NDPs), a new class of processes determined by a distribution over Neural ODEs. We show that our model can successfully capture the dynamics of low-dimensional systems from just a few data-points.
  arXiv  Detail & Related papers  (2021-03-23T09:32:06Z)
• Physics-Informed Echo State Networks [5.8010446129208155]
  We propose a physics-informed Echo State Network (ESN) to predict the evolution of chaotic systems. Compared to conventional ESNs, the physics-informed ESNs are trained to solve supervised learning tasks. The proposed framework shows the potential of using machine learning combined with prior physical knowledge to improve the time-accurate prediction of chaotic systems.
  arXiv  Detail & Related papers  (2020-10-31T11:47:33Z)
• Bayesian Perceptron: Towards fully Bayesian Neural Networks [5.5510642465908715]
  Training and predictions of a perceptron are performed within the Bayesian inference framework in closed-form. The weights and the predictions of the perceptron are considered Gaussian random variables. This approach requires no computationally expensive gradient calculations and further allows sequential learning.
  arXiv  Detail & Related papers  (2020-09-03T15:08:49Z)
• Time-Reversal Symmetric ODE Network [138.02741983098454]
  Time-reversal symmetry is a fundamental property that frequently holds in classical and quantum mechanics. We propose a novel loss function that measures how well our ordinary differential equation (ODE) networks comply with this time-reversal symmetry. We show that, even for systems that do not possess the full time-reversal symmetry, TRS-ODENs can achieve better predictive performances over baselines.
  arXiv  Detail & Related papers  (2020-07-22T12:19:40Z)
• The role of boundary conditions in quantum computations of scattering observables [58.720142291102135]
  Quantum computing may offer the opportunity to simulate strongly-interacting field theories, such as quantum chromodynamics, with physical time evolution. As with present-day calculations, quantum computation strategies still require the restriction to a finite system size. We quantify the volume effects for various $1+1$D Minkowski-signature quantities and show that these can be a significant source of systematic uncertainty.
  arXiv  Detail & Related papers  (2020-07-01T17:43:11Z)
• Robust Reinforcement Learning with Wasserstein Constraint [49.86490922809473]
  We show the existence of optimal robust policies, provide a sensitivity analysis for the perturbations, and then design a novel robust learning algorithm. The effectiveness of the proposed algorithm is verified in the Cart-Pole environment.
  arXiv  Detail & Related papers  (2020-06-01T13:48:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.