Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation

• URL: http://arxiv.org/abs/2503.16666v1
• Date: Thu, 20 Mar 2025 19:26:54 GMT
• Title: Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation
• Authors: Qiyu Kang, Xuhao Li, Kai Zhao, Wenjun Cui, Yanan Zhao, Weihua Deng, Wee Peng Tay,
• Abstract summary: Fractional-order differential equations (FDEs) enhance traditional differential equations by extending the order of differential operators from integers to real numbers.<n>Recent progress at the intersection of FDEs and deep learning has catalyzed a new wave of innovative models.<n>We propose a scalable adjoint backpropagation method for training neural FDEs by solving an augmented FDE backward in time.
• Score: 19.331085375128048
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fractional-order differential equations (FDEs) enhance traditional differential equations by extending the order of differential operators from integers to real numbers, offering greater flexibility in modeling complex dynamical systems with nonlocal characteristics. Recent progress at the intersection of FDEs and deep learning has catalyzed a new wave of innovative models, demonstrating the potential to address challenges such as graph representation learning. However, training neural FDEs has primarily relied on direct differentiation through forward-pass operations in FDE numerical solvers, leading to increased memory usage and computational complexity, particularly in large-scale applications. To address these challenges, we propose a scalable adjoint backpropagation method for training neural FDEs by solving an augmented FDE backward in time, which substantially reduces memory requirements. This approach provides a practical neural FDE toolbox and holds considerable promise for diverse applications. We demonstrate the effectiveness of our method in several tasks, achieving performance comparable to baseline models while significantly reducing computational overhead.

Related papers

• Neural Variable-Order Fractional Differential Equation Networks [26.06048802504022]
  We introduce the Neural Variable-Order Fractional Differential Equation network (NvoFDE)<n>NvoFDE is a novel neural network framework that integrates variable-order fractional derivatives with learnable neural networks.<n>Our results demonstrate that NvoFDE outperforms traditional constant-order fractional and integer models across a range of tasks.
  arXiv  Detail & Related papers  (2025-03-20T14:54:19Z)
• Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
  Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations.<n>We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
  arXiv  Detail & Related papers  (2024-11-28T13:16:20Z)
• Partial-differential-algebraic equations of nonlinear dynamics by Physics-Informed Neural-Network: (I) Operator splitting and framework assessment [51.3422222472898]
  Several forms for constructing novel physics-informed-networks (PINN) for the solution of partial-differential-algebraic equations are proposed. Among these novel methods are the PDE forms, which evolve from the lower-level form with fewer unknown dependent variables to higher-level form with more dependent variables.
  arXiv  Detail & Related papers  (2024-07-13T22:48:17Z)
• Neural Fractional Differential Equations [2.812395851874055]
  Fractional Differential Equations (FDEs) are essential tools for modelling complex systems in science and engineering. We propose the Neural FDE, a novel deep neural network architecture that adjusts a FDE to the dynamics of data.
  arXiv  Detail & Related papers  (2024-03-05T07:45:29Z)
• Latent Neural PDE Solver: a reduced-order modelling framework for partial differential equations [5.949599220326208]
  We propose to learn the dynamics of the system in the latent space with much coarser discretizations.<n>A non-linear autoencoder is first trained to project the full-order representation of the system onto the mesh-reduced space.<n>We showcase that it has competitive accuracy and efficiency compared to the neural PDE solver that operates on full-order space.
  arXiv  Detail & Related papers  (2024-02-27T19:36:27Z)
• Efficient Neural PDE-Solvers using Quantization Aware Training [71.0934372968972]
  We show that quantization can successfully lower the computational cost of inference while maintaining performance. Our results on four standard PDE datasets and three network architectures show that quantization-aware training works across settings and three orders of FLOPs magnitudes.
  arXiv  Detail & Related papers  (2023-08-14T09:21:19Z)
• Neural Basis Functions for Accelerating Solutions to High Mach Euler Equations [63.8376359764052]
  We propose an approach to solving partial differential equations (PDEs) using a set of neural networks. We regress a set of neural networks onto a reduced order Proper Orthogonal Decomposition (POD) basis. These networks are then used in combination with a branch network that ingests the parameters of the prescribed PDE to compute a reduced order approximation to the PDE.
  arXiv  Detail & Related papers  (2022-08-02T18:27:13Z)
• LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data [47.49194807524502]
  We propose LordNet, a tunable and efficient neural network for modeling entanglements. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements can be well modeled by the LordNet.
  arXiv  Detail & Related papers  (2022-06-19T14:41:08Z)
• Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics [11.981731023317945]
  Physics-informed deep learning (PiDL) addresses these challenges by incorporating physical principles into the model. We propose to leverage physics prior knowledge by baking'' the discretized governing equations into the neural network architecture. This method, embedding discretized PDEs through convolutional residual networks in a multi-resolution setting, largely improves the generalizability and long-term prediction.
  arXiv  Detail & Related papers  (2022-05-09T01:27:58Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Adversarial Multi-task Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations [9.823102211212582]
  We introduce the novel approach of employing multi-task learning techniques, the uncertainty-weighting loss and the gradients surgery, in the context of learning PDE solutions. In the experiments, our proposed methods are found to be effective and reduce the error on the unseen data points as compared to the previous approaches.
  arXiv  Detail & Related papers  (2021-04-29T13:17:46Z)
• Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
  Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
  arXiv  Detail & Related papers  (2020-09-08T13:26:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.