Semi-Explicit Neural DAEs: Learning Long-Horizon Dynamical Systems with Algebraic Constraints

• URL: http://arxiv.org/abs/2505.20515v1
• Date: Mon, 26 May 2025 20:31:15 GMT
• Title: Semi-Explicit Neural DAEs: Learning Long-Horizon Dynamical Systems with Algebraic Constraints
• Authors: Avik Pal, Alan Edelman, Christopher Rackauckas,
• Abstract summary: We propose a method that explicitly enforces algebraic constraints by projecting each ODE step onto the constraint manifold.<n>PNODEs consistently outperform baselines across six benchmark problems achieving a mean constraint violation error below $10-10$.<n>These results show that constraint projection offers a simple strategy for learning physically consistent long-horizon dynamics.
• Score: 2.66269503676104
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the promise of scientific machine learning (SciML) in combining data-driven techniques with mechanistic modeling, existing approaches for incorporating hard constraints in neural differential equations (NDEs) face significant limitations. Scalability issues and poor numerical properties prevent these neural models from being used for modeling physical systems with complicated conservation laws. We propose Manifold-Projected Neural ODEs (PNODEs), a method that explicitly enforces algebraic constraints by projecting each ODE step onto the constraint manifold. This framework arises naturally from semi-explicit differential-algebraic equations (DAEs), and includes both a robust iterative variant and a fast approximation requiring a single Jacobian factorization. We further demonstrate that prior works on relaxation methods are special cases of our approach. PNODEs consistently outperform baselines across six benchmark problems achieving a mean constraint violation error below $10^{-10}$. Additionally, PNODEs consistently achieve lower runtime compared to other methods for a given level of error tolerance. These results show that constraint projection offers a simple strategy for learning physically consistent long-horizon dynamics.

Related papers

• Single-loop Algorithms for Stochastic Non-convex Optimization with Weakly-Convex Constraints [49.76332265680669]
  This paper examines a crucial subset of problems where both the objective and constraint functions are weakly convex.<n>Existing methods often face limitations, including slow convergence rates or reliance on double-loop designs.<n>We introduce a novel single-loop penalty-based algorithm to overcome these challenges.
  arXiv  Detail & Related papers  (2025-04-21T17:15:48Z)
• A Simultaneous Approach for Training Neural Differential-Algebraic Systems of Equations [0.4935512063616847]
  We study neural differential-algebraic systems of equations (DAEs), where some unknown relationships are learned from data.<n>We apply the simultaneous approach to neural DAE problems, resulting in a fully discretized nonlinear optimization problem.<n>We achieve promising results in terms of accuracy, model generalizability and computational cost, across different problem settings.
  arXiv  Detail & Related papers  (2025-04-07T01:26:55Z)
• Training Deep Learning Models with Norm-Constrained LMOs [56.00317694850397]
  We study optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball.<n>We propose a new family of algorithms that uses the LMO to adapt to the geometry of the problem and, perhaps surprisingly, show that they can be applied to unconstrained problems.
  arXiv  Detail & Related papers  (2025-02-11T13:10:34Z)
• Learning Controlled Stochastic Differential Equations [61.82896036131116]
  This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion. We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity. Our method is available as an open-source Python library.
  arXiv  Detail & Related papers  (2024-11-04T11:09:58Z)
• Projected Neural Differential Equations for Learning Constrained Dynamics [3.570367665112327]
  We introduce a new method for constraining neural differential equations based on projection of the learned vector field to the tangent space of the constraint manifold. PNDEs outperform existing methods while requiring fewer hyper parameters. The proposed approach demonstrates significant potential for enhancing the modeling of constrained dynamical systems.
  arXiv  Detail & Related papers  (2024-10-31T06:32:43Z)
• Constrained Synthesis with Projected Diffusion Models [47.56192362295252]
  This paper introduces an approach to generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional process of generative diffusion as a constrained distribution problem to ensure adherence to constraints.
  arXiv  Detail & Related papers  (2024-02-05T22:18:16Z)
• Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints [4.656302602746229]
  We propose stabilized neural differential equations (SNDEs) to enforce arbitrary manifold constraints for neural differential equations. Our approach is based on a stabilization term that, when added to the original dynamics, renders the constraint manifold provably stable. Due to its simplicity, our method is compatible with all common neural differential equation (NDE) models and broadly applicable.
  arXiv  Detail & Related papers  (2023-06-16T10:16:59Z)
• Neural Fields with Hard Constraints of Arbitrary Differential Order [61.49418682745144]
  We develop a series of approaches for enforcing hard constraints on neural fields. The constraints can be specified as a linear operator applied to the neural field and its derivatives. Our approaches are demonstrated in a wide range of real-world applications.
  arXiv  Detail & Related papers  (2023-06-15T08:33:52Z)
• Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics [97.38308257547186]
  Many NN approaches learn an end-to-end model that implicitly models both the governing PDE and material models. We argue that the governing PDEs are often well-known and should be explicitly enforced rather than learned. We introduce a new framework termed "Neural Constitutive Laws" (NCLaw) which utilizes a network architecture that strictly guarantees standard priors.
  arXiv  Detail & Related papers  (2023-04-27T17:42:24Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• Generalized Neural Closure Models with Interpretability [28.269731698116257]
  We develop a novel and versatile methodology of unified neural partial delay differential equations. We augment existing/low-fidelity dynamical models directly in their partial differential equation (PDE) forms with both Markovian and non-Markovian neural network (NN) closure parameterizations. We demonstrate the new generalized neural closure models (gnCMs) framework using four sets of experiments based on advecting nonlinear waves, shocks, and ocean acidification models.
  arXiv  Detail & Related papers  (2023-01-15T21:57:43Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Physics and Equality Constrained Artificial Neural Networks: Application to Partial Differential Equations [1.370633147306388]
  Physics-informed neural networks (PINNs) have been proposed to learn the solution of partial differential equations (PDE) Here, we show that this specific way of formulating the objective function is the source of severe limitations in the PINN approach. We propose a versatile framework that can tackle both inverse and forward problems.
  arXiv  Detail & Related papers  (2021-09-30T05:55:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.