Differentiable Programming for Differential Equations: A Review

• URL: http://arxiv.org/abs/2406.09699v1
• Date: Fri, 14 Jun 2024 03:54:25 GMT
• Title: Differentiable Programming for Differential Equations: A Review
• Authors: Facundo Sapienza, Jordi Bolibar, Frank Schäfer, Brian Groenke, Avik Pal, Victor Boussange, Patrick Heimbach, Giles Hooker, Fernando Pérez, Per-Olof Persson, Christopher Rackauckas,
• Abstract summary: Differentiable programming is a cornerstone of modern scientific computing. Differentiating functions based on the numerical solution of differential equations is non-trivial. We provide a comprehensive review of existing techniques to compute derivatives of numerical solutions of differential equations.
• Score: 36.67198631261628
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The differentiable programming paradigm is a cornerstone of modern scientific computing. It refers to numerical methods for computing the gradient of a numerical model's output. Many scientific models are based on differential equations, where differentiable programming plays a crucial role in calculating model sensitivities, inverting model parameters, and training hybrid models that combine differential equations with data-driven approaches. Furthermore, recognizing the strong synergies between inverse methods and machine learning offers the opportunity to establish a coherent framework applicable to both fields. Differentiating functions based on the numerical solution of differential equations is non-trivial. Numerous methods based on a wide variety of paradigms have been proposed in the literature, each with pros and cons specific to the type of problem investigated. Here, we provide a comprehensive review of existing techniques to compute derivatives of numerical solutions of differential equations. We first discuss the importance of gradients of solutions of differential equations in a variety of scientific domains. Second, we lay out the mathematical foundations of the various approaches and compare them with each other. Third, we cover the computational considerations and explore the solutions available in modern scientific software. Last but not least, we provide best-practices and recommendations for practitioners. We hope that this work accelerates the fusion of scientific models and data, and fosters a modern approach to scientific modelling.

Related papers

• Towards true discovery of the differential equations [57.089645396998506]
  Differential equation discovery is a machine learning subfield used to develop interpretable models. This paper explores the prerequisites and tools for independent equation discovery without expert input.
  arXiv  Detail & Related papers  (2023-08-09T12:03:12Z)
• Symbolic Recovery of Differential Equations: The Identifiability Problem [52.158782751264205]
  Symbolic recovery of differential equations is the ambitious attempt at automating the derivation of governing equations. We provide both necessary and sufficient conditions for a function to uniquely determine the corresponding differential equation. We then use our results to devise numerical algorithms aiming to determine whether a function solves a differential equation uniquely.
  arXiv  Detail & Related papers  (2022-10-15T17:32:49Z)
• D-CIPHER: Discovery of Closed-form Partial Differential Equations [80.46395274587098]
  We propose D-CIPHER, which is robust to measurement artifacts and can uncover a new and very general class of differential equations. We further design a novel optimization procedure, CoLLie, to help D-CIPHER search through this class efficiently.
  arXiv  Detail & Related papers  (2022-06-21T17:59:20Z)
• Accelerating numerical methods by gradient-based meta-solving [15.90188271828615]
  In science and engineering applications, it is often required to solve similar computational problems repeatedly. We propose a gradient-based algorithm to solve them in a unified way. We demonstrate the performance and versatility of our method through theoretical analysis and numerical experiments.
  arXiv  Detail & Related papers  (2022-06-17T07:31:18Z)
• Neural Laplace: Learning diverse classes of differential equations in the Laplace domain [86.52703093858631]
  We propose a unified framework for learning diverse classes of differential equations (DEs) including all the aforementioned ones. Instead of modelling the dynamics in the time domain, we model it in the Laplace domain, where the history-dependencies and discontinuities in time can be represented as summations of complex exponentials. In the experiments, Neural Laplace shows superior performance in modelling and extrapolating the trajectories of diverse classes of DEs.
  arXiv  Detail & Related papers  (2022-06-10T02:14:59Z)
• Differentiable Physics: A Position Piece [0.0]
  We argue that differentiable physics offers a new paradigm for modeling physical phenomena by combining classical analytic solutions with numerical methodology using the bridge of differentiable programming.
  arXiv  Detail & Related papers  (2021-09-14T15:08:54Z)
• A Probabilistic State Space Model for Joint Inference from Differential Equations and Data [23.449725313605835]
  We show a new class of solvers for ordinary differential equations (ODEs) that phrase the solution process directly in terms of Bayesian filtering. It then becomes possible to perform approximate Bayesian inference on the latent force as well as the ODE solution in a single, linear complexity pass of an extended Kalman filter. We demonstrate the expressiveness and performance of the algorithm by training a non-parametric SIRD model on data from the COVID-19 outbreak.
  arXiv  Detail & Related papers  (2021-03-18T10:36:09Z)
• Symbolically Solving Partial Differential Equations using Deep Learning [5.1964883240501605]
  We describe a neural-based method for generating exact or approximate solutions to differential equations. Unlike other neural methods, our system returns symbolic expressions that can be interpreted directly.
  arXiv  Detail & Related papers  (2020-11-12T22:16:03Z)
• A Neuro-Symbolic Method for Solving Differential and Functional Equations [6.899578710832262]
  We introduce a method for generating symbolic expressions to solve differential equations. Unlike existing methods, our system does not require learning a language model over symbolic mathematics. We show how the system can be effortlessly generalized to find symbolic solutions to other mathematical tasks.
  arXiv  Detail & Related papers  (2020-11-04T17:13:25Z)
• The data-driven physical-based equations discovery using evolutionary approach [77.34726150561087]
  We describe the algorithm for the mathematical equations discovery from the given observations data. The algorithm combines genetic programming with the sparse regression. It could be used for governing analytical equation discovery as well as for partial differential equations (PDE) discovery.
  arXiv  Detail & Related papers  (2020-04-03T17:21:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.