Differential Equations for Continuous-Time Deep Learning

• URL: http://arxiv.org/abs/2401.03965v1
• Date: Mon, 8 Jan 2024 15:40:11 GMT
• Title: Differential Equations for Continuous-Time Deep Learning
• Authors: Lars Ruthotto
• Abstract summary: It primarily targets readers familiar with ordinary and partial differential equations and their analysis. We will see how neural ODEs can provide new insights into deep learning and a foundation for more efficient algorithms.
• Score: 2.163953126557988
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This short, self-contained article seeks to introduce and survey continuous-time deep learning approaches that are based on neural ordinary differential equations (neural ODEs). It primarily targets readers familiar with ordinary and partial differential equations and their analysis who are curious to see their role in machine learning. Using three examples from machine learning and applied mathematics, we will see how neural ODEs can provide new insights into deep learning and a foundation for more efficient algorithms.

Related papers

• A Unified Framework for Neural Computation and Learning Over Time [56.44910327178975]
  Hamiltonian Learning is a novel unified framework for learning with neural networks "over time" It is based on differential equations that: (i) can be integrated without the need of external software solvers; (ii) generalize the well-established notion of gradient-based learning in feed-forward and recurrent networks; (iii) open to novel perspectives.
  arXiv  Detail & Related papers  (2024-09-18T14:57:13Z)
• An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning [5.75055574132362]
  approximation of solutions of partial differential equations with numerical algorithms is a central topic in applied mathematics. One class of methods which has received a lot of attention in recent years are machine learning-based methods. This article aims to provide an introduction to some of these methods and the mathematical theory on which they are based.
  arXiv  Detail & Related papers  (2024-08-23T16:57:34Z)
• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• Stop overkilling simple tasks with black-box models and use transparent models instead [57.42190785269343]
  Deep learning approaches are able to extract features autonomously from raw data. This allows for bypassing the feature engineering process. Deep learning strategies often outperform traditional models in terms of accuracy.
  arXiv  Detail & Related papers  (2023-02-06T14:28:49Z)
• A Survey of Deep Learning for Mathematical Reasoning [71.88150173381153]
  We review the key tasks, datasets, and methods at the intersection of mathematical reasoning and deep learning over the past decade. Recent advances in large-scale neural language models have opened up new benchmarks and opportunities to use deep learning for mathematical reasoning.
  arXiv  Detail & Related papers  (2022-12-20T18:46:16Z)
• Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics [77.34726150561087]
  Recent developments in artificial neural networks, particularly deep learning (DL), are reviewed in detail. Both hybrid and pure machine learning (ML) methods are discussed. History and limitations of AI are recounted and discussed, with particular attention at pointing out misstatements or misconceptions of the classics.
  arXiv  Detail & Related papers  (2022-12-18T02:03:00Z)
• Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems [1.7150329136228712]
  More and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs) Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than standard feed-forward neural networks, recurrent neural networks, or convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering.
  arXiv  Detail & Related papers  (2022-12-06T16:53:34Z)
• Unsupervised Legendre-Galerkin Neural Network for Stiff Partial Differential Equations [9.659504024299896]
  We propose an unsupervised machine learning algorithm based on the Legendre-Galerkin neural network to find an accurate approximation to the solution of different types of PDEs. The proposed neural network is applied to the general 1D and 2D PDEs as well as singularly perturbed PDEs that possess boundary layer behavior.
  arXiv  Detail & Related papers  (2022-07-21T00:47:47Z)
• Learning differential equations from data [0.0]
  In recent times, due to the abundance of data, there is an active search for data-driven methods to learn Differential equation models from data. We propose a forward-Euler based neural network model and test its performance by learning ODEs from data using different number of hidden layers and different neural network width.
  arXiv  Detail & Related papers  (2022-05-23T17:36:28Z)
• On Neural Differential Equations [13.503274710499971]
  In particular, neural differential equations (NDEs) demonstrate that neural networks and differential equations are two sides of the same coin. NDEs are suitable for tackling generative problems, dynamical systems, and time series. NDEs offer high-capacity function approximation, strong priors on model space, the ability to handle irregular data, memory efficiency, and a wealth of available theory on both sides.
  arXiv  Detail & Related papers  (2022-02-04T23:32:29Z)
• Fourier Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  We formulate a new neural operator by parameterizing the integral kernel directly in Fourier space. We perform experiments on Burgers' equation, Darcy flow, and Navier-Stokes equation. It is up to three orders of magnitude faster compared to traditional PDE solvers.
  arXiv  Detail & Related papers  (2020-10-18T00:34:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.