Recurrent Neural Networks for Partially Observed Dynamical Systems

• URL: http://arxiv.org/abs/2109.11629v1
• Date: Tue, 21 Sep 2021 20:15:20 GMT
• Title: Recurrent Neural Networks for Partially Observed Dynamical Systems
• Authors: Uttam Bhat and Stephan B. Munch
• Abstract summary: Delay embedding allows us to account for unobserved state variables. We provide an approach to delay embedding that permits explicit approximation of error. We also provide the dependence of the first order approximation error on the system size.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Complex nonlinear dynamics are ubiquitous in many fields. Moreover, we rarely have access to all of the relevant state variables governing the dynamics. Delay embedding allows us, in principle, to account for unobserved state variables. Here we provide an algebraic approach to delay embedding that permits explicit approximation of error. We also provide the asymptotic dependence of the first order approximation error on the system size. More importantly, this formulation of delay embedding can be directly implemented using a Recurrent Neural Network (RNN). This observation expands the interpretability of both delay embedding and RNN and facilitates principled incorporation of structure and other constraints into these approaches.

Related papers

• Correctness Verification of Neural Networks Approximating Differential Equations [0.0]
  Neural Networks (NNs) approximate the solution of Partial Differential Equations (PDEs) NNs can become integral parts of simulation software tools which can accelerate the simulation of complex dynamic systems more than 100 times. This work addresses the verification of these functions by defining the NN derivative as a finite difference approximation. For the first time, we tackle the problem of bounding an NN function without a priori knowledge of the output domain.
  arXiv  Detail & Related papers  (2024-02-12T12:55:35Z)
• Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets [57.06026574261203]
  We provide theoretical evidence for learning low-dimensional state spaces, which can also model long-term memory. Experiments corroborate our theory, demonstrating extrapolation via learning low-dimensional state spaces with both linear and non-linear RNNs.
  arXiv  Detail & Related papers  (2022-10-25T14:45:15Z)
• Characterizing possible failure modes in physics-informed neural networks [55.83255669840384]
  Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
  arXiv  Detail & Related papers  (2021-09-02T16:06:45Z)
• Metric Entropy Limits on Recurrent Neural Network Learning of Linear Dynamical Systems [0.0]
  We show that RNNs can optimally learn - or identify in system-theory parlance - stable LTI systems. For LTI systems whose input-output relation is characterized through a difference equation, this means that RNNs can learn the difference equation from input-output traces in a metric-entropy optimal manner.
  arXiv  Detail & Related papers  (2021-05-06T10:12:30Z)
• Neural ODE Processes [64.10282200111983]
  We introduce Neural ODE Processes (NDPs), a new class of processes determined by a distribution over Neural ODEs. We show that our model can successfully capture the dynamics of low-dimensional systems from just a few data-points.
  arXiv  Detail & Related papers  (2021-03-23T09:32:06Z)
• UnICORNN: A recurrent model for learning very long time dependencies [0.0]
  We propose a novel RNN architecture based on a structure preserving discretization of a Hamiltonian system of second-order ordinary differential equations. The resulting RNN is fast, invertible (in time), memory efficient and we derive rigorous bounds on the hidden state gradients to prove the mitigation of the exploding and vanishing gradient problem.
  arXiv  Detail & Related papers  (2021-03-09T15:19:59Z)
• Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies [15.2292571922932]
  We propose a novel architecture for recurrent neural networks. Our proposed RNN is based on a time-discretization of a system of second-order ordinary differential equations. Experiments show that the proposed RNN is comparable in performance to the state of the art on a variety of benchmarks.
  arXiv  Detail & Related papers  (2020-10-02T12:35:04Z)
• Theory of gating in recurrent neural networks [5.672132510411465]
  Recurrent neural networks (RNNs) are powerful dynamical models, widely used in machine learning (ML) and neuroscience. Here, we show that gating offers flexible control of two salient features of the collective dynamics. The gate controlling timescales leads to a novel, marginally stable state, where the network functions as a flexible integrator.
  arXiv  Detail & Related papers  (2020-07-29T13:20:58Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• Liquid Time-constant Networks [117.57116214802504]
  We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations.
  arXiv  Detail & Related papers  (2020-06-08T09:53:35Z)
• Time Dependence in Non-Autonomous Neural ODEs [74.78386661760662]
  We propose a novel family of Neural ODEs with time-varying weights. We outperform previous Neural ODE variants in both speed and representational capacity.
  arXiv  Detail & Related papers  (2020-05-05T01:41:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.