Catch-22s of reservoir computing

• URL: http://arxiv.org/abs/2210.10211v3
• Date: Mon, 25 Sep 2023 22:39:46 GMT
• Title: Catch-22s of reservoir computing
• Authors: Yuanzhao Zhang and Sean P. Cornelius
• Abstract summary: Reservoir Computing is a simple and efficient framework for forecasting the behavior of nonlinear dynamical systems from data. We focus on the important problem of basin prediction -- determining which attractor a system will converge to from its initial conditions. By incorporating the exact nonlinearities in the original equations, we show that NGRC can accurately reconstruct intricate and high-dimensional basins of attraction.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reservoir Computing (RC) is a simple and efficient model-free framework for forecasting the behavior of nonlinear dynamical systems from data. Here, we show that there exist commonly-studied systems for which leading RC frameworks struggle to learn the dynamics unless key information about the underlying system is already known. We focus on the important problem of basin prediction -- determining which attractor a system will converge to from its initial conditions. First, we show that the predictions of standard RC models (echo state networks) depend critically on warm-up time, requiring a warm-up trajectory containing almost the entire transient in order to identify the correct attractor. Accordingly, we turn to Next-Generation Reservoir Computing (NGRC), an attractive variant of RC that requires negligible warm-up time. By incorporating the exact nonlinearities in the original equations, we show that NGRC can accurately reconstruct intricate and high-dimensional basins of attraction, even with sparse training data (e.g., a single transient trajectory). Yet, a tiny uncertainty in the exact nonlinearity can render prediction accuracy no better than chance. Our results highlight the challenges faced by data-driven methods in learning the dynamics of multistable systems and suggest potential avenues to make these approaches more robust.

Related papers

• How more data can hurt: Instability and regularization in next-generation reservoir computing [0.0]
  We show that a more extreme version of the phenomenon occurs in data-driven models of dynamical systems. We find that, despite learning a better representation of the flow map with more training data, NGRC can adopt an ill-conditioned integrator'' and lose stability.
  arXiv  Detail & Related papers  (2024-07-11T16:22:13Z)
• Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems [0.0]
  Reservoir computers (RCs) are powerful machine learning architectures for time series prediction. Next generation reservoir computers (NGRCs) have been introduced, offering distinct advantages over RCs. Here, we introduce a hybrid RC-NGRC approach for time series forecasting of dynamical systems.
  arXiv  Detail & Related papers  (2024-03-04T17:35:17Z)
• Kalman Filter for Online Classification of Non-Stationary Data [101.26838049872651]
  In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. We introduce a probabilistic Bayesian online learning model by using a neural representation and a state space model over the linear predictor weights. In experiments in multi-class classification we demonstrate the predictive ability of the model and its flexibility to capture non-stationarity.
  arXiv  Detail & Related papers  (2023-06-14T11:41:42Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics [0.0]
  Reservoir computer (RC) is a type of recurrent neural network that has demonstrated success in prediction architecture of intrinsicly chaotic dynamical systems. We explore the architecture and design choices for a "best in class" RC for a number of characteristic dynamical systems. We show the application of these choices in scaling up to larger models using localization.
  arXiv  Detail & Related papers  (2022-01-21T22:31:12Z)
• A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data [0.0]
  We present a data-driven approach to automatically extract an appropriate model structure from experimentally observed processes. We demonstrate our approach on a simple benchmark problem, where it beats the state of the art meta-learning techniques.
  arXiv  Detail & Related papers  (2021-10-07T18:23:14Z)
• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics [84.18625250574853]
  We present KalmanNet, a real-time state estimator that learns from data to carry out Kalman filtering under non-linear dynamics. We numerically demonstrate that KalmanNet overcomes nonlinearities and model mismatch, outperforming classic filtering methods.
  arXiv  Detail & Related papers  (2021-07-21T12:26:46Z)
• Incorporating Causal Graphical Prior Knowledge into Predictive Modeling via Simple Data Augmentation [92.96204497841032]
  Causal graphs (CGs) are compact representations of the knowledge of the data generating processes behind the data distributions. We propose a model-agnostic data augmentation method that allows us to exploit the prior knowledge of the conditional independence (CI) relations. We experimentally show that the proposed method is effective in improving the prediction accuracy, especially in the small-data regime.
  arXiv  Detail & Related papers  (2021-02-27T06:13:59Z)
• Reservoir Computing as a Tool for Climate Predictability Studies [0.0]
  We show that Reservoir Computing provides an alternative nonlinear approach that improves on the predictive skill of the Linear-Inverse-Modeling approach. The improved predictive skill of the RC approach over a wide range of conditions suggests that this machine-learning technique may have a use in climate predictability studies.
  arXiv  Detail & Related papers  (2021-02-24T22:22:59Z)
• Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components [52.77024349608834]
  We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is noisy partial measurements. We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model.
  arXiv  Detail & Related papers  (2021-02-15T19:56:48Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.