Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics

• URL: http://arxiv.org/abs/2103.03174v1
• Date: Tue, 9 Feb 2021 22:24:00 GMT
• Title: Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics
• Authors: Alberto Racca and Luca Magri
• Abstract summary: An approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Existing studies showed that small changes in the hyper parameters may markedly affect the network's performance. This paper aims to assess and improve the robustness of Echo State Networks for the time-accurate prediction of chaotic solutions.
• Score: 6.345523830122166
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: An approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Echo State Networks (ESNs), which are a class of Reservoir Computing, can accurately predict the chaotic dynamics well beyond the predictability time. Existing studies, however, also showed that small changes in the hyperparameters may markedly affect the network's performance. The aim of this paper is to assess and improve the robustness of Echo State Networks for the time-accurate prediction of chaotic solutions. The goal is three-fold. First, we investigate the robustness of routinely used validation strategies. Second, we propose the Recycle Validation, and the chaotic versions of existing validation strategies, to specifically tackle the forecasting of chaotic systems. Third, we compare Bayesian optimization with the traditional Grid Search for optimal hyperparameter selection. Numerical tests are performed on two prototypical nonlinear systems that have both chaotic and quasiperiodic solutions. Both model-free and model-informed Echo State Networks are analysed. By comparing the network's robustness in learning chaotic versus quasiperiodic solutions, we highlight fundamental challenges in learning chaotic solutions. The proposed validation strategies, which are based on the dynamical systems properties of chaotic time series, are shown to outperform the state-of-the-art validation strategies. Because the strategies are principled-they are based on chaos theory such as the Lyapunov time-they can be applied to other Recurrent Neural Networks architectures with little modification. This work opens up new possibilities for the robust design and application of Echo State Networks, and Recurrent Neural Networks, to the time-accurate prediction of chaotic systems.

Related papers

• Certified Neural Approximations of Nonlinear Dynamics [52.79163248326912]
  In safety-critical contexts, the use of neural approximations requires formal bounds on their closeness to the underlying system.<n>We propose a novel, adaptive, and parallelizable verification method based on certified first-order models.
  arXiv  Detail & Related papers  (2025-05-21T13:22:20Z)
• Topology-Aware Conformal Prediction for Stream Networks [54.505880918607296]
  We propose Spatio-Temporal Adaptive Conformal Inference (textttCISTA), a novel framework that integrates network topology and temporal dynamics into the conformal prediction framework. Our results show that textttCISTA effectively balances prediction efficiency and coverage, outperforming existing conformal prediction methods for stream networks.
  arXiv  Detail & Related papers  (2025-03-06T21:21:15Z)
• Feedback Favors the Generalization of Neural ODEs [24.342023073252395]
  We present feedback neural networks, showing that a feedback loop can flexibly correct the learned latent dynamics of neural ordinary differential equations (neural ODEs) The feedback neural network is a novel two-DOF neural network, which possesses robust performance in unseen scenarios with no loss of accuracy performance on previous tasks.
  arXiv  Detail & Related papers  (2024-10-14T08:09:45Z)
• Improved deep learning of chaotic dynamical systems with multistep penalty losses [0.0]
  Predicting the long-term behavior of chaotic systems remains a formidable challenge. This paper introduces a novel framework that addresses these challenges by leveraging the recently proposed multi-step penalty operators.
  arXiv  Detail & Related papers  (2024-10-08T00:13:57Z)
• Comprehensive Analysis of Network Robustness Evaluation Based on Convolutional Neural Networks with Spatial Pyramid Pooling [4.366824280429597]
  Connectivity robustness, a crucial aspect for understanding, optimizing, and repairing complex networks, has traditionally been evaluated through simulations. We address these challenges by designing a convolutional neural networks (CNN) model with spatial pyramid pooling networks (SPP-net) We show that the proposed CNN model consistently achieves accurate evaluations of both attack curves and robustness values across all removal scenarios.
  arXiv  Detail & Related papers  (2023-08-10T09:54:22Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• A Learning Convolutional Neural Network Approach for Network Robustness Prediction [13.742495880357493]
  Network robustness is critical for various societal and industrial networks again malicious attacks. In this paper, an improved method for network robustness prediction is developed based on learning feature representation using convolutional neural network (LFR-CNN) In this scheme, higher-dimensional network data are compressed to lower-dimensional representations, and then passed to a CNN to perform robustness prediction.
  arXiv  Detail & Related papers  (2022-03-20T13:45:55Z)
• Learning Fast and Slow for Online Time Series Forecasting [76.50127663309604]
  Fast and Slow learning Networks (FSNet) is a holistic framework for online time-series forecasting. FSNet balances fast adaptation to recent changes and retrieving similar old knowledge. Our code will be made publicly available.
  arXiv  Detail & Related papers  (2022-02-23T18:23:07Z)
• Revisiting the double-well problem by deep learning with a hybrid network [7.308730248177914]
  We propose a novel hybrid network which integrates two different kinds of neural networks: LSTM and ResNet. Such a hybrid network can be applied for solving cooperative dynamics in a system with fast spatial or temporal modulations.
  arXiv  Detail & Related papers  (2021-04-25T07:51:43Z)
• Adversarial Refinement Network for Human Motion Prediction [61.50462663314644]
  Two popular methods, recurrent neural networks and feed-forward deep networks, are able to predict rough motion trend. We propose an Adversarial Refinement Network (ARNet) following a simple yet effective coarse-to-fine mechanism with novel adversarial error augmentation.
  arXiv  Detail & Related papers  (2020-11-23T05:42:20Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)
• DANR: Discrepancy-aware Network Regularization [15.239252118069762]
  Network regularization is an effective tool for learning coherent models over networks. We propose a novel approach that is robust to inadequate regularizations and effectively captures model evolution and structural changes over-temporal networks. We develop a scalable and scalable algorithm based on the alternating method of multipliers (ADMM) to solve the proposed problem with guaranteed convergence to global optimum solutions.
  arXiv  Detail & Related papers  (2020-05-31T02:01:19Z)
• Dynamic Inference: A New Approach Toward Efficient Video Action Recognition [69.9658249941149]
  Action recognition in videos has achieved great success recently, but it remains a challenging task due to the massive computational cost. We propose a general dynamic inference idea to improve inference efficiency by leveraging the variation in the distinguishability of different videos.
  arXiv  Detail & Related papers  (2020-02-09T11:09:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.