Reservoir Computing with Generalized Readout based on Generalized Synchronization

• URL: http://arxiv.org/abs/2405.14885v1
• Date: Fri, 3 May 2024 10:03:59 GMT
• Title: Reservoir Computing with Generalized Readout based on Generalized Synchronization
• Authors: Akane Ookubo, Masanobu Inubushi,
• Abstract summary: Reservoir computing is a machine learning framework that exploits nonlinear dynamics. We propose a novel reservoir computing framework with generalized readout, including a nonlinear combination of reservoir variables. In a numerical study, we find that introducing the generalized readout leads to a significant improvement in accuracy and an unexpected enhancement in robustness.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reservoir computing is a machine learning framework that exploits nonlinear dynamics, exhibiting significant computational capabilities. One of the defining characteristics of reservoir computing is its low cost and straightforward training algorithm, i.e. only the readout, given by a linear combination of reservoir variables, is trained. Inspired by recent mathematical studies based on dynamical system theory, in particular generalized synchronization, we propose a novel reservoir computing framework with generalized readout, including a nonlinear combination of reservoir variables. The first crucial advantage of using the generalized readout is its mathematical basis for improving information processing capabilities. Secondly, it is still within a linear learning framework, which preserves the original strength of reservoir computing. In summary, the generalized readout is naturally derived from mathematical theory and allows the extraction of useful basis functions from reservoir dynamics without sacrificing simplicity. In a numerical study, we find that introducing the generalized readout leads to a significant improvement in accuracy and an unexpected enhancement in robustness for the short- and long-term prediction of Lorenz chaos, with a particular focus on how to harness low-dimensional reservoir dynamics. A novel way and its advantages for physical implementations of reservoir computing with generalized readout are briefly discussed.

Related papers

• A Unified Framework for Neural Computation and Learning Over Time [56.44910327178975]
  Hamiltonian Learning is a novel unified framework for learning with neural networks "over time" It is based on differential equations that: (i) can be integrated without the need of external software solvers; (ii) generalize the well-established notion of gradient-based learning in feed-forward and recurrent networks; (iii) open to novel perspectives.
  arXiv  Detail & Related papers  (2024-09-18T14:57:13Z)
• Stochastic Reservoir Computers [0.0]
  In reservoir computing, the number of distinct states of the entire reservoir computer can potentially scale exponentially with the size of the reservoir hardware. While shot noise is a limiting factor in the performance of reservoir computing, we show significantly improved performance compared to a reservoir computer with similar hardware in cases where the effects of noise are small.
  arXiv  Detail & Related papers  (2024-05-20T21:26:00Z)
• Universality conditions of unified classical and quantum reservoir computing [1.1510009152620668]
  Reservoir computing is a versatile paradigm in computational neuroscience and machine learning. We present a unified theoretical framework and propose a ready-made setting to secure universality.
  arXiv  Detail & Related papers  (2024-01-26T18:48:23Z)
• Randomized Polar Codes for Anytime Distributed Machine Learning [66.46612460837147]
  We present a novel distributed computing framework that is robust to slow compute nodes, and is capable of both approximate and exact computation of linear operations. We propose a sequential decoding algorithm designed to handle real valued data while maintaining low computational complexity for recovery. We demonstrate the potential applications of this framework in various contexts, such as large-scale matrix multiplication and black-box optimization.
  arXiv  Detail & Related papers  (2023-09-01T18:02:04Z)
• Optimization of a Hydrodynamic Computational Reservoir through Evolution [58.720142291102135]
  We interface with a model of a hydrodynamic system, under development by a startup, as a computational reservoir. We optimized the readout times and how inputs are mapped to the wave amplitude or frequency using an evolutionary search algorithm. Applying evolutionary methods to this reservoir system substantially improved separability on an XNOR task, in comparison to implementations with hand-selected parameters.
  arXiv  Detail & Related papers  (2023-04-20T19:15:02Z)
• Infinite-dimensional reservoir computing [9.152759278163954]
  Reservoir computing approximation and generalization bounds are proved for a new concept class of input/output systems. The results in the paper yield a fully implementable recurrent neural network-based learning algorithm with provable convergence guarantees.
  arXiv  Detail & Related papers  (2023-04-02T08:59:12Z)
• Learning unseen coexisting attractors [0.0]
  Reservoir computing is a machine learning approach that can generate a surrogate model of a dynamical system. Here, we study a challenging problem of learning a dynamical system that has both disparate time scales and multiple co-existing dynamical states (attractors) We show that the next-generation reservoir computing approach uses $sim 1.7 times$ less training data, requires $103 times$ shorter warm up' time, and has an $sim 100times$ higher accuracy in predicting the co-existing attractor characteristics.
  arXiv  Detail & Related papers  (2022-07-28T14:55:14Z)
• Deep Archimedean Copulas [98.96141706464425]
  ACNet is a novel differentiable neural network architecture that enforces structural properties. We show that ACNet is able to both approximate common Archimedean Copulas and generate new copulas which may provide better fits to data.
  arXiv  Detail & Related papers  (2020-12-05T22:58:37Z)
• Relative gradient optimization of the Jacobian term in unsupervised deep learning [9.385902422987677]
  Learning expressive probabilistic models correctly describing the data is a ubiquitous problem in machine learning. Deep density models have been widely used for this task, but their maximum likelihood based training requires estimating the log-determinant of the Jacobian. We propose a new approach for exact training of such neural networks.
  arXiv  Detail & Related papers  (2020-06-26T16:41:08Z)
• Predictive Coding Approximates Backprop along Arbitrary Computation Graphs [68.8204255655161]
  We develop a strategy to translate core machine learning architectures into their predictive coding equivalents. Our models perform equivalently to backprop on challenging machine learning benchmarks. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry.
  arXiv  Detail & Related papers  (2020-06-07T15:35:47Z)
• On dissipative symplectic integration with applications to gradient-based optimization [77.34726150561087]
  We propose a geometric framework in which discretizations can be realized systematically. We show that a generalization of symplectic to nonconservative and in particular dissipative Hamiltonian systems is able to preserve rates of convergence up to a controlled error.
  arXiv  Detail & Related papers  (2020-04-15T00:36:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.