Reservoir Computers Modal Decomposition and Optimization

• URL: http://arxiv.org/abs/2101.07219v1
• Date: Wed, 13 Jan 2021 23:30:21 GMT
• Title: Reservoir Computers Modal Decomposition and Optimization
• Authors: Chad Nathe, Enrico Del Frate, Thomas Carroll, Louis Pecora, Afroza Shirin, Francesco Sorrentino
• Abstract summary: We obtain a decomposition of the reservoir dynamics into modes, which can be computed independently of one another. We then take a parametric approach in which the eigenvalues are parameters that can be appropriately designed and optimized. We show that manipulations of the individual modes, either in terms of the eigenvalues or the time shifts, can lead to dramatic reductions in the training error.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The topology of a network associated with a reservoir computer is often taken so that the connectivity and the weights are chosen randomly. Optimization is hardly considered as the parameter space is typically too large. Here we investigate this problem for a class of reservoir computers for which we obtain a decomposition of the reservoir dynamics into modes, which can be computed independently of one another. Each mode depends on an eigenvalue of the network adjacency matrix. We then take a parametric approach in which the eigenvalues are parameters that can be appropriately designed and optimized. In addition, we introduce the application of a time shift to each individual mode. We show that manipulations of the individual modes, either in terms of the eigenvalues or the time shifts, can lead to dramatic reductions in the training error.

Related papers

• Simple Cycle Reservoirs are Universal [0.358439716487063]
  Reservoir models form a subclass of recurrent neural networks with fixed non-trainable input and dynamic coupling weights. We show that they are capable of universal approximation of any unrestricted linear reservoir system.
  arXiv  Detail & Related papers  (2023-08-21T15:35:59Z)
• Sparse Quadratic Optimisation over the Stiefel Manifold with Application to Permutation Synchronisation [71.27989298860481]
  We address the non- optimisation problem of finding a matrix on the Stiefel manifold that maximises a quadratic objective function. We propose a simple yet effective sparsity-promoting algorithm for finding the dominant eigenspace matrix.
  arXiv  Detail & Related papers  (2021-09-30T19:17:35Z)
• KaFiStO: A Kalman Filtering Framework for Stochastic Optimization [27.64040983559736]
  We show that when training neural networks the loss function changes over (iteration) time due to the randomized selection of a subset of the samples. This randomization turns the optimization problem into an optimum one. We propose to consider the loss as a noisy observation with respect to some reference.
  arXiv  Detail & Related papers  (2021-07-07T16:13:57Z)
• Robust Value Iteration for Continuous Control Tasks [99.00362538261972]
  When transferring a control policy from simulation to a physical system, the policy needs to be robust to variations in the dynamics to perform well. We present Robust Fitted Value Iteration, which uses dynamic programming to compute the optimal value function on the compact state domain. We show that robust value is more robust compared to deep reinforcement learning algorithm and the non-robust version of the algorithm.
  arXiv  Detail & Related papers  (2021-05-25T19:48:35Z)
• Solving weakly supervised regression problem using low-rank manifold regularization [77.34726150561087]
  We solve a weakly supervised regression problem. Under "weakly" we understand that for some training points the labels are known, for some unknown, and for others uncertain due to the presence of random noise or other reasons such as lack of resources. In the numerical section, we applied the suggested method to artificial and real datasets using Monte-Carlo modeling.
  arXiv  Detail & Related papers  (2021-04-13T23:21:01Z)
• Self Normalizing Flows [65.73510214694987]
  We propose a flexible framework for training normalizing flows by replacing expensive terms in the gradient by learned approximate inverses at each layer. This reduces the computational complexity of each layer's exact update from $mathcalO(D3)$ to $mathcalO(D2)$. We show experimentally that such models are remarkably stable and optimize to similar data likelihood values as their exact gradient counterparts.
  arXiv  Detail & Related papers  (2020-11-14T09:51:51Z)
• Training Invertible Linear Layers through Rank-One Perturbations [0.0]
  This work presents a novel approach for training invertible linear layers. In lieu of directly optimizing the network parameters, we train rank-one perturbations and add them to the actual weight matrices infrequently. We show how such invertible blocks improve the mixing and thus normalizing the mode separation of the resulting flows.
  arXiv  Detail & Related papers  (2020-10-14T12:43:47Z)
• Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems [107.3868459697569]
  We introduce an eigendecomposition-free approach to training a deep network. We show that our approach is much more robust than explicit differentiation of the eigendecomposition. Our method has better convergence properties and yields state-of-the-art results.
  arXiv  Detail & Related papers  (2020-04-15T04:29:34Z)
• Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation [72.40827239394565]
  We propose to compute features only at sparsely sampled locations. We then densely reconstruct the feature map with an efficient procedure. The presented network is experimentally shown to save substantial computation while maintaining accuracy over a variety of computer vision tasks.
  arXiv  Detail & Related papers  (2020-03-19T15:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.