RcTorch: a PyTorch Reservoir Computing Package with Automated Hyper-Parameter Optimization

• URL: http://arxiv.org/abs/2207.05870v1
• Date: Tue, 12 Jul 2022 22:24:36 GMT
• Title: RcTorch: a PyTorch Reservoir Computing Package with Automated Hyper-Parameter Optimization
• Abstract summary: Reservoir computers (RCs) are among the fastest to train of all neural networks. RC adoption has lagged because of the model's sensitivity to its hyper- parameters. We introduce RcTorch, a PyTorch based RC neural network package with automated HP tuning.
• Score: 1.2891210250935146
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reservoir computers (RCs) are among the fastest to train of all neural networks, especially when they are compared to other recurrent neural networks. RC has this advantage while still handling sequential data exceptionally well. However, RC adoption has lagged other neural network models because of the model's sensitivity to its hyper-parameters (HPs). A modern unified software package that automatically tunes these parameters is missing from the literature. Manually tuning these numbers is very difficult, and the cost of traditional grid search methods grows exponentially with the number of HPs considered, discouraging the use of the RC and limiting the complexity of the RC models which can be devised. We address these problems by introducing RcTorch, a PyTorch based RC neural network package with automated HP tuning. Herein, we demonstrate the utility of RcTorch by using it to predict the complex dynamics of a driven pendulum being acted upon by varying forces. This work includes coding examples. Example Python Jupyter notebooks can be found on our GitHub repository https://github.com/blindedjoy/RcTorch and documentation can be found at https://rctorch.readthedocs.io/.

Related papers

• OpenReservoirComputing: GPU-Accelerated Reservoir Computing in JAX [3.6137862091320936]
  OpenReservoirComputing (ORC) is a Python library for reservoir computing (RC) written in JAX.<n>JAX is a Python library for high-performance numerical computing.<n>ORC offers GPU acceleration, JIT compilation, and automatic vectorization.
  arXiv  Detail & Related papers  (2026-03-16T03:59:46Z)
• Recurrent Diffusion for Large-Scale Parameter Generation [52.98888368644455]
  We introduce Recurrent Diffusion for Large Scale Generation (RPG), a novel framework that generates full neural network parameters up to hundreds of millions on a single GPU. RPG serves as a critical advance in AI generating AI, potentially enabling efficient weight generation at scales previously deemed infeasible.
  arXiv  Detail & Related papers  (2025-01-20T16:46:26Z)
• Exploring Green AI for Audio Deepfake Detection [21.17957700009653]
  State-of-the-art audio deepfake detectors leveraging deep neural networks exhibit impressive recognition performance. Deep NLP models produce around 626k lbs of COtextsubscript2 which is equivalent to five times of average US car emission at its lifetime. This study presents a novel framework for audio deepfake detection that can be seamlessly trained using standard CPU resources.
  arXiv  Detail & Related papers  (2024-03-21T10:54:21Z)
• HyperPPO: A scalable method for finding small policies for robotic control [14.789594427174052]
  HyperPPO is an on-policy reinforcement learning algorithm that estimates the weights of multiple neural networks simultaneously. Our method estimates weights for networks that are much smaller than those in common-use networks yet encode highly performant policies. We demonstrate that the neural policies estimated by HyperPPO are capable of decentralized control of a Crazyflie2.1 quadrotor.
  arXiv  Detail & Related papers  (2023-09-28T17:58:26Z)
• RoseNNa: A performant, portable library for neural network inference with application to computational fluid dynamics [0.0]
  We present the roseNNa library, which bridges the gap between neural network inference and CFD. RoseNNa is a non-invasive, lightweight (1000 lines) tool for neural network inference.
  arXiv  Detail & Related papers  (2023-07-30T21:11:55Z)
• Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch [1.3816155029354618]
  We propose Rockmate to control the memory requirements when training PyTorch DNN models. Rockmate is an automatic tool that starts from the model code and generates an equivalent model, using a predefined amount of memory for activations. We show through experiments on many models that Rockmate is as fast as Rotor and as efficient as Checkmate, and that it allows in many cases to obtain a significantly lower memory consumption for activations.
  arXiv  Detail & Related papers  (2023-07-03T11:42:14Z)
• Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs [64.26714148634228]
  congestion control (CC) algorithms become extremely difficult to design. It is currently not possible to deploy AI models on network devices due to their limited computational capabilities. We build a computationally-light solution based on a recent reinforcement learning CC algorithm.
  arXiv  Detail & Related papers  (2022-07-05T20:42:24Z)
• Variable Bitrate Neural Fields [75.24672452527795]
  We present a dictionary method for compressing feature grids, reducing their memory consumption by up to 100x. We formulate the dictionary optimization as a vector-quantized auto-decoder problem which lets us learn end-to-end discrete neural representations in a space where no direct supervision is available.
  arXiv  Detail & Related papers  (2022-06-15T17:58:34Z)
• Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$ [118.04625413322827]
  $texttt5x$ and $texttseqio$ are open source software libraries for building and training language models. These libraries have been used to train models with hundreds of billions of parameters on datasets with multiple terabytes of training data.
  arXiv  Detail & Related papers  (2022-03-31T17:12:13Z)
• Hyperparameter Tuning is All You Need for LISTA [92.7008234085887]
  Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces the concept of unrolling an iterative algorithm and training it like a neural network. We show that adding momentum to intermediate variables in the LISTA network achieves a better convergence rate. We call this new ultra-light weight network HyperLISTA.
  arXiv  Detail & Related papers  (2021-10-29T16:35:38Z)
• Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data with Spatial Information [52.635997570873194]
  This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information. The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
  arXiv  Detail & Related papers  (2021-01-12T20:08:18Z)
• Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network [2.277447144331876]
  Unconstrained handwritten text recognition is a major step in most document analysis tasks. One alternative solution to using LSTM cells is to compensate the long time memory loss with an heavy use of convolutional layers. We present a Gated Fully Convolutional Network architecture that is a recurrence-free alternative to the well-known CNN+LSTM architectures.
  arXiv  Detail & Related papers  (2020-12-09T10:30:13Z)
• On the distance between two neural networks and the stability of learning [59.62047284234815]
  This paper relates parameter distance to gradient breakdown for a broad class of nonlinear compositional functions. The analysis leads to a new distance function called deep relative trust and a descent lemma for neural networks.
  arXiv  Detail & Related papers  (2020-02-09T19:18:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.