Attention-Enhanced Reservoir Computing

• URL: http://arxiv.org/abs/2312.16503v2
• Date: Thu, 2 May 2024 05:27:40 GMT
• Title: Attention-Enhanced Reservoir Computing
• Abstract summary: Photonic reservoir computing has been successfully utilized in time-series prediction. We introduce an attention mechanism to the reservoir computing model in the output stage. Our results show that a photonic reservoir computer enhanced with the attention mechanism exhibits improved prediction capabilities for smaller reservoirs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Photonic reservoir computing has been successfully utilized in time-series prediction as the need for hardware implementations has increased. Prediction of chaotic time series remains a significant challenge, an area where the conventional reservoir computing framework encounters limitations of prediction accuracy. We introduce an attention mechanism to the reservoir computing model in the output stage. This attention layer is designed to prioritize distinct features and temporal sequences, thereby substantially enhancing the prediction accuracy. Our results show that a photonic reservoir computer enhanced with the attention mechanism exhibits improved prediction capabilities for smaller reservoirs. These advancements highlight the transformative possibilities of reservoir computing for practical applications where accurate prediction of chaotic time series is crucial.

Related papers

• Anticipating tipping in spatiotemporal systems with machine learning [0.0]
  We use a machine-learning framework of parameter-adaptable reservoir computing to accurately anticipate tipping in complex dynamical systems.<n>We demonstrate tipping can be identified within a prediction window across a variety of timetemporal dynamical systems, as well as in CMIP5Coupled Model Intercomparison Project 5 climate.
  arXiv  Detail & Related papers  (2026-04-07T20:51:58Z)
• Centrality-Based Pruning for Efficient Echo State Networks [0.0]
  We propose a graph-based pruning approach that interprets the reservoir as a weighted directed graph and removes structurally less important nodes using centrality measures.<n> Experiments on Mackey-Glass time-series prediction and electric load forecasting demonstrate that the proposed method can significantly reduce reservoir size while maintaining centrality.
  arXiv  Detail & Related papers  (2026-03-21T06:55:50Z)
• Accuracy Law for the Future of Deep Time Series Forecasting [65.46625911002202]
  Time series forecasting inherently faces a non-zero error lower bound due to its partially observable and uncertain nature.<n>This paper focuses on a fundamental question: how to estimate the performance upper bound of deep time series forecasting.<n>Based on rigorous statistical tests of over 2,800 newly trained deep forecasters, we discover a significant exponential relationship between the minimum forecasting error of deep models and the complexity of window-wise series patterns.
  arXiv  Detail & Related papers  (2025-10-03T05:18:47Z)
• Oscillations enhance time-series prediction in reservoir computing with feedback [3.3686252536891454]
  Reservoir computing is a machine learning framework used for modeling the brain. It is difficult to accurately reproduce the long-term target time series because the reservoir system becomes unstable. This study proposes oscillation-driven reservoir computing (ODRC) with feedback.
  arXiv  Detail & Related papers  (2024-06-05T02:30:29Z)
• A novel Reservoir Architecture for Periodic Time Series Prediction [4.7368661961661775]
  This paper introduces a novel approach to predicting periodic time series using reservoir computing. The model is tailored to deliver precise forecasts of rhythms, a crucial aspect for tasks such as generating musical rhythm. Our network accurately predicts rhythmic signals within the human frequency perception range.
  arXiv  Detail & Related papers  (2024-05-16T13:55:53Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Predicting Temporal Aspects of Movement for Predictive Replication in Fog Environments [0.0]
  Blind or reactive data falls short in harnessing the potential of fog computing. We propose a novel model using Holt-Winter's Exponential Smoothing for temporal prediction. In a fog network simulation with real user trajectories our model achieves a 15% reduction in excess data with a marginal 1% decrease in data availability.
  arXiv  Detail & Related papers  (2023-06-01T11:45:13Z)
• Prediction-Powered Inference [68.97619568620709]
  Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients. Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning.
  arXiv  Detail & Related papers  (2023-01-23T18:59:28Z)
• What Should I Know? Using Meta-gradient Descent for Predictive Feature Discovery in a Single Stream of Experience [63.75363908696257]
  computational reinforcement learning seeks to construct an agent's perception of the world through predictions of future sensations. An open challenge in this line of work is determining from the infinitely many predictions that the agent could possibly make which predictions might best support decision-making. We introduce a meta-gradient descent process by which an agent learns what predictions to make, 2) the estimates for its chosen predictions, and 3) how to use those estimates to generate policies that maximize future reward.
  arXiv  Detail & Related papers  (2022-06-13T21:31:06Z)
• Extreme precipitation forecasting using attention augmented convolutions [0.913755431537592]
  We propose a self-attention augmented convolution mechanism for extreme precipitation forecasting. Our experimental results show that the framework outperforms classical convolutional models by 12%. The proposed method increases machine learning as a tool for gaining insights into the physical causes of changing extremes.
  arXiv  Detail & Related papers  (2022-01-31T18:16:03Z)
• Multi-head Temporal Attention-Augmented Bilinear Network for Financial time series prediction [77.57991021445959]
  We propose a neural layer based on the ideas of temporal attention and multi-head attention to extend the capability of the underlying neural network. The effectiveness of our approach is validated using large-scale limit-order book market data.
  arXiv  Detail & Related papers  (2022-01-14T14:02:19Z)
• Learning Output Embeddings in Structured Prediction [73.99064151691597]
  A powerful and flexible approach to structured prediction consists in embedding the structured objects to be predicted into a feature space of possibly infinite dimension. A prediction in the original space is computed by solving a pre-image problem. In this work, we propose to jointly learn a finite approximation of the output embedding and the regression function into the new feature space.
  arXiv  Detail & Related papers  (2020-07-29T09:32:53Z)
• Multioutput Gaussian Processes with Functional Data: A Study on Coastal Flood Hazard Assessment [0.0]
  We introduce a surrogate model that accounts for time-varying inputs and provides information on spatially varying inland flooding. In several experiments, we demonstrate the versatility of the model for both learning maps and inferring unobserved maps. We conclude that our framework is a promising approach for forecast and early-warning systems.
  arXiv  Detail & Related papers  (2020-07-28T08:15:17Z)
• AutoCP: Automated Pipelines for Accurate Prediction Intervals [84.16181066107984]
  This paper proposes an AutoML framework called Automatic Machine Learning for Conformal Prediction (AutoCP) Unlike the familiar AutoML frameworks that attempt to select the best prediction model, AutoCP constructs prediction intervals that achieve the user-specified target coverage rate. We tested AutoCP on a variety of datasets and found that it significantly outperforms benchmark algorithms.
  arXiv  Detail & Related papers  (2020-06-24T23:13:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.