POLA: Online Time Series Prediction by Adaptive Learning Rates

• URL: http://arxiv.org/abs/2102.08907v1
• Date: Wed, 17 Feb 2021 17:56:12 GMT
• Title: POLA: Online Time Series Prediction by Adaptive Learning Rates
• Authors: Wenyu Zhang
• Abstract summary: We propose POLA to automatically regulate the learning rate of recurrent neural network models to adapt to changing time series patterns across time. POLA demonstrates overall comparable or better predictive performance over other online prediction methods.
• Score: 4.105553918089042
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online prediction for streaming time series data has practical use for many real-world applications where downstream decisions depend on accurate forecasts for the future. Deployment in dynamic environments requires models to adapt quickly to changing data distributions without overfitting. We propose POLA (Predicting Online by Learning rate Adaptation) to automatically regulate the learning rate of recurrent neural network models to adapt to changing time series patterns across time. POLA meta-learns the learning rate of the stochastic gradient descent (SGD) algorithm by assimilating the prequential or interleaved-test-then-train evaluation scheme for online prediction. We evaluate POLA on two real-world datasets across three commonly-used recurrent neural network models. POLA demonstrates overall comparable or better predictive performance over other online prediction methods.

Related papers

• ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting [11.261457967759688]
  ODEStream is a buffer-free continual learning framework that incorporates a temporal isolation layer that integrates temporal dependencies within the data. Our approach focuses on learning how the dynamics and distribution of historical data change with time, facilitating the direct processing of streaming sequences. Evaluations on benchmark real-world datasets demonstrate that ODEStream outperforms the state-of-the-art online learning and streaming analysis baselines.
  arXiv  Detail & Related papers  (2024-11-11T22:36:33Z)
• Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models [88.47454470043552]
  We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. Online adaptation turns parameters into temporally changing states and provides a form of context-length extension with memory in weights.
  arXiv  Detail & Related papers  (2024-03-03T14:03:48Z)
• Online Test-Time Adaptation of Spatial-Temporal Traffic Flow Forecasting [13.770733370640565]
  This paper conducts the first study of the online test-time adaptation techniques for spatial-temporal traffic flow forecasting problems. We propose an Adaptive Double Correction by Series Decomposition (ADCSD) method, which first decomposes the output of the trained model into seasonal and trend-cyclical parts. In the proposed ADCSD method, instead of fine-tuning the whole trained model during the testing phase, a lite network is attached after the trained model, and only the lite network is fine-tuned in the testing process each time a data entry is observed.
  arXiv  Detail & Related papers  (2024-01-08T12:04:39Z)
• An Adaptive Framework for Generalizing Network Traffic Prediction towards Uncertain Environments [51.99765487172328]
  We have developed a new framework using time-series analysis for dynamically assigning mobile network traffic prediction models. Our framework employs learned behaviors, outperforming any single model with over a 50% improvement relative to current studies.
  arXiv  Detail & Related papers  (2023-11-30T18:58:38Z)
• PILOT: A Pre-Trained Model-Based Continual Learning Toolbox [71.63186089279218]
  This paper introduces a pre-trained model-based continual learning toolbox known as PILOT. On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt. On the other hand, PILOT fits typical class-incremental learning algorithms within the context of pre-trained models to evaluate their effectiveness.
  arXiv  Detail & Related papers  (2023-09-13T17:55:11Z)
• Kalman Filter for Online Classification of Non-Stationary Data [101.26838049872651]
  In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. We introduce a probabilistic Bayesian online learning model by using a neural representation and a state space model over the linear predictor weights. In experiments in multi-class classification we demonstrate the predictive ability of the model and its flexibility to capture non-stationarity.
  arXiv  Detail & Related papers  (2023-06-14T11:41:42Z)
• Brain-Inspired Spiking Neural Network for Online Unsupervised Time Series Prediction [13.521272923545409]
  We present a novel Continuous Learning-based Unsupervised Recurrent Spiking Neural Network Model (CLURSNN) CLURSNN makes online predictions by reconstructing the underlying dynamical system using Random Delay Embedding. We show that the proposed online time series prediction methodology outperforms state-of-the-art DNN models when predicting an evolving Lorenz63 dynamical system.
  arXiv  Detail & Related papers  (2023-04-10T16:18:37Z)
• Online Evolutionary Neural Architecture Search for Multivariate Non-Stationary Time Series Forecasting [72.89994745876086]
  This work presents the Online Neuro-Evolution-based Neural Architecture Search (ONE-NAS) algorithm. ONE-NAS is a novel neural architecture search method capable of automatically designing and dynamically training recurrent neural networks (RNNs) for online forecasting tasks. Results demonstrate that ONE-NAS outperforms traditional statistical time series forecasting methods.
  arXiv  Detail & Related papers  (2023-02-20T22:25:47Z)
• Probabilistic AutoRegressive Neural Networks for Accurate Long-range Forecasting [6.295157260756792]
  We introduce the Probabilistic AutoRegressive Neural Networks (PARNN) PARNN is capable of handling complex time series data exhibiting non-stationarity, nonlinearity, non-seasonality, long-range dependence, and chaotic patterns. We evaluate the performance of PARNN against standard statistical, machine learning, and deep learning models, including Transformers, NBeats, and DeepAR.
  arXiv  Detail & Related papers  (2022-04-01T17:57:36Z)
• Online Learning with Radial Basis Function Networks [0.0]
  We consider the sequential and continual learning sub-genres of online learning. We find that the online learning techniques outperform the offline learning ones.
  arXiv  Detail & Related papers  (2021-03-15T14:39:40Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.