Implicit Reasoning in Deep Time Series Forecasting

• URL: http://arxiv.org/abs/2409.10840v4
• Date: Sun, 10 Nov 2024 18:46:12 GMT
• Title: Implicit Reasoning in Deep Time Series Forecasting
• Authors: Willa Potosnak, Cristian Challu, Mononito Goswami, Michał Wiliński, Nina Żukowska, Artur Dubrawski,
• Abstract summary: This work takes an initial step toward assessing the reasoning abilities of deep time series forecasting models. We find that certain linear, patch-based Transformer models generalize effectively in systematically orchestrated out-of-distribution scenarios.
• Score: 16.750280337155647
• License:
• Abstract: Recently, time series foundation models have shown promising zero-shot forecasting performance on time series from a wide range of domains. However, it remains unclear whether their success stems from a true understanding of temporal dynamics or simply from memorizing the training data. While implicit reasoning in language models has been studied, similar evaluations for time series models have been largely unexplored. This work takes an initial step toward assessing the reasoning abilities of deep time series forecasting models. We find that certain linear, MLP-based, and patch-based Transformer models generalize effectively in systematically orchestrated out-of-distribution scenarios, suggesting underexplored reasoning capabilities beyond simple pattern memorization.

Related papers

• Recurrent Neural Goodness-of-Fit Test for Time Series [8.22915954499148]
  Time series data are crucial across diverse domains such as finance and healthcare. Traditional evaluation metrics fall short due to the temporal dependencies and potential high dimensionality of the features. We propose the REcurrent NeurAL (RENAL) Goodness-of-Fit test, a novel and statistically rigorous framework for evaluating generative time series models.
  arXiv  Detail & Related papers  (2024-10-17T19:32:25Z)
• Zero-shot forecasting of chaotic systems [6.445605125467573]
  Foundation models pre-trained on vast amounts of time-series data from diverse domains. We evaluate whether the zero-shot learning paradigm extends to the challenging task of forecasting chaotic systems.
  arXiv  Detail & Related papers  (2024-09-24T05:56:58Z)
• Deep Time Series Models: A Comprehensive Survey and Benchmark [74.28364194333447]
  Time series data is of great significance in real-world scenarios. Recent years have witnessed remarkable breakthroughs in the time series community. We release Time Series Library (TSLib) as a fair benchmark of deep time series models for diverse analysis tasks.
  arXiv  Detail & Related papers  (2024-07-18T08:31:55Z)
• TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling [67.02157180089573]
  Time series pre-training has recently garnered wide attention for its potential to reduce labeling expenses and benefit various downstream tasks. This paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
  arXiv  Detail & Related papers  (2024-02-04T13:10:51Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)
• Revisiting the Temporal Modeling in Spatio-Temporal Predictive Learning under A Unified View [73.73667848619343]
  We introduce USTEP (Unified S-TEmporal Predictive learning), an innovative framework that reconciles the recurrent-based and recurrent-free methods by integrating both micro-temporal and macro-temporal scales.
  arXiv  Detail & Related papers  (2023-10-09T16:17:42Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)
• MPR-Net:Multi-Scale Pattern Reproduction Guided Universality Time Series Interpretable Forecasting [13.790498420659636]
  Time series forecasting has received wide interest from existing research due to its broad applications inherent challenging. This paper proposes a forecasting model, MPR-Net. It first adaptively decomposes multi-scale historical series patterns using convolution operation, then constructs a pattern extension forecasting method based on the prior knowledge of pattern reproduction, and finally reconstructs future patterns into future series using deconvolution operation. By leveraging the temporal dependencies present in the time series, MPR-Net not only achieves linear time complexity, but also makes the forecasting process interpretable.
  arXiv  Detail & Related papers  (2023-07-13T13:16:01Z)
• Meta-Forecasting by combining Global DeepRepresentations with Local Adaptation [12.747008878068314]
  We introduce a novel forecasting method called Meta Global-Local Auto-Regression (Meta-GLAR) It adapts to each time series by learning in closed-form the mapping from the representations produced by a recurrent neural network (RNN) to one-step-ahead forecasts. Our method is competitive with the state-of-the-art in out-of-sample forecasting accuracy reported in earlier work.
  arXiv  Detail & Related papers  (2021-11-05T11:45:02Z)
• Analysis and modeling to forecast in time series: a systematic review [0.0]
  This paper surveys state-of-the-art methods and models dedicated to time series analysis and modeling, with the final aim of prediction. This review aims to offer a structured and comprehensive view of the full process flow, and encompasses time series decomposition, stationary tests, modeling and forecasting.
  arXiv  Detail & Related papers  (2021-03-31T23:48:46Z)
• Model-Attentive Ensemble Learning for Sequence Modeling [86.4785354333566]
  We present Model-Attentive Ensemble learning for Sequence modeling (MAES) MAES is a mixture of time-series experts which leverages an attention-based gating mechanism to specialize the experts on different sequence dynamics and adaptively weight their predictions. We demonstrate that MAES significantly out-performs popular sequence models on datasets subject to temporal shift.
  arXiv  Detail & Related papers  (2021-02-23T05:23:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.