Related papers: Towards Long-Context Time Series Foundation Models

Towards Long-Context Time Series Foundation Models

URL: http://arxiv.org/abs/2409.13530v1
Date: Fri, 20 Sep 2024 14:19:59 GMT
Title: Towards Long-Context Time Series Foundation Models
Authors: Nina Żukowska, Mononito Goswami, Michał Wiliński, Willa Potosnak, Artur Dubrawski,
Abstract summary: Time series foundation models have shown impressive performance on a variety of tasks, across a wide range of domains, even in zero-shot settings. This study bridges the gap by systematically comparing various context expansion techniques from both language and time series domains.
Score: 17.224575072056627
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Time series foundation models have shown impressive performance on a variety of tasks, across a wide range of domains, even in zero-shot settings. However, most of these models are designed to handle short univariate time series as an input. This limits their practical use, especially in domains such as healthcare with copious amounts of long and multivariate data with strong temporal and intra-variate dependencies. Our study bridges this gap by cataloging and systematically comparing various context expansion techniques from both language and time series domains, and introducing a novel compressive memory mechanism to allow encoder-only TSFMs to effectively model intra-variate dependencies. We demonstrate the benefits of our approach by imbuing MOMENT, a recent family of multi-task time series foundation models, with the multivariate context.

Related papers

Towards Generalisable Time Series Understanding Across Domains [10.350643783811174]
We introduce OTiS, an open model for general time series analysis. We propose a novel pre-training paradigm including a tokeniser with learnable domain-specific signatures. Our model is pre-trained on a large corpus of 640,187 samples and 11 billion time points spanning 8 distinct domains.
arXiv Detail & Related papers (2024-10-09T17:09:30Z)
Deep Time Series Models: A Comprehensive Survey and Benchmark [74.28364194333447]
Time series data is of great significance in real-world scenarios. Recent years have witnessed remarkable breakthroughs in the time series community. We release Time Series Library (TSLib) as a fair benchmark of deep time series models for diverse analysis tasks.
arXiv Detail & Related papers (2024-07-18T08:31:55Z)
PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the perspective of partial differential equations [49.80959046861793]
We present PDETime, a novel LMTF model inspired by the principles of Neural PDE solvers. Our experimentation across seven diversetemporal real-world LMTF datasets reveals that PDETime adapts effectively to the intrinsic nature of the data.
arXiv Detail & Related papers (2024-02-25T17:39:44Z)
UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting [59.11817101030137]
This research advocates for a unified model paradigm that transcends domain boundaries. Learning an effective cross-domain model presents the following challenges. We propose UniTime for effective cross-domain time series learning.
arXiv Detail & Related papers (2023-10-15T06:30:22Z)
Forecasting large collections of time series: feature-based methods [7.353918137830393]
When forecasting large collections of time series, two lines of approaches have been developed using time series features. This chapter discusses the state-of-the-art feature-based methods, with reference to open-source software implementations.
arXiv Detail & Related papers (2023-09-25T01:23:02Z)
SageFormer: Series-Aware Framework for Long-term Multivariate Time Series Forecasting [16.395374003276817]
This paper introduces a novel series-aware framework, explicitly designed to emphasize the significance of inter-series dependencies. As a Series-aware Graph-enhanced Transformer model, SageFormer proficiently discerns and models the intricate relationships between series using graph structures. Notably, the series-aware framework seamlessly integrates with existing Transformer-based models, enriching their ability to comprehend inter-series relationships.
arXiv Detail & Related papers (2023-07-04T10:08:25Z)
Feature Programming for Multivariate Time Series Prediction [7.0220697993232]
We introduce the concept of programmable feature engineering for time series modeling. We propose a feature programming framework that generates large amounts of predictive features for noisy time series.
arXiv Detail & Related papers (2023-06-09T20:46:55Z)
Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
arXiv Detail & Related papers (2022-05-16T07:53:42Z)
Instance-wise Graph-based Framework for Multivariate Time Series Forecasting [69.38716332931986]
We propose a simple yet efficient instance-wise graph-based framework to utilize the inter-dependencies of different variables at different time stamps. The key idea of our framework is aggregating information from the historical time series of different variables to the current time series that we need to forecast.
arXiv Detail & Related papers (2021-09-14T07:38:35Z)
Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks [91.65637773358347]
We propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module. Our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets.
arXiv Detail & Related papers (2020-05-24T04:02:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.