ViTime: A Visual Intelligence-Based Foundation Model for Time Series Forecasting

• URL: http://arxiv.org/abs/2407.07311v3
• Date: Sat, 08 Feb 2025 05:05:56 GMT
• Title: ViTime: A Visual Intelligence-Based Foundation Model for Time Series Forecasting
• Authors: Luoxiao Yang, Yun Wang, Xinqi Fan, Israel Cohen, Jingdong Chen, Yue Zhao, Zijun Zhang,
• Abstract summary: Time series forecasting (TSF) possesses great practical values in various fields, including power and energy, transportation, etc.<n>This paper offers a pioneering study in developing a TSF foundation model and proposes a vision intelligence-powered framework, ViTime, for the first time.
• Score: 38.87384888881476
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time series forecasting (TSF) possesses great practical values in various fields, including power and energy, transportation, etc. TSF methods have been studied based on knowledge from classical statistics to modern deep learning. Yet, all of them were developed based on one fundamental concept, the numerical data fitting. Thus, the models developed have been long known for being problem-specific and lacking application generalizability. A TSF foundation model serving TSF tasks across different applications can reverse such an impression. The central question is then how to develop such a TSF foundation model. This paper offers a pioneering study in developing a TSF foundation model and proposes a vision intelligence-powered framework, ViTime, for the first time. In ViTime, a method synthesizing authentic time series periodic and trend patterns is developed to enrich sample pattern diversity. A deep architecture operating TSF in image metric space is designed to achieve significantly enhanced TSF generalizability. Extensive experiments demonstrate ViTime's SOTA performance across multiple settings. In zero-shot scenarios, ViTime outperforms TimesFM by 9-15%. With just 10% fine-tuning data, ViTime surpasses both foundation models and fully-supervised benchmarks trained on complete datasets, with this performance gap widening further at 100\% fine-tuning. Additionally, ViTime exhibits exceptional robustness, handling missing data without imputation and outperforming TimesFM by 20-30% under various data perturbations.

Related papers

• UniSTD: Towards Unified Spatio-Temporal Learning across Diverse Disciplines [64.84631333071728]
  We introduce bfUnistage, a unified Transformer-based framework fortemporal modeling. Our work demonstrates that a task-specific vision-text can build a generalizable model fortemporal learning. We also introduce a temporal module to incorporate temporal dynamics explicitly.
  arXiv  Detail & Related papers  (2025-03-26T17:33:23Z)
• TimesBERT: A BERT-Style Foundation Model for Time Series Understanding [72.64824086839631]
  GPT-style models have been positioned as foundation models for time series forecasting. BERT-style architecture has not been fully unlocked for time series understanding. We design TimesBERT to learn generic representations of time series. Our model is pre-trained on 260 billion time points across diverse domains.
  arXiv  Detail & Related papers  (2025-02-28T17:14:44Z)
• Beyond Fixed Variables: Expanding-variate Time Series Forecasting via Flat Scheme and Spatio-temporal Focal Learning [9.205228068704141]
  In real-world applications, Cyber-Physical Systems often expand as new sensors are, increasing variables in MTSF. This task presents unique challenges, specifically (1) handling inconsistent data caused by adding new variables, and (2) addressing imbalanced-temporal learning. To address these challenges, we propose STEV, a flexible-temporal forecasting framework.
  arXiv  Detail & Related papers  (2025-02-21T08:43:26Z)
• FoundTS: Comprehensive and Unified Benchmarking of Foundation Models for Time Series Forecasting [44.33565276128137]
  Time Series Forecasting (TSF) is key functionality in numerous fields, including in finance, weather services, and energy management. Foundation models exhibit promising inferencing capabilities in new or unseen data. We propose a new benchmark, FoundTS, to enable thorough and fair evaluation and comparison of such models.
  arXiv  Detail & Related papers  (2024-10-15T17:23:49Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters [27.80286758290421]
  This paper explores a new road to building a TSF foundation model from rich, high-quality natural images. By reformulating TSF as an image reconstruction task, we bridge the gap between image pre-training and TSF downstream tasks. The proposed VisionTS could achieve superior zero-shot forecasting performance compared to existing TSF foundation models.
  arXiv  Detail & Related papers  (2024-08-30T12:51:55Z)
• Deep Time Series Models: A Comprehensive Survey and Benchmark [74.28364194333447]
  Time series data is of great significance in real-world scenarios. Recent years have witnessed remarkable breakthroughs in the time series community. We release Time Series Library (TSLib) as a fair benchmark of deep time series models for diverse analysis tasks.
  arXiv  Detail & Related papers  (2024-07-18T08:31:55Z)
• LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecasting [69.33802286580786]
  We introduce LTSM-Bundle, a comprehensive toolbox, and benchmark for training LTSMs. It modularized and benchmarked LTSMs from multiple dimensions, encompassing prompting strategies, tokenization approaches, base model selection, data quantity, and dataset diversity. Empirical results demonstrate that this combination achieves superior zero-shot and few-shot performances compared to state-of-the-art LTSMs and traditional TSF methods.
  arXiv  Detail & Related papers  (2024-06-20T07:09:19Z)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models [83.03091523806668]
  This paper aims at the early development of large time series models (LTSM) During pre-training, we curate large-scale datasets with up to 1 billion time points. To meet diverse application needs, we convert forecasting, imputation, and anomaly detection of time series into a unified generative task.
  arXiv  Detail & Related papers  (2024-02-04T06:55:55Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)
• Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain [54.67888148566323]
  We introduce three large-scale time series forecasting datasets from the cloud operations domain. We show it is a strong zero-shot baseline and benefits from further scaling, both in model and dataset size. Accompanying these datasets and results is a suite of comprehensive benchmark results comparing classical and deep learning baselines to our pre-trained method.
  arXiv  Detail & Related papers  (2023-10-08T08:09:51Z)
• TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting [24.834846119163885]
  We propose a novel framework, TEMPO, that can effectively learn time series representations. TEMPO expands the capability for dynamically modeling real-world temporal phenomena from data within diverse domains.
  arXiv  Detail & Related papers  (2023-10-08T00:02:25Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)
• FrAug: Frequency Domain Augmentation for Time Series Forecasting [6.508992154478217]
  Data augmentation (DA) has become a de facto solution to expand training data size for deep learning. This paper proposes simple yet effective frequency domain augmentation techniques that ensure the semantic consistency of augmented data-label pairs in forecasting. Our results show that FrAug can boost the forecasting accuracy of TSF models in most cases.
  arXiv  Detail & Related papers  (2023-02-18T11:25:42Z)
• VQ-AR: Vector Quantized Autoregressive Probabilistic Time Series Forecasting [10.605719154114354]
  Time series models aim for accurate predictions of the future given the past, where the forecasts are used for important downstream tasks like business decision making. In this paper, we introduce a novel autoregressive architecture, VQ-AR, which instead learns a emphdiscrete set of representations that are used to predict the future.
  arXiv  Detail & Related papers  (2022-05-31T15:43:46Z)
• Visualising Deep Network's Time-Series Representations [93.73198973454944]
  Despite the popularisation of machine learning models, more often than not they still operate as black boxes with no insight into what is happening inside the model. In this paper, a method that addresses that issue is proposed, with a focus on visualising multi-dimensional time-series data. Experiments on a high-frequency stock market dataset show that the method provides fast and discernible visualisations.
  arXiv  Detail & Related papers  (2021-03-12T09:53:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.