Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis

• URL: http://arxiv.org/abs/2310.06119v1
• Date: Mon, 9 Oct 2023 19:52:22 GMT
• Title: Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis
• Authors: Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Guangyin Jin, Xin Cao, Gao Cong, Christian S. Jensen, Xueqi Cheng
• Abstract summary: We introduce BasicTS, a benchmark designed for fair comparisons in MTS forecasting. We highlight the heterogeneity among MTS datasets and classify them based on temporal and spatial characteristics.
• Score: 72.18987459587682
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multivariate Time Series (MTS) widely exists in real-word complex systems, such as traffic and energy systems, making their forecasting crucial for understanding and influencing these systems. Recently, deep learning-based approaches have gained much popularity for effectively modeling temporal and spatial dependencies in MTS, specifically in Long-term Time Series Forecasting (LTSF) and Spatial-Temporal Forecasting (STF). However, the fair benchmarking issue and the choice of technical approaches have been hotly debated in related work. Such controversies significantly hinder our understanding of progress in this field. Thus, this paper aims to address these controversies to present insights into advancements achieved. To resolve benchmarking issues, we introduce BasicTS, a benchmark designed for fair comparisons in MTS forecasting. BasicTS establishes a unified training pipeline and reasonable evaluation settings, enabling an unbiased evaluation of over 30 popular MTS forecasting models on more than 18 datasets. Furthermore, we highlight the heterogeneity among MTS datasets and classify them based on temporal and spatial characteristics. We further prove that neglecting heterogeneity is the primary reason for generating controversies in technical approaches. Moreover, based on the proposed BasicTS and rich heterogeneous MTS datasets, we conduct an exhaustive and reproducible performance and efficiency comparison of popular models, providing insights for researchers in selecting and designing MTS forecasting models.

Related papers

• METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark [48.78602579128459]
  We introduce METER, a unified benchmark for interpretable forgery detection spanning images, videos, audio, and audio-visual content.<n>Our dataset comprises four tracks, each requiring not only real-vs-fake classification but also evidence-chain-based explanations.
  arXiv  Detail & Related papers  (2025-07-22T03:42:51Z)
• Benchmarking Multi-modal Semantic Segmentation under Sensor Failures: Missing and Noisy Modality Robustness [61.87055159919641]
  Multi-modal semantic segmentation (MMSS) addresses the limitations of single-modality data by integrating complementary information across modalities. Despite notable progress, a significant gap persists between research and real-world deployment due to variability and uncertainty in multi-modal data quality. We introduce a robustness benchmark that evaluates MMSS models under three scenarios: Entire-Missing Modality (EMM), Random-Missing Modality (RMM), and Noisy Modality (NM)
  arXiv  Detail & Related papers  (2025-03-24T08:46:52Z)
• IMTS-Mixer: Mixer-Networks for Irregular Multivariate Time Series Forecasting [5.854515369288696]
  We introduce IMTS-Mixer, a novel forecasting architecture designed specifically for IMTS. Our approach retains the core principles of TS mixer models while introducing innovative methods to transform IMTS into fixed-size matrix representations. Our results demonstrate that IMTS-Mixer establishes a new state-of-the-art in forecasting accuracy while also improving computational efficiency.
  arXiv  Detail & Related papers  (2025-02-17T14:06:36Z)
• Contrast Similarity-Aware Dual-Pathway Mamba for Multivariate Time Series Node Classification [9.159556125198305]
  We propose contrast similarity-aware dual-pathway Mamba for MTS node classification (CS-DPMamba) We construct a similarity matrix between MTS representations using Fast Dynamic Time Warping (FastDTW) By considering the long-range dependencies and dynamic similarity features, we achieved precise MTS node classification.
  arXiv  Detail & Related papers  (2024-11-19T04:32:41Z)
• RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification [70.9087014537896]
  Target-oriented Multimodal Sentiment Classification (TMSC) has gained significant attention among scholars. To investigate the causes of this problem, we perform extensive empirical evaluation and in-depth analysis of the datasets.
  arXiv  Detail & Related papers  (2023-10-14T14:52:37Z)
• Graph-Aware Contrasting for Multivariate Time-Series Classification [50.84488941336865]
  Existing contrastive learning methods mainly focus on achieving temporal consistency with temporal augmentation and contrasting techniques. We propose Graph-Aware Contrasting for spatial consistency across MTS data. Our proposed method achieves state-of-the-art performance on various MTS classification tasks.
  arXiv  Detail & Related papers  (2023-09-11T02:35:22Z)
• Beyond Sharing: Conflict-Aware Multivariate Time Series Anomaly Detection [18.796225184893874]
  We introduce CAD, a Conflict-aware Anomaly Detection algorithm. We find that the poor performance of vanilla MMoE mainly comes from the input-output misalignment settings of MTS formulation. We show that CAD obtains an average F1-score of 0.943 across three public datasets, notably outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-17T11:00:01Z)
• Contextualizing MLP-Mixers Spatiotemporally for Urban Data Forecast at Scale [54.15522908057831]
  We propose an adapted version of the computationally-Mixer for STTD forecast at scale. Our results surprisingly show that this simple-yeteffective solution can rival SOTA baselines when tested on several traffic benchmarks. Our findings contribute to the exploration of simple-yet-effective models for real-world STTD forecasting.
  arXiv  Detail & Related papers  (2023-07-04T05:19:19Z)
• SMATE: Semi-Supervised Spatio-Temporal Representation Learning on Multivariate Time Series [0.6445605125467572]
  We propose SMATE, a novel semi-supervised model for learning the interpretable Spatio-Temporal representation from weakly labeled MTS. We validate the learned representation on 22 public datasets from the UEA MTS archive.
  arXiv  Detail & Related papers  (2021-10-01T17:59:46Z)
• LIFE: Learning Individual Features for Multivariate Time Series Prediction with Missing Values [71.52335136040664]
  We propose a Learning Individual Features (LIFE) framework, which provides a new paradigm for MTS prediction with missing values. LIFE generates reliable features for prediction by using the correlated dimensions as auxiliary information and suppressing the interference from uncorrelated dimensions with missing values. Experiments on three real-world data sets verify the superiority of LIFE to existing state-of-the-art models.
  arXiv  Detail & Related papers  (2021-09-30T04:53:24Z)
• Mutual-Information Based Few-Shot Classification [34.95314059362982]
  We introduce Transductive Infomation Maximization (TIM) for few-shot learning. Our method maximizes the mutual information between the query features and their label predictions for a given few-shot task. We propose a new alternating-direction solver, which speeds up transductive inference over gradient-based optimization.
  arXiv  Detail & Related papers  (2021-06-23T09:17:23Z)
• Transductive Information Maximization For Few-Shot Learning [41.461586994394565]
  We introduce Transductive Infomation Maximization (TIM) for few-shot learning. Our method maximizes the mutual information between the query features and their label predictions for a given few-shot task. We propose a new alternating-direction solver for our mutual-information loss.
  arXiv  Detail & Related papers  (2020-08-25T22:38:41Z)
• Multi-Task Learning for Dense Prediction Tasks: A Survey [87.66280582034838]
  Multi-task learning (MTL) techniques have shown promising results w.r.t. performance, computations and/or memory footprint. We provide a well-rounded view on state-of-the-art deep learning approaches for MTL in computer vision.
  arXiv  Detail & Related papers  (2020-04-28T09:15:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.