Partial Channel Dependence with Channel Masks for Time Series Foundation Models

• URL: http://arxiv.org/abs/2410.23222v1
• Date: Wed, 30 Oct 2024 17:12:03 GMT
• Title: Partial Channel Dependence with Channel Masks for Time Series Foundation Models
• Authors: Seunghan Lee, Taeyoung Park, Kibok Lee,
• Abstract summary: We introduce the concept of partial channel dependence (PCD), which enables a more sophisticated adjustment of channel dependencies based on dataset-specific information. We validate the effectiveness of PCD across four tasks in time series (TS) including forecasting, classification, imputation, and anomaly detection.
• Score: 5.752266579415516
• License:
• Abstract: Recent advancements in foundation models have been successfully extended to the time series (TS) domain, facilitated by the emergence of large-scale TS datasets. However, previous efforts have primarily focused on designing model architectures to address explicit heterogeneity among datasets such as various numbers of channels, while often overlooking implicit heterogeneity such as varying dependencies between channels. In this work, we introduce the concept of partial channel dependence (PCD), which enables a more sophisticated adjustment of channel dependencies based on dataset-specific information. To achieve PCD, we propose a channel mask that captures the relationships between channels within a dataset using two key components: 1) a correlation matrix that encodes relative dependencies between channels, and 2) domain parameters that learn the absolute dependencies specific to each dataset, refining the correlation matrix. We validate the effectiveness of PCD across four tasks in TS including forecasting, classification, imputation, and anomaly detection, under diverse settings, including few-shot and zero-shot scenarios with both TS foundation models and single-task models. Code is available at https://github.com/seunghan96/CM.

Related papers

• DisenTS: Disentangled Channel Evolving Pattern Modeling for Multivariate Time Series Forecasting [43.071713191702486]
  DisenTS is a tailored framework for modeling disentangled channel evolving patterns in general time series forecasting. We introduce a novel Forecaster Aware Gate (FAG) module that generates the routing signals adaptively according to both the forecasters' states and input series' characteristics.
  arXiv  Detail & Related papers  (2024-10-30T12:46:14Z)
• A SSM is Polymerized from Multivariate Time Series [0.0]
  We develop Poly-Mamba, a novel method for MTS forecasting. Experiments on six real-world datasets demonstrate that Poly-Mamba outperforms the SOTA methods.
  arXiv  Detail & Related papers  (2024-09-30T14:10:02Z)
• Scalable Transformer for High Dimensional Multivariate Time Series Forecasting [10.17270031004674]
  This study investigates the reasons behind the suboptimal performance of channel-dependent models on high-dimensional MTS data. We propose STHD, the Scalable Transformer for High-Dimensional Multidimensional Time Series Forecasting. Experiments show STHD's considerable improvement on three high-dimensional datasets: Crime-Chicago, Wiki-People, and Traffic.
  arXiv  Detail & Related papers  (2024-08-08T06:17:13Z)
• UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting [98.12558945781693]
  We propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our experiments on several datasets for time series forecasting.
  arXiv  Detail & Related papers  (2024-06-07T14:39:28Z)
• SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion [59.96233305733875]
  Time series forecasting plays a crucial role in various fields such as finance, traffic management, energy, and healthcare. Several methods utilize mechanisms like attention or mixer to address this by capturing channel correlations. This paper presents an efficient-based model, the Series-cOre Fused Time Series forecaster (SOFTS)
  arXiv  Detail & Related papers  (2024-04-22T14:06:35Z)
• From Similarity to Superiority: Channel Clustering for Time Series Forecasting [61.96777031937871]
  We develop a novel and adaptable Channel Clustering Module ( CCM) CCM dynamically groups channels characterized by intrinsic similarities and leverages cluster information instead of individual channel identities. CCM can boost the performance of CI and CD models by an average margin of 2.4% and 7.2% on long-term and short-term forecasting, respectively.
  arXiv  Detail & Related papers  (2024-03-31T02:46:27Z)
• Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data [50.84488941336865]
  We propose a novel method called Fully- Spatial-Temporal Graph Neural Network (FC-STGNN) For graph construction, we design a decay graph to connect sensors across all timestamps based on their temporal distances. For graph convolution, we devise FC graph convolution with a moving-pooling GNN layer to effectively capture the ST dependencies for learning effective representations.
  arXiv  Detail & Related papers  (2023-09-11T08:44:07Z)
• Graphical Modeling for Multi-Source Domain Adaptation [56.05348879528149]
  Multi-Source Domain Adaptation (MSDA) focuses on transferring the knowledge from multiple source domains to the target domain. We propose two types of graphical models,i.e. Conditional Random Field for MSDA (CRF-MSDA) and Markov Random Field for MSDA (MRF-MSDA) We evaluate these two models on four standard benchmark data sets of MSDA with distinct domain shift and data complexity.
  arXiv  Detail & Related papers  (2021-04-27T09:04:22Z)
• Ensemble Model with Batch Spectral Regularization and Data Blending for Cross-Domain Few-Shot Learning with Unlabeled Data [75.94147344921355]
  We build a multi-branch ensemble framework by using diverse feature transformation matrices. We propose a data blending method to exploit the unlabeled data and augment the sparse support set in the target domain.
  arXiv  Detail & Related papers  (2020-06-08T02:27:34Z)
• Forecast Network-Wide Traffic States for Multiple Steps Ahead: A Deep Learning Approach Considering Dynamic Non-Local Spatial Correlation and Non-Stationary Temporal Dependency [6.019104024723682]
  This research studies two particular problems in traffic forecasting: (1) capture the dynamic and non-local spatial correlation between traffic links and (2) model the dynamics of temporal dependency for accurate multiple steps ahead predictions. We propose a deep learning framework named Spatial-Temporal Sequence to Sequence model (STSeq2Seq) to address these issues. This model builds on sequence to sequence (seq2seq) architecture to capture temporal feature and relies on graph convolution for aggregating spatial information.
  arXiv  Detail & Related papers  (2020-04-06T03:40:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.