Enhancing Channel-Independent Time Series Forecasting via Cross-Variate Patch Embedding

• URL: http://arxiv.org/abs/2505.12761v3
• Date: Thu, 22 May 2025 23:29:44 GMT
• Title: Enhancing Channel-Independent Time Series Forecasting via Cross-Variate Patch Embedding
• Abstract summary: We propose Cross-Variate Patch Embeddings (CVPE), a lightweight CD module that injects cross-variate context into channel-independent (CI) models.<n>We then integrate CVPE into Time-LLM, a multimodal CI forecasting model, to demonstrate its effectiveness.
• Score: 1.1607669836339873
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformers have recently gained popularity in time series forecasting due to their ability to capture long-term dependencies. However, many existing models focus only on capturing temporal dependencies while omitting intricate relationships between variables. Recent models have tried tackling this by explicitly modeling both cross-time and cross-variate dependencies through a sequential or unified attention mechanism, but they are entirely channel dependent (CD) across all layers, making them potentially susceptible to overfitting. To address this, we propose Cross-Variate Patch Embeddings (CVPE), a lightweight CD module that injects cross-variate context into channel-independent (CI) models by simply modifying the patch embedding process. We achieve this by adding a learnable positional encoding and a lightweight router-attention block to the vanilla patch embedding layer. We then integrate CVPE into Time-LLM, a multimodal CI forecasting model, to demonstrate its effectiveness in capturing cross-variate dependencies and enhance the CI model's performance. Extensive experimental results on seven real-world datasets show that our enhanced Time-LLM outperforms the original baseline model simply by incorporating the CVPE module, with no other changes.

Related papers

• PMDformer: Patch-Mean Decoupling Information Transformer for Long-term Forecasting [44.13811146979237]
  Long-term time series forecasting (LTSF) plays a crucial role in fields such as energy management, finance, and traffic prediction.<n>We introduce patch-mean decoupling (PMD), which separates the trend and residual shape information by subtracting the mean of each patch.<n>Futhermore, to more effectively model long-range dependencies and capture cross-variable relationships, we propose Trend Restoration Attention (TRA) and Proximal Variable Attention (PVA)
  arXiv  Detail & Related papers  (2026-06-25T02:49:08Z)
• XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis [10.642410240494238]
  We introduce XCTFormer, a transformer-based channel-dependent (CD) model that captures cross-temporal and cross-channel dependencies.<n>XCTFormer achieves strong results compared to widely recognized baselines.<n>It attains state-of-the-art performance on the imputation task, outperforming the second-best method by an average of 20.8% in MSE and 15.3% in MAE.
  arXiv  Detail & Related papers  (2026-05-18T15:18:22Z)
• Perceive, Route and Modulate: Dynamic Pattern Recalibration for Time Series Forecasting [68.53528085362642]
  Current deep forecasting models rely on fixed weight matrices applied uniformly to all temporal tokens.<n>We introduce Dynamic Pattern Recalibration, a backbone-agnostic mechanism that resolves this via token-level recalibration.<n>DPRNet achieves competitive performance across 12 benchmarks, validating dynamic recalibration against macroscopic parameter scaling.
  arXiv  Detail & Related papers  (2026-05-07T14:12:47Z)
• DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters [50.43534351968113]
  Existing generative time series models do not address the multi-dimensional properties of time series data well.<n>Inspired by Multimodal Diffusion Transformers that integrate textual guidance into video generation, we propose Diffusion Transformers for Time Series (DiTS)
  arXiv  Detail & Related papers  (2026-02-06T10:48:13Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models [52.502867924372275]
  Mixture-of-Experts (MoE) models achieve efficient scaling through sparse expert activation, but often suffer from suboptimal routing decisions due to distribution shifts in deployment.<n>We propose textita data-free, online test-time framework that continuously adapts MoE routing decisions during text generation without external supervision or data.
  arXiv  Detail & Related papers  (2025-10-16T16:24:36Z)
• Time-Correlated Video Bridge Matching [49.94768097995648]
  Time-Correlated Video Bridge Matching (TCVBM) is a framework that extends Bridge Matching (BM) to time-correlated data sequences in the video domain.<n>TCVBM achieves superior performance across multiple quantitative metrics, demonstrating enhanced generation quality and reconstruction fidelity.
  arXiv  Detail & Related papers  (2025-10-14T12:35:30Z)
• EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series Forecasting [50.794700596484894]
  We propose EntroPE (Entropy-Guided Dynamic Patch), a novel, temporally informed framework that dynamically detects transition points via conditional entropy.<n>This preserves temporal structure while retaining the computational benefits of patching.<n> Experiments across long-term forecasting benchmarks demonstrate that EntroPE improves both accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-09-30T12:09:56Z)
• CrossLinear: Plug-and-Play Cross-Correlation Embedding for Time Series Forecasting with Exogenous Variables [20.703629642499664]
  CrossLinear is a novel Linear-based forecasting model that addresses the challenges of modeling dependencies between variables.<n>This lightweight module captures the dependencies between variables with minimal computational cost and seamlessly integrates into existing neural networks.<n>Experiments on 12 real-world datasets demonstrate that CrossLinear achieves superior performance in both short-term and long-term forecasting tasks.
  arXiv  Detail & Related papers  (2025-05-29T05:34:40Z)
• VE: Modeling Multivariate Time Series Correlation with Variate Embedding [0.4893345190925178]
  Current channel-independent (CI) models and models with a CI final projection layer are unable to capture correlations. We present the variate embedding (VE) pipeline, which learns a unique and consistent embedding for each variate. The VE pipeline can be integrated into any model with a CI final projection layer to improve multivariate forecasting.
  arXiv  Detail & Related papers  (2024-09-10T02:49:30Z)
• UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting [98.12558945781693]
  We propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our experiments on several datasets for time series forecasting.
  arXiv  Detail & Related papers  (2024-06-07T14:39:28Z)
• VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting [1.5165632546654102]
  We propose Variable Correlation Transformer (VCformer) to mine the correlations among variables. VCA calculates and integrates the cross-correlation scores corresponding to different lags between queries and keys. Inspired by Koopman dynamics theory, we also develop Koopman Temporal Detector (KTD) to better address the non-stationarity in time series.
  arXiv  Detail & Related papers  (2024-05-19T07:39:22Z)
• SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion [59.96233305733875]
  Time series forecasting plays a crucial role in various fields such as finance, traffic management, energy, and healthcare. Several methods utilize mechanisms like attention or mixer to address this by capturing channel correlations. This paper presents an efficient-based model, the Series-cOre Fused Time Series forecaster (SOFTS)
  arXiv  Detail & Related papers  (2024-04-22T14:06:35Z)
• ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time Series Analysis [7.979501926410114]
  ConvTimeNet is a hierarchical pure convolutional model designed for time series analysis.<n>It adaptively perceives local patterns of temporally dependent basic units in a data-driven manner.<n>A large kernel mechanism is employed to ensure that convolutional blocks can be deeply stacked.
  arXiv  Detail & Related papers  (2024-03-03T12:05:49Z)
• Transform-Equivariant Consistency Learning for Temporal Sentence Grounding [66.10949751429781]
  We introduce a novel Equivariant Consistency Regulation Learning framework to learn more discriminative representations for each video. Our motivation comes from that the temporal boundary of the query-guided activity should be consistently predicted. In particular, we devise a self-supervised consistency loss module to enhance the completeness and smoothness of the augmented video.
  arXiv  Detail & Related papers  (2023-05-06T19:29:28Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)
• Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks [91.65637773358347]
  We propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module. Our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets.
  arXiv  Detail & Related papers  (2020-05-24T04:02:18Z)
• Convolutional Tensor-Train LSTM for Spatio-temporal Learning [116.24172387469994]
  We propose a higher-order LSTM model that can efficiently learn long-term correlations in the video sequence. This is accomplished through a novel tensor train module that performs prediction by combining convolutional features across time. Our results achieve state-of-the-art performance-art in a wide range of applications and datasets.
  arXiv  Detail & Related papers  (2020-02-21T05:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.