Accurate and Efficient Multi-Channel Time Series Forecasting via Sparse Attention Mechanism

• URL: http://arxiv.org/abs/2603.18712v1
• Date: Thu, 19 Mar 2026 10:11:23 GMT
• Title: Accurate and Efficient Multi-Channel Time Series Forecasting via Sparse Attention Mechanism
• Abstract summary: This paper proposes Linear-Network (Li-Net), a novel architecture designed for multi-channel time series forecasting.<n>Li-Net compresses representations across sequence and channel dimensions, processes the information through a non-linear module and subsequently reconstructs the forecasts.<n>A core innovation is its ability to seamlessly incorporate and fuse multi-modal embeddings, guiding the sparse attention process to focus on the most informative time steps.
• Score: 5.742623694016163
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The task of multi-channel time series forecasting is ubiquitous in numerous fields such as finance, supply chain management, and energy planning. It is critical to effectively capture complex dynamic dependencies within and between channels for accurate predictions. However, traditional method paid few attentions on learning the interaction among channels. This paper proposes Linear-Network (Li-Net), a novel architecture designed for multi-channel time series forecasting that captures the linear and non-linear dependencies among channels. Li-Net dynamically compresses representations across sequence and channel dimensions, processes the information through a configurable non-linear module and subsequently reconstructs the forecasts. Moreover, Li-Net integrates a sparse Top-K Softmax attention mechanism within a multi-scale projection framework to address these challenges. A core innovation is its ability to seamlessly incorporate and fuse multi-modal embeddings, guiding the sparse attention process to focus on the most informative time steps and feature channels. Through the experiment results on multiple real-world benchmark datasets demonstrate that Li-Net achieves competitive performance compared to state-of-the-art baseline methods. Furthermore, Li-Net provides a superior balance between prediction accuracy and computational burden, exhibiting significantly lower memory usage and faster inference times. Detailed ablation studies and parameter sensitivity analyses validate the effectiveness of each key component in our proposed architecture. Keywords: Multivariate Time Series Forecasting, Sparse Attention Mechanism, Multimodal Information Fusion, Non-linear relationship

Related papers

• Adapformer: Adaptive Channel Management for Multivariate Time Series Forecasting [49.40321003932633]
  Adapformer is an advanced Transformer-based framework that merges the benefits of CI and CD methodologies through effective channel management.<n>Adapformer achieves superior performance over existing models, enhancing both predictive accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2025-11-18T16:24:05Z)
• A Wireless Foundation Model for Multi-Task Prediction [50.21098141769079]
  We propose a unified foundation model for multi-task prediction in wireless networks that supports diverse prediction intervals.<n>After trained on large-scale datasets, the proposed foundation model demonstrates strong generalization to unseen scenarios and zero-shot performance on new tasks.
  arXiv  Detail & Related papers  (2025-07-08T12:37:55Z)
• Accurate and Efficient Multivariate Time Series Forecasting via Offline Clustering [22.545533166145706]
  We introduce the Forecaster with Offline Clustering Using Segments (FOCUS)<n>FOCUS is a novel approach to MTS forecasting that simplifies long-range dependency modeling.<n>It achieves state-of-the-art accuracy while significantly reducing computational costs.
  arXiv  Detail & Related papers  (2025-05-09T02:34:06Z)
• In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention [52.159541540613915]
  We study how multi-head softmax attention models are trained to perform in-context learning on linear data.<n>Our results reveal that in-context learning ability emerges from the trained transformer as an aggregated effect of its architecture and the underlying data distribution.
  arXiv  Detail & Related papers  (2025-03-17T02:00:49Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies.<n>We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis.<n> Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• DisenTS: Disentangled Channel Evolving Pattern Modeling for Multivariate Time Series Forecasting [43.071713191702486]
  DisenTS is a tailored framework for modeling disentangled channel evolving patterns in general time series forecasting. We introduce a novel Forecaster Aware Gate (FAG) module that generates the routing signals adaptively according to both the forecasters' states and input series' characteristics.
  arXiv  Detail & Related papers  (2024-10-30T12:46:14Z)
• Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning [0.0]
  patio-temporal forecasting plays a crucial role in various sectors such as transportation systems, logistics, and supply chain management. We introduce a hybrid approach that combines the strengths of open-source large and small-scale language models (LLMs and LMs) with traditional forecasting methods.
  arXiv  Detail & Related papers  (2024-08-26T16:11:53Z)
• SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion [59.96233305733875]
  Time series forecasting plays a crucial role in various fields such as finance, traffic management, energy, and healthcare. Several methods utilize mechanisms like attention or mixer to address this by capturing channel correlations. This paper presents an efficient-based model, the Series-cOre Fused Time Series forecaster (SOFTS)
  arXiv  Detail & Related papers  (2024-04-22T14:06:35Z)
• CSformer: Combining Channel Independence and Mixing for Robust Multivariate Time Series Forecasting [3.6814181034608664]
  We propose a strategy of channel independence followed by mixing in time series analysis.<n>We introduce CSformer, a novel framework featuring a two-stage multiheaded self-attention mechanism.<n>Our framework effectively incorporates sequence and channel adapters, significantly improving the model's ability to identify important information.
  arXiv  Detail & Related papers  (2023-12-11T09:10:38Z)
• Perceiver-based CDF Modeling for Time Series Forecasting [25.26713741799865]
  We propose a new architecture, called perceiver-CDF, for modeling cumulative distribution functions (CDF) of time series data. Our approach combines the perceiver architecture with a copula-based attention mechanism tailored for multimodal time series prediction. Experiments on the unimodal and multimodal benchmarks consistently demonstrate a 20% improvement over state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-10-03T01:13:17Z)
• TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series [57.4208255711412]
  Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS) We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-02T16:45:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.