FRWKV:Frequency-Domain Linear Attention for Long-Term Time Series Forecasting

• URL: http://arxiv.org/abs/2512.07539v2
• Date: Tue, 09 Dec 2025 06:21:22 GMT
• Title: FRWKV:Frequency-Domain Linear Attention for Long-Term Time Series Forecasting
• Authors: Qingyuan Yang, Shizhuo Deng, Dongyue Chen, Da Teng, Zehua Gan,
• Abstract summary: Traditional Transformers face a major bottleneck in long-sequence time series forecasting due to their quadratic complexity.<n>Inspired by RWKV's $mathcalO(T)$ linear attention and frequency-domain modeling, we propose FRWKV.<n>Our model integrates linear attention mechanisms with frequency-domain analysis, achieving $mathcalO(T)$ computational complexity in the attention path.
• Score: 9.807623025167093
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Traditional Transformers face a major bottleneck in long-sequence time series forecasting due to their quadratic complexity $(\mathcal{O}(T^2))$ and their limited ability to effectively exploit frequency-domain information. Inspired by RWKV's $\mathcal{O}(T)$ linear attention and frequency-domain modeling, we propose FRWKV, a frequency-domain linear-attention framework that overcomes these limitations. Our model integrates linear attention mechanisms with frequency-domain analysis, achieving $\mathcal{O}(T)$ computational complexity in the attention path while exploiting spectral information to enhance temporal feature representations for scalable long-sequence modeling. Across eight real-world datasets, FRWKV achieves a first-place average rank. Our ablation studies confirm the critical roles of both the linear attention and frequency-encoder components. This work demonstrates the powerful synergy between linear attention and frequency analysis, establishing a new paradigm for scalable time series modeling. Code is available at this repository: https://github.com/yangqingyuan-byte/FRWKV.

Related papers

• FuXi-Linear: Unleashing the Power of Linear Attention in Long-term Time-aware Sequential Recommendation [86.55349738440087]
  FuXi-Linear is a linear-complexity model designed for efficient long-sequence recommendation.<n>Our approach introduces two key components: (1) a Temporal Retention Channel that independently computes periodic attention weights using temporal data, preventing crosstalk between temporal and semantic signals; and (2) a Linear Positional Channel that integrates positional information through learnable kernels within linear complexity.
  arXiv  Detail & Related papers  (2026-02-27T04:38:28Z)
• A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
  Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
  arXiv  Detail & Related papers  (2025-10-11T09:59:25Z)
• Multivariate Long-term Time Series Forecasting with Fourier Neural Filter [42.60778405812048]
  We introduce FNF as the backbone and DBD as architecture to provide excellent learning capabilities and optimal learning pathways for spatial-temporal modeling.<n>We show that FNF unifies local time-domain and global frequency-domain information processing within a single backbone that extends naturally to spatial modeling.
  arXiv  Detail & Related papers  (2025-06-10T18:40:20Z)
• Log-Linear Attention [81.09631871212211]
  This paper develops log-linear attention, an attention mechanism that balances linear attention's efficiency and the expressiveness of softmax attention.<n>We show that with a particular growth function, log-linear attention admits a similarly matmul-rich parallel form whose compute cost is log-linear in sequence length.<n>Log-linear attention is a general framework and can be applied on top of existing linear attention variants.
  arXiv  Detail & Related papers  (2025-06-05T08:44:51Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies.<n>We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis.<n> Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting [17.738942892605234]
  This paper presents textbfFreEformer, a simple yet effective model that leverages a textbfFrequency textbfEnhanced Transtextbfformer.<n>Experiments demonstrate that FreEformer consistently outperforms state-of-the-art models on eighteen real-world benchmarks.
  arXiv  Detail & Related papers  (2025-01-23T08:53:45Z)
• Local Attention Mechanism: Boosting the Transformer Architecture for Long-Sequence Time Series Forecasting [8.841114905151152]
  Local Attention Mechanism (LAM) is an efficient attention mechanism tailored for time series analysis.<n>LAM exploits the continuity properties of time series to reduce the number of attention scores computed.<n>We present an algorithm for implementing LAM in algebra tensor that runs in time and memory O(nlogn)
  arXiv  Detail & Related papers  (2024-10-04T11:32:02Z)
• Fi$^2$VTS: Time Series Forecasting Via Capturing Intra- and Inter-Variable Variations in the Frequency Domain [6.61394789494625]
  Time series forecasting (TSF) plays a crucial role in various applications, including medical monitoring and crop growth. We introduce the Fi$2$VBlock, which leverages a textbfFrequency domain perspective to capture textbfintra- and textbfinter- textbfVariations. Inception blocks are employed to integrate information, thus capturing correlations across different variables. Our backbone network, Fi$2$VTS, employs a residual architecture by concatenating multiple Fi$2$
  arXiv  Detail & Related papers  (2024-07-31T01:50:39Z)
• FAITH: Frequency-domain Attention In Two Horizons for Time Series Forecasting [13.253624747448935]
  Time Series Forecasting plays a crucial role in various fields such as industrial equipment maintenance, meteorology, energy consumption, traffic flow and financial investment. Current deep learning-based predictive models often exhibit a significant deviation between their forecasting outcomes and the ground truth. We propose a novel model Frequency-domain Attention In Two Horizons, which decomposes time series into trend and seasonal components.
  arXiv  Detail & Related papers  (2024-05-22T02:37:02Z)
• Time Series Diffusion in the Frequency Domain [54.60573052311487]
  We analyze whether representing time series in the frequency domain is a useful inductive bias for score-based diffusion models. We show that a dual diffusion process occurs in the frequency domain with an important nuance. We show how to adapt the denoising score matching approach to implement diffusion models in the frequency domain.
  arXiv  Detail & Related papers  (2024-02-08T18:59:05Z)
• CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting [50.23240107430597]
  We design a special Transformer, i.e., Channel Aligned Robust Blend Transformer (CARD for short), that addresses key shortcomings of CI type Transformer in time series forecasting. First, CARD introduces a channel-aligned attention structure that allows it to capture both temporal correlations among signals. Second, in order to efficiently utilize the multi-scale knowledge, we design a token blend module to generate tokens with different resolutions. Third, we introduce a robust loss function for time series forecasting to alleviate the potential overfitting issue.
  arXiv  Detail & Related papers  (2023-05-20T05:16:31Z)
• Learning Wave Propagation with Attention-Based Convolutional Recurrent Autoencoder Net [0.0]
  We present an end-to-end attention-based convolutional recurrent autoencoder (AB-CRAN) network for data-driven modeling of wave propagation phenomena. We employ a denoising-based convolutional autoencoder from the full-order snapshots given by time-dependent hyperbolic partial differential equations for wave propagation. The attention-based sequence-to-sequence network increases the time-horizon of prediction by five times compared to the plain RNN-LSTM.
  arXiv  Detail & Related papers  (2022-01-17T20:51:59Z)
• Deep Autoregressive Models with Spectral Attention [74.08846528440024]
  We propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise.
  arXiv  Detail & Related papers  (2021-07-13T11:08:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.