FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

• URL: http://arxiv.org/abs/2606.01306v1
• Date: Sun, 31 May 2026 15:51:22 GMT
• Title: FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting
• Abstract summary: FAiT is a Frequency-Aware inverted Transformer that rectifies the spectral bias internally through Inverted Attention.<n>DTFM synthesizes instance-conditioned weights to adaptively re-calibrate the energy of spectral sub-bands.<n>Experiments on widely used benchmarks demonstrate that FAiT consistently outperforms state-of-the-art Transformer-based and frequency-enhanced baselines.
• Score: 17.617858319449443
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Transformer-based architectures have established themselves as a dominant paradigm in Multivariate Time Series Forecasting (MTSF), their core self-attention mechanism inherently functions as a low-pass filter, systematically smoothing out high-frequency signals vital for sharp local changes. Recent advancements have increasingly incorporated frequency-domain operations to address this bias, however, most existing designs rely on fixed spectral bases and apply sequence-wise (uniform) modulation, implicitly assuming a time-invariant frequency response. This overlooks a key property of real-world series that their spectral characteristics often evolve over time, making uniform modulation insufficient for capturing fine-grained temporal dynamics. To tackle these limitations, we propose FAiT, a Frequency-Aware inverted Transformer. Specifically, FAiT rectifies the spectral bias internally through Inverted Attention, which interprets the attention map as a learnable low-pass operator and constructs a dedicated complementary high-pass branch by inverting the attention matrix to recover attenuated transient signals. Furthermore, FAiT introduces Dynamic Temporal-Frequency Modulation (DTFM), which synthesizes instance-conditioned weights to adaptively re-calibrate the energy of spectral sub-bands, enabling fine-grained control over evolving multi-scale patterns. Extensive experiments on widely used benchmarks demonstrate that FAiT consistently outperforms state-of-the-art Transformer-based and frequency-enhanced baselines, while maintaining computational efficiency.

Related papers

• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• FADTI: Fourier and Attention Driven Diffusion for Multivariate Time Series Imputation [26.523421387059035]
  We propose FADTI, a diffusion-based framework that injects frequency-informed feature modulation via a learnable Fourier Bias Projection (FBP) module.<n>Experiments on multiple benchmarks, including a newly introduced biological time series dataset, show that FADTI consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-12-17T06:16:31Z)
• A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
  Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
  arXiv  Detail & Related papers  (2025-10-11T09:59:25Z)
• Frequency-Constrained Learning for Long-Term Forecasting [15.31488551912888]
  Real-world time series exhibit strong periodic structures arising from physical laws, human routines, or seasonal cycles.<n>Modern deep forecasting models often fail to capture these recurring patterns due to spectral bias and a lack of frequency-aware inductive priors.<n>We propose a simple yet effective method that enhances long-term forecasting by explicitly modeling periodicity.
  arXiv  Detail & Related papers  (2025-08-02T22:12:15Z)
• Multi-Order Wavelet Derivative Transform for Deep Time Series Forecasting [40.57415617022347]
  We introduce the multi-order Wavelet Derivative Transform (WDT), enabling the extraction of time-aware patterns spanning both the overall trend and subtle fluctuations.<n> Practically, we embed the WDT into a multi-branch framework named WaveTS, which decomposes the input series into multi-scale time-frequency coefficients, refines them via linear layers, and reconstructs them into the time domain via the inverse WDT.<n>Experiments on ten benchmark datasets demonstrate that WaveTS achieves state-of-the-art forecasting accuracy while retaining high computational efficiency.
  arXiv  Detail & Related papers  (2025-05-17T01:24:09Z)
• FAD: Frequency Adaptation and Diversion for Cross-domain Few-shot Learning [35.40065954148091]
  Cross-domain few-shot learning requires models to generalize from limited labeled samples under significant distribution shifts.<n>We introduce Frequency Adaptation and Diversion (FAD), a frequency-aware framework that explicitly models and modulates spectral components.<n>FAD consistently outperforms state-of-the-art methods on both seen and unseen domains.
  arXiv  Detail & Related papers  (2025-05-13T08:48:06Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies.<n>We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis.<n> Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting [17.738942892605234]
  This paper presents textbfFreEformer, a simple yet effective model that leverages a textbfFrequency textbfEnhanced Transtextbfformer.<n>Experiments demonstrate that FreEformer consistently outperforms state-of-the-art models on eighteen real-world benchmarks.
  arXiv  Detail & Related papers  (2025-01-23T08:53:45Z)
• Tuning Frequency Bias of State Space Models [48.60241978021799]
  State space models (SSMs) leverage linear, time-invariant (LTI) systems to learn sequences with long-range dependencies. We find that SSMs exhibit an implicit bias toward capturing low-frequency components more effectively than high-frequency ones.
  arXiv  Detail & Related papers  (2024-10-02T21:04:22Z)
• PRformer: Pyramidal Recurrent Transformer for Multivariate Time Series Forecasting [82.03373838627606]
  Self-attention mechanism in Transformer architecture requires positional embeddings to encode temporal order in time series prediction. We argue that this reliance on positional embeddings restricts the Transformer's ability to effectively represent temporal sequences. We present a model integrating PRE with a standard Transformer encoder, demonstrating state-of-the-art performance on various real-world datasets.
  arXiv  Detail & Related papers  (2024-08-20T01:56:07Z)
• WFTNet: Exploiting Global and Local Periodicity in Long-term Time Series Forecasting [61.64303388738395]
  We propose a Wavelet-Fourier Transform Network (WFTNet) for long-term time series forecasting. Tests on various time series datasets show WFTNet consistently outperforms other state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-09-20T13:44:18Z)
• Transform Once: Efficient Operator Learning in Frequency Domain [69.74509540521397]
  We study deep neural networks designed to harness the structure in frequency domain for efficient learning of long-range correlations in space or time. This work introduces a blueprint for frequency domain learning through a single transform: transform once (T1)
  arXiv  Detail & Related papers  (2022-11-26T01:56:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.