FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks

• URL: http://arxiv.org/abs/2510.25800v1
• Date: Wed, 29 Oct 2025 03:22:51 GMT
• Title: FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks
• Authors: Jialong Sun, Xinpeng Ling, Jiaxuan Zou, Jiawen Kang, Kejia Zhang,
• Abstract summary: We show that models tend to fit low-frequency signals before high-frequency ones.<n>We propose the FreLE algorithm, which enhances model generalization through both explicit and implicit frequency regularization.
• Score: 5.215187965365735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The inherent autocorrelation of time series data presents an ongoing challenge to multivariate time series prediction. Recently, a widely adopted approach has been the incorporation of frequency domain information to assist in long-term prediction tasks. Many researchers have independently observed the spectral bias phenomenon in neural networks, where models tend to fit low-frequency signals before high-frequency ones. However, these observations have often been attributed to the specific architectures designed by the researchers, rather than recognizing the phenomenon as a universal characteristic across models. To unify the understanding of the spectral bias phenomenon in long-term time series prediction, we conducted extensive empirical experiments to measure spectral bias in existing mainstream models. Our findings reveal that virtually all models exhibit this phenomenon. To mitigate the impact of spectral bias, we propose the FreLE (Frequency Loss Enhancement) algorithm, which enhances model generalization through both explicit and implicit frequency regularization. This is a plug-and-play model loss function unit. A large number of experiments have proven the superior performance of FreLE. Code is available at https://github.com/Chenxing-Xuan/FreLE.

Related papers

• A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
  Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
  arXiv  Detail & Related papers  (2025-10-11T09:59:25Z)
• Frequency-Constrained Learning for Long-Term Forecasting [15.31488551912888]
  Real-world time series exhibit strong periodic structures arising from physical laws, human routines, or seasonal cycles.<n>Modern deep forecasting models often fail to capture these recurring patterns due to spectral bias and a lack of frequency-aware inductive priors.<n>We propose a simple yet effective method that enhances long-term forecasting by explicitly modeling periodicity.
  arXiv  Detail & Related papers  (2025-08-02T22:12:15Z)
• LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators [20.77877474840923]
  High-frequency information is a critical challenge in machine learning.<n>Deep neural nets exhibit the so-called spectral bias toward learning low-frequency components.<n>We propose a novel frequency-sensitive loss term based on radially binned spectral errors.
  arXiv  Detail & Related papers  (2025-04-05T19:35:04Z)
• General Time-series Model for Universal Knowledge Representation of Multivariate Time-Series data [61.163542597764796]
  We show that time series with different time granularities (or corresponding frequency resolutions) exhibit distinct joint distributions in the frequency domain.<n>A novel Fourier knowledge attention mechanism is proposed to enable learning time-aware representations from both the temporal and frequency domains.<n>An autoregressive blank infilling pre-training framework is incorporated to time series analysis for the first time, leading to a generative tasks agnostic pre-training strategy.
  arXiv  Detail & Related papers  (2025-02-05T15:20:04Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• Fredformer: Frequency Debiased Transformer for Time Series Forecasting [8.356290446630373]
  The Transformer model has shown leading performance in time series forecasting. It tends to learn low-frequency features in the data and overlook high-frequency features, showing a frequency bias. We propose Fredformer, a framework designed to mitigate frequency bias by learning features equally across different frequency bands.
  arXiv  Detail & Related papers  (2024-06-13T11:29:21Z)
• Time Series Diffusion in the Frequency Domain [54.60573052311487]
  We analyze whether representing time series in the frequency domain is a useful inductive bias for score-based diffusion models. We show that a dual diffusion process occurs in the frequency domain with an important nuance. We show how to adapt the denoising score matching approach to implement diffusion models in the frequency domain.
  arXiv  Detail & Related papers  (2024-02-08T18:59:05Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• Spectral Bias in Practice: The Role of Function Frequency in Generalization [10.7218588164913]
  We propose methodologies for measuring spectral bias in modern image classification networks. We find that networks that generalize well strike a balance between having enough complexity to fit the data while being simple enough to avoid overfitting. Our work enables measuring and ultimately controlling the spectral behavior of neural networks used for image classification.
  arXiv  Detail & Related papers  (2021-10-06T00:16:10Z)
• Deep Autoregressive Models with Spectral Attention [74.08846528440024]
  We propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise.
  arXiv  Detail & Related papers  (2021-07-13T11:08:47Z)
• Bayesian Reconstruction of Fourier Pairs [21.104218472462907]
  General literature fails to cater for missing observations and noise-corrupted data. Our aim is to address the lack of a principled treatment of data acquired indistinctly in the temporal and frequency domains. We show that the proposed model is able to perform joint time and frequency reconstruction of real-world audio, healthcare and astronomy signals.
  arXiv  Detail & Related papers  (2020-11-09T17:30:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.