Learning Robust Spectral Dynamics for Temporal Domain Generalization

• URL: http://arxiv.org/abs/2505.12585v1
• Date: Mon, 19 May 2025 00:38:18 GMT
• Title: Learning Robust Spectral Dynamics for Temporal Domain Generalization
• Authors: En Yu, Jie Lu, Xiaoyu Yang, Guangquan Zhang, Zhen Fang,
• Abstract summary: Temporal Domain Generalization seeks to enable model generalization across evolving domains.<n>We introduce FreKoo, which tackles these challenges via a novel frequency-domain analysis of parameter trajectories.<n>FreKoo excels in real-world streaming scenarios with complex drifts and uncertainties.
• Score: 35.98513351187109
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern machine learning models struggle to maintain performance in dynamic environments where temporal distribution shifts, \emph{i.e., concept drift}, are prevalent. Temporal Domain Generalization (TDG) seeks to enable model generalization across evolving domains, yet existing approaches typically assume smooth incremental changes, struggling with complex real-world drifts involving long-term structure (incremental evolution/periodicity) and local uncertainties. To overcome these limitations, we introduce FreKoo, which tackles these challenges via a novel frequency-domain analysis of parameter trajectories. It leverages the Fourier transform to disentangle parameter evolution into distinct spectral bands. Specifically, low-frequency component with dominant dynamics are learned and extrapolated using the Koopman operator, robustly capturing diverse drift patterns including both incremental and periodicity. Simultaneously, potentially disruptive high-frequency variations are smoothed via targeted temporal regularization, preventing overfitting to transient noise and domain uncertainties. In addition, this dual spectral strategy is rigorously grounded through theoretical analysis, providing stability guarantees for the Koopman prediction, a principled Bayesian justification for the high-frequency regularization, and culminating in a multiscale generalization bound connecting spectral dynamics to improved generalization. Extensive experiments demonstrate FreKoo's significant superiority over SOTA TDG approaches, particularly excelling in real-world streaming scenarios with complex drifts and uncertainties.

Related papers

• Frequency-Constrained Learning for Long-Term Forecasting [15.31488551912888]
  Real-world time series exhibit strong periodic structures arising from physical laws, human routines, or seasonal cycles.<n>Modern deep forecasting models often fail to capture these recurring patterns due to spectral bias and a lack of frequency-aware inductive priors.<n>We propose a simple yet effective method that enhances long-term forecasting by explicitly modeling periodicity.
  arXiv  Detail & Related papers  (2025-08-02T22:12:15Z)
• Transformer with Koopman-Enhanced Graph Convolutional Network for Spatiotemporal Dynamics Forecasting [12.301897782320967]
  TK-GCN is a two-stage framework that integrates geometry-aware spatial encoding with long-range temporal modeling.<n>We show that TK-GCN consistently delivers superior predictive accuracy across a range of forecast horizons.
  arXiv  Detail & Related papers  (2025-07-05T01:26:03Z)
• Multivariate Long-term Time Series Forecasting with Fourier Neural Filter [55.09326865401653]
  We introduce FNF as the backbone and DBD as architecture to provide excellent learning capabilities and optimal learning pathways for spatial-temporal modeling.<n>We show that FNF unifies local time-domain and global frequency-domain information processing within a single backbone that extends naturally to spatial modeling.
  arXiv  Detail & Related papers  (2025-06-10T18:40:20Z)
• Turb-L1: Achieving Long-term Turbulence Tracing By Tackling Spectral Bias [43.0262112921538]
  We propose Turb-L1, an innovative turbulence prediction method.<n>It accurately captures cross-scale interactions and preserves the fidelity of high-frequency dynamics.<n>In long-term predictions, it reduces Mean Squared Error (MSE) by $80.3%$ and increases Structural Similarity (SSIM) by over $9times$ compared to the SOTA baseline.
  arXiv  Detail & Related papers  (2025-05-25T08:38:55Z)
• BEAT: Balanced Frequency Adaptive Tuning for Long-Term Time-Series Forecasting [46.922741972636025]
  Time-series forecasting is crucial for numerous real-world applications including weather prediction and financial market modeling.<n>We propose BEAT (Balanced frEquency Adaptive Tuning), a novel framework that monitors the training status for each frequency and adaptively adjusts their gradient updates.<n>BEAT consistently outperforms state-of-the-art approaches in experiments on seven real-world datasets.
  arXiv  Detail & Related papers  (2025-01-31T11:52:35Z)
• Graph Fourier Neural ODEs: Modeling Spatial-temporal Multi-scales in Molecular Dynamics [38.53044197103943]
  GF-NODE integrates a graph Fourier transform for spatial frequency decomposition with a Neural ODE framework for continuous-time evolution.<n>We show that GF-NODE achieves state-of-the-art accuracy while preserving essential geometrical features over extended simulations.<n>These findings highlight the promise of bridging spectral decomposition with continuous-time modeling to improve the robustness and predictive power of MD simulations.
  arXiv  Detail & Related papers  (2024-11-03T15:10:48Z)
• Continuous Temporal Domain Generalization [17.529690717937267]
  Temporal Domain Generalization (TDG) addresses the challenge of training predictive models under temporally varying data distributions. This work formalizes the concept of Continuous Temporal Domain Generalization (CTDG), where domain data are derived from continuous times and are collected at arbitrary times.
  arXiv  Detail & Related papers  (2024-05-25T05:52:04Z)
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective [63.60312929416228]
  textbftextitAttraos incorporates chaos theory into long-term time series forecasting. We show that Attraos outperforms various LTSF methods on mainstream datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.
  arXiv  Detail & Related papers  (2024-02-18T05:35:01Z)
• Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction [49.66486092259376]
  The mean-field Langevin dynamics (MFLD) is a nonlinear generalization of the Langevin dynamics that incorporates a distribution-dependent drift. Recent works have shown that MFLD globally minimizes an entropy-regularized convex functional in the space of measures. We provide a framework to prove a uniform-in-time propagation of chaos for MFLD that takes into account the errors due to finite-particle approximation, time-discretization, and gradient approximation.
  arXiv  Detail & Related papers  (2023-06-12T16:28:11Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)
• On the Generalization of Stochastic Gradient Descent with Momentum [84.54924994010703]
  momentum-based accelerated variants of gradient descent (SGD) are widely used when training machine learning models. We first show that there exists a convex loss function for which the stability gap for multiple epochs of SGD with standard heavy-ball momentum (SGDM) becomes unbounded. For smooth Lipschitz loss functions, we analyze a modified momentum-based update rule, i.e., SGD with early momentum (SGDEM) under a broad range of step-sizes.
  arXiv  Detail & Related papers  (2018-09-12T17:02:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.