Oscillators Are All You Need: Irregular Time Series Modelling via Damped Harmonic Oscillators with Closed-Form Solutions

• URL: http://arxiv.org/abs/2602.12139v1
• Date: Thu, 12 Feb 2026 16:27:09 GMT
• Title: Oscillators Are All You Need: Irregular Time Series Modelling via Damped Harmonic Oscillators with Closed-Form Solutions
• Abstract summary: Transformers excel at time series modelling through attention mechanisms that capture long-term temporal patterns.<n>They assume uniform time intervals and therefore struggle with irregular time series.<n>We show how to handle irregular time series by modelling hidden states as continuously evolving trajectories.
• Score: 3.8401108926879495
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformers excel at time series modelling through attention mechanisms that capture long-term temporal patterns. However, they assume uniform time intervals and therefore struggle with irregular time series. Neural Ordinary Differential Equations (NODEs) effectively handle irregular time series by modelling hidden states as continuously evolving trajectories. ContiFormers arxiv:2402.10635 combine NODEs with Transformers, but inherit the computational bottleneck of the former by using heavy numerical solvers. This bottleneck can be removed by using a closed-form solution for the given dynamical system - but this is known to be intractable in general! We obviate this by replacing NODEs with a novel linear damped harmonic oscillator analogy - which has a known closed-form solution. We model keys and values as damped, driven oscillators and expand the query in a sinusoidal basis up to a suitable number of modes. This analogy naturally captures the query-key coupling that is fundamental to any transformer architecture by modelling attention as a resonance phenomenon. Our closed-form solution eliminates the computational overhead of numerical ODE solvers while preserving expressivity. We prove that this oscillator-based parameterisation maintains the universal approximation property of continuous-time attention; specifically, any discrete attention matrix realisable by ContiFormer's continuous keys can be approximated arbitrarily well by our fixed oscillator modes. Our approach delivers both theoretical guarantees and scalability, achieving state-of-the-art performance on irregular time series benchmarks while being orders of magnitude faster.

Related papers

• Generative Modeling of Approximately Periodic Time Series by a Posterior-Weighted Gaussian Process [0.4460583138505672]
  We propose a generative model for approximately periodic time series.<n>Our approach decouples intra-repetition structure from inter-repetition variability through a two-stage construction.
  arXiv  Detail & Related papers  (2026-05-13T08:17:00Z)
• Beyond Similarity: Temporal Operator Attention for Time Series Analysis [5.084091872064936]
  A persistent paradox in time-series forecasting is that structurally simple and linear models often outperform high-capacity Transformers.<n>We propose $textbf Operator Attention (TOA)$, a framework that augments attention with explicit learnable sequence-space operators.<n>To make dense $N times operators practical, we introduce a high-variance dropout mechanism that stabilizes training and prevents trivial memorization.
  arXiv  Detail & Related papers  (2026-05-11T22:13:53Z)
• TimeSynth: A Framework for Uncovering Systematic Biases in Time Series Forecasting [0.9332987715848716]
  Time Synth is a structured framework that emulates key properties of real world time series.<n>We evaluate four model families Linear, Multi Layer Perceptrons (MLP), Convolutional Neural Networks (CNN), and Transformers.
  arXiv  Detail & Related papers  (2026-02-11T22:31:29Z)
• Parallel Complex Diffusion for Scalable Time Series Generation [50.01609741902786]
  PaCoDi is a spectral-native architecture that decouples generative modeling in the frequency domain.<n>We show that PaCoDi outperforms existing baselines in both generation quality and inference speed.
  arXiv  Detail & Related papers  (2026-02-10T14:31:53Z)
• Forecasting as Rendering: A 2D Gaussian Splatting Framework for Time Series Forecasting [79.37674445572462]
  Time series forecasting (TSF) remains a challenging problem due to the intricate entanglement of intraperiod-fluctuations and interperiod-trends.<n>Treating reshaped tensors as static images results in a topological mismatch.<n> relying on uniform fixed-size representations allocates modeling capacity inefficiently.<n>TimeGS is a novel framework that fundamentally shifts the forecasting paradigm from regression to 2D generative rendering.
  arXiv  Detail & Related papers  (2026-02-10T14:13:36Z)
• Modeling Time Series Dynamics with Fourier Ordinary Differential Equations [5.156484100374059]
  We introduce FODEs, an approach that embeds the dynamics in the Fourier domain.<n>FODEs uncover global patterns and periodic behaviors that remain elusive in the time domain.<n> Experiments on various time series datasets demonstrate that FODEs outperform existing methods in terms of both accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-10-05T10:27:45Z)
• High-order Magnus Expansion for Hamiltonian Simulation [2.6968321526169508]
  We develop a high-order quantum algorithm for quantum dynamics with time-dependent Hamiltonians.<n>The algorithm achieves cost scaling with the commutator structure in the high-precision regime and only logarithmically on the Hamiltonian's time variation.
  arXiv  Detail & Related papers  (2025-09-07T13:44:55Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies.<n>We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis.<n> Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling [30.12824131306359]
  Modeling continuous-time dynamics on irregular time series is critical to account for data evolution and correlations that occur continuously. Traditional methods including recurrent neural networks or Transformer models leverage inductive bias via powerful neural architectures to capture complex patterns. We propose ContiFormer that extends the relation modeling of vanilla Transformer to the continuous-time domain.
  arXiv  Detail & Related papers  (2024-02-16T12:34:38Z)
• Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs [50.25683648762602]
  We introduce Koopman VAE, a new generative framework that is based on a novel design for the model prior. Inspired by Koopman theory, we represent the latent conditional prior dynamics using a linear map. KoVAE outperforms state-of-the-art GAN and VAE methods across several challenging synthetic and real-world time series generation benchmarks.
  arXiv  Detail & Related papers  (2023-10-04T07:14:43Z)
• Diffeomorphic Transformations for Time Series Analysis: An Efficient Approach to Nonlinear Warping [0.0]
  The proliferation and ubiquity of temporal data across many disciplines has sparked interest for similarity, classification and clustering methods. Traditional distance measures such as the Euclidean are not well-suited due to the time-dependent nature of the data. This thesis proposes novel elastic alignment methods that use parametric & diffeomorphic warping transformations.
  arXiv  Detail & Related papers  (2023-09-25T10:51:47Z)
• Time Dependent Hamiltonian Simulation Using Discrete Clock Constructions [42.3779227963298]
  We provide a framework for encoding time dependent dynamics as time independent systems. First, we create a time dependent simulation algorithm based on performing qubitization on the augmented clock system. Second, we define a natural generalization of multiproduct formulas for time-ordered exponentials.
  arXiv  Detail & Related papers  (2022-03-21T21:29:22Z)
• Extracting Dynamical Models from Data [0.0]
  The problem of determining the underlying dynamics of a system when only given data of its state over time has challenged scientists for decades. In this paper, the approach of using machine learning to model the updates of the phase space variables is introduced.
  arXiv  Detail & Related papers  (2021-10-13T17:55:06Z)
• Policy Analysis using Synthetic Controls in Continuous-Time [101.35070661471124]
  Counterfactual estimation using synthetic controls is one of the most successful recent methodological developments in causal inference. We propose a continuous-time alternative that models the latent counterfactual path explicitly using the formalism of controlled differential equations.
  arXiv  Detail & Related papers  (2021-02-02T16:07:39Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• Fast and differentiable simulation of driven quantum systems [58.720142291102135]
  We introduce a semi-analytic method based on the Dyson expansion that allows us to time-evolve driven quantum systems much faster than standard numerical methods. We show results of the optimization of a two-qubit gate using transmon qubits in the circuit QED architecture.
  arXiv  Detail & Related papers  (2020-12-16T21:43:38Z)
• Liquid Time-constant Networks [117.57116214802504]
  We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations.
  arXiv  Detail & Related papers  (2020-06-08T09:53:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.