Continuous-Time Linear Positional Embedding for Irregular Time Series Forecasting

• URL: http://arxiv.org/abs/2409.20092v1
• Date: Mon, 30 Sep 2024 08:46:18 GMT
• Title: Continuous-Time Linear Positional Embedding for Irregular Time Series Forecasting
• Abstract summary: We propose CTLPE, a method learning a continuous linear function for encoding temporal information. CTLPE outperforms existing techniques across various irregularly-sampled time series datasets.
• Score: 14.038703353884033
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Irregularly sampled time series forecasting, characterized by non-uniform intervals, is prevalent in practical applications. However, previous research have been focused on regular time series forecasting, typically relying on transformer architectures. To extend transformers to handle irregular time series, we tackle the positional embedding which represents the temporal information of the data. We propose CTLPE, a method learning a continuous linear function for encoding temporal information. The two challenges of irregular time series, inconsistent observation patterns and irregular time gaps, are solved by learning a continuous-time function and concise representation of position. Additionally, the linear continuous function is empirically shown superior to other continuous functions by learning a neural controlled differential equation-based positional embedding, and theoretically supported with properties of ideal positional embedding. CTLPE outperforms existing techniques across various irregularly-sampled time series datasets, showcasing its enhanced efficacy.

Related papers

• Enhancing Irregular Time Series Forecasting with Continuous-Time Modeling Framework [15.900704071560698]
  WrapFlow is a continuous-time modeling framework for irregular time series forecasting.<n>We develop a training paradigm for Residual Flow Matching, which learns conditional residual vector fields around base predictions.<n>Experiments on multiple real-world datasets demonstrate that WrapFlow achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-07-30T11:18:18Z)
• Latent Laplace Diffusion for Irregular Multivariate Time Series [16.474933482743513]
  We present Latent Laplace Diffusion, a generative framework that models the target as a low-dimensional latent trajectory.<n>LLapDiff enables horizon-wide generation without step-by-step integration over physical time.<n>We also link continuous dynamics to irregular observations through renewal-averaging analysis.
  arXiv  Detail & Related papers  (2026-05-19T13:04:53Z)
• TimeAPN: Adaptive Amplitude-Phase Non-Stationarity Normalization for Time Series Forecasting [67.91113180885601]
  TimeAPN explicitly models and predicts non-stationary factors from both the time and frequency domains.<n>TimeAPN consistently improves long-term forecasting accuracy across multiple prediction horizons.<n>It outperforms state-of-the-art reversible normalization methods.
  arXiv  Detail & Related papers  (2026-03-18T07:21:24Z)
• TimeRadar: A Domain-Rotatable Foundation Model for Time Series Anomaly Detection [41.0416229540611]
  Time series foundation models (TSFMs) primarily focus on learning prevalent and regular patterns within a predefined time or frequency domain.<n>TimeRadar is an innovative TSFM built in a fractional time-frequency domain to support generalist TSAD across diverse unseen datasets.
  arXiv  Detail & Related papers  (2026-02-22T06:53:45Z)
• Forecasting as Rendering: A 2D Gaussian Splatting Framework for Time Series Forecasting [79.37674445572462]
  Time series forecasting (TSF) remains a challenging problem due to the intricate entanglement of intraperiod-fluctuations and interperiod-trends.<n>Treating reshaped tensors as static images results in a topological mismatch.<n> relying on uniform fixed-size representations allocates modeling capacity inefficiently.<n>TimeGS is a novel framework that fundamentally shifts the forecasting paradigm from regression to 2D generative rendering.
  arXiv  Detail & Related papers  (2026-02-10T14:13:36Z)
• A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
  Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
  arXiv  Detail & Related papers  (2025-10-11T09:59:25Z)
• Modeling Time Series Dynamics with Fourier Ordinary Differential Equations [5.156484100374059]
  We introduce FODEs, an approach that embeds the dynamics in the Fourier domain.<n>FODEs uncover global patterns and periodic behaviors that remain elusive in the time domain.<n> Experiments on various time series datasets demonstrate that FODEs outperform existing methods in terms of both accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-10-05T10:27:45Z)
• Multivariate Long-term Time Series Forecasting with Fourier Neural Filter [55.09326865401653]
  We introduce FNF as the backbone and DBD as architecture to provide excellent learning capabilities and optimal learning pathways for spatial-temporal modeling.<n>We show that FNF unifies local time-domain and global frequency-domain information processing within a single backbone that extends naturally to spatial modeling.
  arXiv  Detail & Related papers  (2025-06-10T18:40:20Z)
• FreCT: Frequency-augmented Convolutional Transformer for Robust Time Series Anomaly Detection [8.779286654383796]
  Time series anomaly detection is critical for system monitoring and risk identification.<n>We propose a novel Frequency-augmented Convolutional Transformer (FreCT)<n>FreCT utilizes patch operations to generate contrastive views and employs an improved Transformer architecture integrated with a convolution module.
  arXiv  Detail & Related papers  (2025-05-02T00:56:24Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies. We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis. Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a generative Transformer for unified time series forecasting. Timer-XL achieves state-of-the-art performance across challenging forecasting benchmarks through a unified approach.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• PRformer: Pyramidal Recurrent Transformer for Multivariate Time Series Forecasting [82.03373838627606]
  Self-attention mechanism in Transformer architecture requires positional embeddings to encode temporal order in time series prediction. We argue that this reliance on positional embeddings restricts the Transformer's ability to effectively represent temporal sequences. We present a model integrating PRE with a standard Transformer encoder, demonstrating state-of-the-art performance on various real-world datasets.
  arXiv  Detail & Related papers  (2024-08-20T01:56:07Z)
• Adaptive Convolutional Forecasting Network Based on Time Series Feature-Driven [9.133955922897371]
  Time series data in real-world scenarios contain a substantial amount of nonlinear information. We introduce multi-resolution convolution and deformable convolution operations. We propose ACNet, an adaptive convolutional network designed to effectively model the local and global temporal dependencies.
  arXiv  Detail & Related papers  (2024-05-20T14:05:35Z)
• TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling [67.02157180089573]
  Time series pre-training has recently garnered wide attention for its potential to reduce labeling expenses and benefit various downstream tasks. This paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
  arXiv  Detail & Related papers  (2024-02-04T13:10:51Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Compatible Transformer for Irregularly Sampled Multivariate Time Series [75.79309862085303]
  We propose a transformer-based encoder to achieve comprehensive temporal-interaction feature learning for each individual sample. We conduct extensive experiments on 3 real-world datasets and validate that the proposed CoFormer significantly and consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2023-10-17T06:29:09Z)
• Probabilistic Learning of Multivariate Time Series with Temporal Irregularity [21.361823581838355]
  Real-world time series often suffer from temporal irregularities, including nonuniform intervals and misaligned variables. We propose an end-to-end framework that models temporal irregularities while capturing the joint distribution of variables at arbitrary continuous-time points.
  arXiv  Detail & Related papers  (2023-06-15T14:08:48Z)
• DRAformer: Differentially Reconstructed Attention Transformer for Time-Series Forecasting [7.805077630467324]
  Time-series forecasting plays an important role in many real-world scenarios, such as equipment life cycle forecasting, weather forecasting, and traffic flow forecasting. It can be observed from recent research that a variety of transformer-based models have shown remarkable results in time-series forecasting. However, there are still some issues that limit the ability of transformer-based models on time-series forecasting tasks.
  arXiv  Detail & Related papers  (2022-06-11T10:34:29Z)
• TACTiS: Transformer-Attentional Copulas for Time Series [76.71406465526454]
  estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance. We propose a versatile method that estimates joint distributions using an attention-based decoder. We show that our model produces state-of-the-art predictions on several real-world datasets.
  arXiv  Detail & Related papers  (2022-02-07T21:37:29Z)
• AutoFITS: Automatic Feature Engineering for Irregular Time Series [0.44198435146063353]
  In irregular time series, the time at which each observation is collected may be helpful to summarise the dynamics of the data and improve forecasting performance. We develop a novel automatic feature engineering framework, which focuses on extracting information from this point of view when each instance is collected. We study how valuable this information is by integrating it in a time series forecasting workflow and investigate how it compares to or complements state-of-the-art methods for regular time series forecasting.
  arXiv  Detail & Related papers  (2021-12-29T19:42:48Z)
• Time Series is a Special Sequence: Forecasting with Sample Convolution and Interaction [9.449017120452675]
  Time series is a special type of sequence data, a set of observations collected at even intervals of time and ordered chronologically. Existing deep learning techniques use generic sequence models for time series analysis, which ignore some of its unique properties. We propose a novel neural network architecture and apply it for the time series forecasting problem, wherein we conduct sample convolution and interaction at multiple resolutions for temporal modeling.
  arXiv  Detail & Related papers  (2021-06-17T08:15:04Z)
• Interpretable Feature Construction for Time Series Extrinsic Regression [0.028675177318965035]
  In some application domains, it occurs that the target variable is numerical and the problem is known as time series extrinsic regression (TSER) We suggest an extension of a Bayesian method for robust and interpretable feature construction and selection in the context of TSER. Our approach exploits a relational way to tackle with TSER: (i), we build various and simple representations of the time series which are stored in a relational data scheme, then, (ii), a propositionalisation technique is applied to build interpretable features from secondary tables to "flatten" the data.
  arXiv  Detail & Related papers  (2021-03-15T08:12:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.