Enhancing Representation Learning for Periodic Time Series with Floss: A Frequency Domain Regularization Approach

• URL: http://arxiv.org/abs/2308.01011v4
• Date: Sat, 2 Sep 2023 01:09:14 GMT
• Title: Enhancing Representation Learning for Periodic Time Series with Floss: A Frequency Domain Regularization Approach
• Authors: Chunwei Yang, Xiaoxu Chen, Lijun Sun, Hongyu Yang, Yuankai Wu
• Abstract summary: We propose an unsupervised method called Floss that automatically regularizes learned representations in the frequency domain. We conduct extensive experiments on common time series classification, forecasting, and anomaly detection tasks to demonstrate the effectiveness of Floss.
• Score: 26.92614573306619
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time series analysis is a fundamental task in various application domains, and deep learning approaches have demonstrated remarkable performance in this area. However, many real-world time series data exhibit significant periodic or quasi-periodic dynamics that are often not adequately captured by existing deep learning-based solutions. This results in an incomplete representation of the underlying dynamic behaviors of interest. To address this gap, we propose an unsupervised method called Floss that automatically regularizes learned representations in the frequency domain. The Floss method first automatically detects major periodicities from the time series. It then employs periodic shift and spectral density similarity measures to learn meaningful representations with periodic consistency. In addition, Floss can be easily incorporated into both supervised, semi-supervised, and unsupervised learning frameworks. We conduct extensive experiments on common time series classification, forecasting, and anomaly detection tasks to demonstrate the effectiveness of Floss. We incorporate Floss into several representative deep learning solutions to justify our design choices and demonstrate that it is capable of automatically discovering periodic dynamics and improving state-of-the-art deep learning models.

Related papers

• Dynamic Contrastive Learning for Time Series Representation [6.086030037869592]
  We propose DynaCL, an unsupervised contrastive representation learning framework for time series. We demonstrate that DynaCL embeds instances from time series into semantically meaningful clusters. Our findings also reveal that high scores on unsupervised clustering metrics do not guarantee that the representations are useful in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-20T15:20:24Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• An Unsupervised Approach for Periodic Source Detection in Time Series [22.053675805215686]
  Detection of periodic patterns of interest within noisy time series data plays a critical role in various tasks. Existing learning techniques often rely on labels or clean versions of signals for detecting the periodicity. We propose a novel method to detect the periodicity in time series without the need for any labels.
  arXiv  Detail & Related papers  (2024-06-01T22:23:51Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization [58.704753031608625]
  Time series is one of the most challenging modalities in machine learning research. OOD detection and generalization on time series tend to suffer due to its non-stationary property. We propose DIVERSIFY, a framework for OOD detection and generalization on dynamic distributions of time series.
  arXiv  Detail & Related papers  (2023-08-04T12:27:11Z)
• PRNet: A Periodic Residual Learning Network for Crowd Flow Forecasting [8.50942649992681]
  We devise a novel periodic residual learning network (PRNet) for better modeling the periodicity in crowd flow data. PRNet frames the crowd flow forecasting as a periodic residual learning problem by modeling the deviation between the input (the previous time period) and the output (the future time period) Experimental results on two real-world datasets demonstrate that PRNet outperforms the state-of-the-art methods in terms of both accuracy and robustness.
  arXiv  Detail & Related papers  (2021-12-08T12:04:27Z)
• Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy [68.86835407617778]
  Anomaly Transformer achieves state-of-the-art performance on six unsupervised time series anomaly detection benchmarks. Anomaly Transformer achieves state-of-the-art performance on six unsupervised time series anomaly detection benchmarks.
  arXiv  Detail & Related papers  (2021-10-06T10:33:55Z)
• Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding [8.45908939323268]
  We propose a self-supervised framework for learning generalizable representations for non-stationary time series. Our motivation stems from the medical field, where the ability to model the dynamic nature of time series data is especially valuable.
  arXiv  Detail & Related papers  (2021-06-01T19:53:24Z)
• Contrastive learning of strong-mixing continuous-time stochastic processes [53.82893653745542]
  Contrastive learning is a family of self-supervised methods where a model is trained to solve a classification task constructed from unlabeled data. We show that a properly constructed contrastive learning task can be used to estimate the transition kernel for small-to-mid-range intervals in the diffusion case.
  arXiv  Detail & Related papers  (2021-03-03T23:06:47Z)
• Multi-Time Attention Networks for Irregularly Sampled Time Series [18.224344440110862]
  Irregular sampling occurs in many time series modeling applications. We propose a new deep learning framework for this setting that we call Multi-Time Attention Networks. Our results show that our approach performs as well or better than a range of baseline and recently proposed models.
  arXiv  Detail & Related papers  (2021-01-25T18:57:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.