Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion

• URL: http://arxiv.org/abs/2202.04770v1
• Date: Tue, 8 Feb 2022 14:04:08 GMT
• Title: Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion
• Authors: Ling Yang, Shenda Hong, Luxia Zhang
• Abstract summary: We propose a unified framework, namely Bilinear Temporal-Spectral Fusion (BTSF) Specifically, we utilize the instance-level augmentation with a simple dropout on the entire time series for maximally capturing long-term dependencies. We devise a novel iterative bilinear temporal-spectral fusion to explicitly encode the affinities of abundant time-frequency pairs.
• Score: 6.154427471704388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised/self-supervised time series representation learning is a challenging problem because of its complex dynamics and sparse annotations. Existing works mainly adopt the framework of contrastive learning with the time-based augmentation techniques to sample positives and negatives for contrastive training. Nevertheless, they mostly use segment-level augmentation derived from time slicing, which may bring about sampling bias and incorrect optimization with false negatives due to the loss of global context. Besides, they all pay no attention to incorporate the spectral information in feature representation. In this paper, we propose a unified framework, namely Bilinear Temporal-Spectral Fusion (BTSF). Specifically, we firstly utilize the instance-level augmentation with a simple dropout on the entire time series for maximally capturing long-term dependencies. We devise a novel iterative bilinear temporal-spectral fusion to explicitly encode the affinities of abundant time-frequency pairs, and iteratively refines representations in a fusion-and-squeeze manner with Spectrum-to-Time (S2T) and Time-to-Spectrum (T2S) Aggregation modules. We firstly conducts downstream evaluations on three major tasks for time series including classification, forecasting and anomaly detection. Experimental results shows that our BTSF consistently significantly outperforms the state-of-the-art methods.

Related papers

• Diffeomorphic Temporal Alignment Nets for Time-series Joint Alignment and Averaging [8.14908648005543]
  In time-series analysis, nonlinear temporal misalignment remains a pivotal challenge that forestalls even simple averaging. DTAN predicts and applies diffeomorphic transformations in an input-dependent manner, thus facilitating the joint alignment (JA) and averaging of time-series ensembles. We extend our framework to incorporate multi-task learning (MT-DTAN), enabling simultaneous timeseries alignment and classification.
  arXiv  Detail & Related papers  (2025-02-10T15:55:08Z)
• General Time-series Model for Universal Knowledge Representation of Multivariate Time-Series data [61.163542597764796]
  We show that time series with different time granularities (or corresponding frequency resolutions) exhibit distinct joint distributions in the frequency domain. A novel Fourier knowledge attention mechanism is proposed to enable learning time-aware representations from both the temporal and frequency domains. An autoregressive blank infilling pre-training framework is incorporated to time series analysis for the first time, leading to a generative tasks agnostic pre-training strategy.
  arXiv  Detail & Related papers  (2025-02-05T15:20:04Z)
• Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning [0.38366697175402226]
  This paper introduces Frequency-masked Embedding Inference (FEI), a novel non-contrastive method that completely eliminates the need for positive and negative samples. FEI significantly outperforms existing contrastive-based methods in terms of generalization. This study provides new insights into self-supervised representation learning for time series.
  arXiv  Detail & Related papers  (2024-12-30T08:12:17Z)
• MuSiCNet: A Gradual Coarse-to-Fine Framework for Irregularly Sampled Multivariate Time Series Analysis [45.34420094525063]
  We introduce a novel perspective that irregularity is essentially relative in some senses. MuSiCNet is an ISMTS analysis framework that competitive with SOTA in three mainstream tasks consistently.
  arXiv  Detail & Related papers  (2024-12-02T02:50:01Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling [67.02157180089573]
  Time series pre-training has recently garnered wide attention for its potential to reduce labeling expenses and benefit various downstream tasks. This paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
  arXiv  Detail & Related papers  (2024-02-04T13:10:51Z)
• Explaining Time Series via Contrastive and Locally Sparse Perturbations [45.055327583283315]
  ContraLSP is a sparse model that introduces counterfactual samples to build uninformative perturbations but keeps distribution using contrastive learning. Empirical studies on both synthetic and real-world datasets show that ContraLSP outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2024-01-16T18:27:37Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• Imputing Missing Observations with Time Sliced Synthetic Minority Oversampling Technique [0.3973560285628012]
  We present a simple yet novel time series imputation technique with the goal of constructing an irregular time series that is uniform across every sample in a data set. We fix a grid defined by the midpoints of non-overlapping bins (dubbed "slices") of observation times and ensure that each sample has values for all of the features at that given time. This allows one to both impute fully missing observations to allow uniform time series classification across the entire data and, in special cases, to impute individually missing features.
  arXiv  Detail & Related papers  (2022-01-14T19:23:24Z)
• Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy [68.86835407617778]
  Anomaly Transformer achieves state-of-the-art performance on six unsupervised time series anomaly detection benchmarks. Anomaly Transformer achieves state-of-the-art performance on six unsupervised time series anomaly detection benchmarks.
  arXiv  Detail & Related papers  (2021-10-06T10:33:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.