Training Robust Deep Models for Time-Series Domain: Novel Algorithms and Theoretical Analysis

• URL: http://arxiv.org/abs/2207.04305v2
• Date: Wed, 13 Jul 2022 03:14:10 GMT
• Title: Training Robust Deep Models for Time-Series Domain: Novel Algorithms and Theoretical Analysis
• Authors: Taha Belkhouja, Yan Yan, Janardhan Rao Doppa
• Abstract summary: We propose a novel framework referred as RObust Training for Time-Series (RO-TS) to create robust DNNs for time-series classification tasks. We show the generality and advantages of our formulation using the summation structure over time-series alignments. Our experiments on real-world benchmarks demonstrate that RO-TS creates more robust DNNs when compared to adversarial training.
• Score: 32.45387153404849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the success of deep neural networks (DNNs) for real-world applications over time-series data such as mobile health, little is known about how to train robust DNNs for time-series domain due to its unique characteristics compared to images and text data. In this paper, we propose a novel algorithmic framework referred as RObust Training for Time-Series (RO-TS) to create robust DNNs for time-series classification tasks. Specifically, we formulate a min-max optimization problem over the model parameters by explicitly reasoning about the robustness criteria in terms of additive perturbations to time-series inputs measured by the global alignment kernel (GAK) based distance. We also show the generality and advantages of our formulation using the summation structure over time-series alignments by relating both GAK and dynamic time warping (DTW). This problem is an instance of a family of compositional min-max optimization problems, which are challenging and open with unclear theoretical guarantee. We propose a principled stochastic compositional alternating gradient descent ascent (SCAGDA) algorithm for this family of optimization problems. Unlike traditional methods for time-series that require approximate computation of distance measures, SCAGDA approximates the GAK based distance on-the-fly using a moving average approach. We theoretically analyze the convergence rate of SCAGDA and provide strong theoretical support for the estimation of GAK based distance. Our experiments on real-world benchmarks demonstrate that RO-TS creates more robust DNNs when compared to adversarial training using prior methods that rely on data augmentation or new definitions of loss functions. We also demonstrate the importance of GAK for time-series data over the Euclidean distance. The source code of RO-TS algorithms is available at https://github.com/tahabelkhouja/Robust-Training-for-Time-Series

Related papers

• Long Range Propagation on Continuous-Time Dynamic Graphs [18.5534584418248]
  Continuous-Time Graph Anti-Symmetric Network (CTAN) is designed for efficient propagation of information. We show how CTAN's empirical performance on synthetic long-range benchmarks and real-world benchmarks is superior to other methods.
  arXiv  Detail & Related papers  (2024-06-04T19:42:19Z)
• Concrete Dense Network for Long-Sequence Time Series Clustering [4.307648859471193]
  Time series clustering is fundamental in data analysis for discovering temporal patterns. Deep temporal clustering methods have been trying to integrate the canonical k-means into end-to-end training of neural networks. LoSTer is a novel dense autoencoder architecture for the long-sequence time series clustering problem.
  arXiv  Detail & Related papers  (2024-05-08T12:31:35Z)
• Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs [50.25683648762602]
  We introduce Koopman VAE, a new generative framework that is based on a novel design for the model prior. Inspired by Koopman theory, we represent the latent conditional prior dynamics using a linear map. KoVAE outperforms state-of-the-art GAN and VAE methods across several challenging synthetic and real-world time series generation benchmarks.
  arXiv  Detail & Related papers  (2023-10-04T07:14:43Z)
• cs-net: structural approach to time-series forecasting for high-dimensional feature space data with limited observations [1.5533753199073637]
  We propose a flexible data feature extraction technique that excels in high-dimensional multivariate forecasting tasks. Our approach was originally developed for the National Science Foundation (NSF) Algorithms for Threat Detection (ATD) 2022 Challenge. Our models trained on the GDELT dataset finished 1st and 2nd places in the ATD sprint series and hold promise for other datasets for time series forecasting.
  arXiv  Detail & Related papers  (2022-12-05T19:46:47Z)
• Dynamic Time Warping based Adversarial Framework for Time-Series Domain [32.45387153404849]
  We propose a novel framework for the time-series domain referred as Dynamic Time Warping for Adrial Robustness (DTW-AR) We develop a principled algorithm justified by theoretical analysis to efficiently create diverse adversarial examples using random alignment paths. Experiments on diverse real-world benchmarks show the effectiveness of DTW-AR to fool DNNs for time-series data and to improve their robustness using adversarial training.
  arXiv  Detail & Related papers  (2022-07-09T17:23:00Z)
• Towards Similarity-Aware Time-Series Classification [51.2400839966489]
  We study time-series classification (TSC), a fundamental task of time-series data mining. We propose Similarity-Aware Time-Series Classification (SimTSC), a framework that models similarity information with graph neural networks (GNNs)
  arXiv  Detail & Related papers  (2022-01-05T02:14:57Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• Multi-Time-Scale Input Approaches for Hourly-Scale Rainfall-Runoff Modeling based on Recurrent Neural Networks [0.0]
  Two approaches are proposed to reduce the required computational time for time-series modeling through a recurrent neural network (RNN) One approach provides coarse fine temporal resolutions of the input time-series to RNN in parallel. The results confirm that both of the proposed approaches can reduce the computational time for the training of RNN significantly.
  arXiv  Detail & Related papers  (2021-01-30T07:51:55Z)
• Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data with Spatial Information [52.635997570873194]
  This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information. The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
  arXiv  Detail & Related papers  (2021-01-12T20:08:18Z)
• Bayesian Optimization with Machine Learning Algorithms Towards Anomaly Detection [66.05992706105224]
  In this paper, an effective anomaly detection framework is proposed utilizing Bayesian Optimization technique. The performance of the considered algorithms is evaluated using the ISCX 2012 dataset. Experimental results show the effectiveness of the proposed framework in term of accuracy rate, precision, low-false alarm rate, and recall.
  arXiv  Detail & Related papers  (2020-08-05T19:29:35Z)
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks [50.42141893913188]
  We study a distributed variable for large-scale AUC for a neural network as with a deep neural network. Our model requires a much less number of communication rounds and still a number of communication rounds in theory. Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
  arXiv  Detail & Related papers  (2020-05-05T18:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.