Meta-learning with GANs for anomaly detection, with deployment in high-speed rail inspection system

• URL: http://arxiv.org/abs/2202.05795v1
• Date: Fri, 11 Feb 2022 17:43:49 GMT
• Title: Meta-learning with GANs for anomaly detection, with deployment in high-speed rail inspection system
• Authors: Haoyang Cao, Xin Guo, Guan Wang
• Abstract summary: Key challenges for anomaly detection in the AI era with big data include lack of prior knowledge of potential anomaly types. Within this framework, we incorporate the idea of generative adversarial networks (GANs) with appropriate choices of loss functions. Our framework has been deployed in five high-speed railways of China since 2021: it has reduced more than 99.7% workload and saved 96.7% inspection time.
• Score: 7.220842608593749
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Anomaly detection has been an active research area with a wide range of potential applications. Key challenges for anomaly detection in the AI era with big data include lack of prior knowledge of potential anomaly types, highly complex and noisy background in input data, scarce abnormal samples, and imbalanced training dataset. In this work, we propose a meta-learning framework for anomaly detection to deal with these issues. Within this framework, we incorporate the idea of generative adversarial networks (GANs) with appropriate choices of loss functions including structural similarity index measure (SSIM). Experiments with limited labeled data for high-speed rail inspection demonstrate that our meta-learning framework is sharp and robust in identifying anomalies. Our framework has been deployed in five high-speed railways of China since 2021: it has reduced more than 99.7% workload and saved 96.7% inspection time.

Related papers

• TeVAE: A Variational Autoencoder Approach for Discrete Online Anomaly Detection in Variable-state Multivariate Time-series Data [0.017476232824732776]
  We propose a temporal variational autoencoder (TeVAE) that can detect anomalies with minimal false positives when trained on unlabelled data. When properly configured, TeVAE flags anomalies only 6% of the time wrongly and detects 65% of anomalies present.
  arXiv  Detail & Related papers  (2024-07-09T13:32:33Z)
• ADT: Agent-based Dynamic Thresholding for Anomaly Detection [4.356615197661274]
  We propose an agent-based dynamic thresholding (ADT) framework based on a deep Q-network. An auto-encoder is utilized in this study to obtain feature representations and produce anomaly scores for complex input data. ADT can adjust thresholds adaptively by utilizing the anomaly scores from the auto-encoder.
  arXiv  Detail & Related papers  (2023-12-03T19:07:30Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• An Outlier Exposure Approach to Improve Visual Anomaly Detection Performance for Mobile Robots [76.36017224414523]
  We consider the problem of building visual anomaly detection systems for mobile robots. Standard anomaly detection models are trained using large datasets composed only of non-anomalous data. We tackle the problem of exploiting these data to improve the performance of a Real-NVP anomaly detection model.
  arXiv  Detail & Related papers  (2022-09-20T15:18:13Z)
• TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data [13.864161788250856]
  TranAD is a deep transformer network based anomaly detection and diagnosis model. It uses attention-based sequence encoders to swiftly perform inference with the knowledge of the broader temporal trends in the data. TranAD can outperform state-of-the-art baseline methods in detection and diagnosis performance with data and time-efficient training.
  arXiv  Detail & Related papers  (2022-01-18T19:41:29Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• Federated Variational Learning for Anomaly Detection in Multivariate Time Series [13.328883578980237]
  We propose an unsupervised time series anomaly detection framework in a federated fashion. We leave the training data distributed at the edge to learn a shared Variational Autoencoder (VAE) based on Convolutional Gated Recurrent Unit (ConvGRU) model. Experiments on three real-world networked sensor datasets illustrate the advantage of our approach over other state-of-the-art models.
  arXiv  Detail & Related papers  (2021-08-18T22:23:15Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)
• Contextual-Bandit Anomaly Detection for IoT Data in Distributed Hierarchical Edge Computing [65.78881372074983]
  IoT devices can hardly afford complex deep neural networks (DNN) models, and offloading anomaly detection tasks to the cloud incurs long delay. We propose and build a demo for an adaptive anomaly detection approach for distributed hierarchical edge computing (HEC) systems. We show that our proposed approach significantly reduces detection delay without sacrificing accuracy, as compared to offloading detection tasks to the cloud.
  arXiv  Detail & Related papers  (2020-04-15T06:13:33Z)
• An Intelligent and Time-Efficient DDoS Identification Framework for Real-Time Enterprise Networks SAD-F: Spark Based Anomaly Detection Framework [0.5811502603310248]
  We will be exploring security analytic techniques for DDoS anomaly detection using different machine learning techniques. In this paper, we are proposing a novel approach which deals with real traffic as input to the system. We study and compare the performance factor of our proposed framework on three different testbeds.
  arXiv  Detail & Related papers  (2020-01-21T06:05:48Z)
• Adaptive Anomaly Detection for IoT Data in Hierarchical Edge Computing [71.86955275376604]
  We propose an adaptive anomaly detection approach for hierarchical edge computing (HEC) systems to solve this problem. We design an adaptive scheme to select one of the models based on the contextual information extracted from input data, to perform anomaly detection. We evaluate our proposed approach using a real IoT dataset, and demonstrate that it reduces detection delay by 84% while maintaining almost the same accuracy as compared to offloading detection tasks to the cloud.
  arXiv  Detail & Related papers  (2020-01-10T05:29:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.