A Real-time Anomaly Detection Using Convolutional Autoencoder with Dynamic Threshold

• URL: http://arxiv.org/abs/2404.04311v1
• Date: Fri, 5 Apr 2024 11:03:36 GMT
• Title: A Real-time Anomaly Detection Using Convolutional Autoencoder with Dynamic Threshold
• Authors: Sarit Maitra, Sukanya Kundu, Aishwarya Shankar,
• Abstract summary: This work introduces a hybrid modeling approach combining statistics and a Convolutional Autoencoder with a dynamic threshold. The solution includes a real-time, meter-level anomaly detection system that connects to an advanced monitoring system.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The majority of modern consumer-level energy is generated by real-time smart metering systems. These frequently contain anomalies, which prevent reliable estimates of the series' evolution. This work introduces a hybrid modeling approach combining statistics and a Convolutional Autoencoder with a dynamic threshold. The threshold is determined based on Mahalanobis distance and moving averages. It has been tested using real-life energy consumption data collected from smart metering systems. The solution includes a real-time, meter-level anomaly detection system that connects to an advanced monitoring system. This makes a substantial contribution by detecting unusual data movements and delivering an early warning. Early detection and subsequent troubleshooting can financially benefit organizations and consumers and prevent disasters from occurring.

Related papers

• A Data Mining-Based Dynamical Anomaly Detection Method for Integrating with an Advance Metering System [0.0]
  Building operations consume 30% of total power consumption and contribute 26% of global power-related emissions. This work investigates both supervised and unsupervised approaches to detect anomalies at the meter level. This system is designed to provide real-time detection of anomalies at the meter level.
  arXiv  Detail & Related papers  (2024-05-04T05:26:13Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• A Robust and Explainable Data-Driven Anomaly Detection Approach For Power Electronics [56.86150790999639]
  We present two anomaly detection and classification approaches, namely the Matrix Profile algorithm and anomaly transformer. The Matrix Profile algorithm is shown to be well suited as a generalizable approach for detecting real-time anomalies in streaming time-series data. A series of custom filters is created and added to the detector to tune its sensitivity, recall, and detection accuracy.
  arXiv  Detail & Related papers  (2022-09-23T06:09:35Z)
• Ranking-Based Physics-Informed Line Failure Detection in Power Grids [66.0797334582536]
  Real-time and accurate detecting of potential line failures is the first step to mitigating the extreme weather impact and activating emergency controls. Power balance equations nonlinearity, increased uncertainty in generation during extreme events, and lack of grid observability compromise the efficiency of traditional data-driven failure detection methods. This paper proposes a Physics-InformEd Line failure Detector (FIELD) that leverages grid topology information to reduce sample and time complexities and improve localization accuracy.
  arXiv  Detail & Related papers  (2022-08-31T18:19:25Z)
• Towards an Awareness of Time Series Anomaly Detection Models' Adversarial Vulnerability [21.98595908296989]
  We demonstrate that the performance of state-of-the-art anomaly detection methods is degraded substantially by adding only small adversarial perturbations to the sensor data. We use different scoring metrics such as prediction errors, anomaly, and classification scores over several public and private datasets. We demonstrate, for the first time, the vulnerabilities of anomaly detection systems against adversarial attacks.
  arXiv  Detail & Related papers  (2022-08-24T01:55:50Z)
• Anomaly Detection and Inter-Sensor Transfer Learning on Smart Manufacturing Datasets [6.114996271792091]
  In many cases, the goal of the smart manufacturing system is to rapidly detect (or anticipate) failures to reduce operational cost and eliminate downtime. This often boils down to detecting anomalies within the sensor date acquired from the system. The smart manufacturing application domain poses certain salient technical challenges. We show that predictive failure classification can be achieved, thus paving the way for predictive maintenance.
  arXiv  Detail & Related papers  (2022-06-13T17:51:24Z)
• Smart Meter Data Anomaly Detection using Variational Recurrent Autoencoders with Attention [0.0]
  This paper proposes an unsupervised anomaly detection method based on a Variational Recurrent Autoencoder with attention mechanism. With "dirty" data from smart meters, our method pre-detects missing values and global anomalies to shrink their contribution while training.
  arXiv  Detail & Related papers  (2022-06-08T19:39:51Z)
• Data-driven Residual Generation for Early Fault Detection with Limited Data [4.129225533930966]
  In many complex systems it is not feasible to develop highly accurate models for the systems. Data-driven solutions have received an immense attention in the industries systems for several practical reasons. Unlike the model-based methods it is straight forward to combine time series measurements such as pressure and voltage with other sources of information.
  arXiv  Detail & Related papers  (2021-09-28T03:18:03Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• A Novel Anomaly Detection Algorithm for Hybrid Production Systems based on Deep Learning and Timed Automata [73.38551379469533]
  DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems. It combines deep learning and timed automata for creating behavioral model from observations. The algorithm has been applied to few data sets including two from real systems and has shown promising results.
  arXiv  Detail & Related papers  (2020-10-29T08:27:43Z)
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
  We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction. We put forward an alternative measure of anomaly score to replace the reconstruction-based metric. Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-24T08:26:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.