Real-Time Outlier Detection with Dynamic Process Limits

• URL: http://arxiv.org/abs/2301.13527v1
• Date: Tue, 31 Jan 2023 10:23:02 GMT
• Title: Real-Time Outlier Detection with Dynamic Process Limits
• Authors: Marek Wadinger and Michal Kvasnica
• Abstract summary: This paper proposes an online anomaly detection algorithm for existing real-time infrastructures. Online inverse cumulative distribution-based approach is introduced to eliminate common problems of offline anomaly detectors. The benefit of the proposed method is the ease of use, fast computation, and deployability as shown in two case studies of real microgrid operation data.
• Score: 0.609170287691728
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Anomaly detection methods are part of the systems where rare events may endanger an operation's profitability, safety, and environmental aspects. Although many state-of-the-art anomaly detection methods were developed to date, their deployment is limited to the operation conditions present during the model training. Online anomaly detection brings the capability to adapt to data drifts and change points that may not be represented during model development resulting in prolonged service life. This paper proposes an online anomaly detection algorithm for existing real-time infrastructures where low-latency detection is required and novel patterns in data occur unpredictably. The online inverse cumulative distribution-based approach is introduced to eliminate common problems of offline anomaly detectors, meanwhile providing dynamic process limits to normal operation. The benefit of the proposed method is the ease of use, fast computation, and deployability as shown in two case studies of real microgrid operation data.

Related papers

• Unsupervised Anomaly Detection Using Diffusion Trend Analysis [48.19821513256158]
  We propose a method to detect anomalies by analysis of reconstruction trend depending on the degree of degradation. The proposed method is validated on an open dataset for industrial anomaly detection.
  arXiv  Detail & Related papers  (2024-07-12T01:50:07Z)
• TeVAE: A Variational Autoencoder Approach for Discrete Online Anomaly Detection in Variable-state Multivariate Time-series Data [0.017476232824732776]
  We propose a temporal variational autoencoder (TeVAE) that can detect anomalies with minimal false positives when trained on unlabelled data. When properly configured, TeVAE flags anomalies only 6% of the time wrongly and detects 65% of anomalies present.
  arXiv  Detail & Related papers  (2024-07-09T13:32:33Z)
• Continuous Test-time Domain Adaptation for Efficient Fault Detection under Evolving Operating Conditions [10.627285023764086]
  We propose a Test-time domain Adaptation Anomaly Detection (TAAD) framework that separates input variables into system parameters and measurements. Our approach, tested on a real-world pump monitoring dataset, shows significant improvements over existing domain adaptation methods in fault detection.
  arXiv  Detail & Related papers  (2024-06-06T15:53:14Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• Adaptable and Interpretable Framework for Novelty Detection in Real-Time IoT Systems [0.609170287691728]
  RAID algorithm adapts to non-stationary effects such as data drift and change points that may not be accounted for during model development. RAID algorithm does not require changes to existing process automation infrastructures, making it highly deployable across different domains.
  arXiv  Detail & Related papers  (2023-04-06T09:16:37Z)
• A Robust and Explainable Data-Driven Anomaly Detection Approach For Power Electronics [56.86150790999639]
  We present two anomaly detection and classification approaches, namely the Matrix Profile algorithm and anomaly transformer. The Matrix Profile algorithm is shown to be well suited as a generalizable approach for detecting real-time anomalies in streaming time-series data. A series of custom filters is created and added to the detector to tune its sensitivity, recall, and detection accuracy.
  arXiv  Detail & Related papers  (2022-09-23T06:09:35Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• A Survey on Anomaly Detection for Technical Systems using LSTM Networks [0.0]
  Anomalies represent deviations from the intended system operation and can lead to decreased efficiency as well as partial or complete system failure. In this article, a survey on state-of-the-art anomaly detection using deep neural and especially long short-term memory networks is conducted. The investigated approaches are evaluated based on the application scenario, data and anomaly types as well as further metrics.
  arXiv  Detail & Related papers  (2021-05-28T13:24:40Z)
• ReRe: A Lightweight Real-time Ready-to-Go Anomaly Detection Approach for Time Series [0.27528170226206433]
  This paper introduces ReRe, a Real-time Ready-to-go proactive Anomaly Detection algorithm for streaming time series. ReRe employs two lightweight Long Short-Term Memory (LSTM) models to predict and jointly determine whether or not an upcoming data point is anomalous. Experiments based on real-world time-series datasets demonstrate the good performance of ReRe in real-time anomaly detection.
  arXiv  Detail & Related papers  (2020-04-05T21:26:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.