Algorithmic Frameworks for the Detection of High Density Anomalies
- URL: http://arxiv.org/abs/2010.04705v2
- Date: Sun, 4 Apr 2021 07:49:39 GMT
- Title: Algorithmic Frameworks for the Detection of High Density Anomalies
- Authors: Ralph Foorthuis
- Abstract summary: High-density anomalies are deviant cases positioned in the most normal regions of the data space.
This study introduces several non-parametric algorithmic frameworks for unsupervised detection.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This study explores the concept of high-density anomalies. As opposed to the
traditional concept of anomalies as isolated occurrences, high-density
anomalies are deviant cases positioned in the most normal regions of the data
space. Such anomalies are relevant for various practical use cases, such as
misbehavior detection and data quality analysis. Effective methods for
identifying them are particularly important when analyzing very large or noisy
sets, for which traditional anomaly detection algorithms will return many false
positives. In order to be able to identify high-density anomalies, this study
introduces several non-parametric algorithmic frameworks for unsupervised
detection. These frameworks are able to leverage existing underlying anomaly
detection algorithms and offer different solutions for the balancing problem
inherent in this detection task. The frameworks are evaluated with both
synthetic and real-world datasets, and are compared with existing baseline
algorithms for detecting traditional anomalies. The Iterative Partial Push
(IPP) framework proves to yield the best detection results.
Related papers
- Can I trust my anomaly detection system? A case study based on explainable AI [0.4416503115535552]
This case study explores the robustness of an anomaly detection system based on variational autoencoder generative models.
The goal is to get a different perspective on the real performances of anomaly detectors that use reconstruction differences.
arXiv Detail & Related papers (2024-07-29T12:39:07Z) - Unsupervised Anomaly Detection Using Diffusion Trend Analysis [48.19821513256158]
We propose a method to detect anomalies by analysis of reconstruction trend depending on the degree of degradation.
The proposed method is validated on an open dataset for industrial anomaly detection.
arXiv Detail & Related papers (2024-07-12T01:50:07Z) - Anomaly Detection Based on Isolation Mechanisms: A Survey [13.449446806837422]
Isolation-based unsupervised anomaly detection is a novel and effective approach for identifying anomalies in data.
We review the state-of-the-art isolation-based anomaly detection methods, including their data partitioning strategies, anomaly score functions, and algorithmic details.
arXiv Detail & Related papers (2024-03-16T04:29:21Z) - Can Tree Based Approaches Surpass Deep Learning in Anomaly Detection? A
Benchmarking Study [0.6291443816903801]
This paper evaluates a diverse array of machine learning-based anomaly detection algorithms.
The paper contributes significantly by conducting an unbiased comparison of various anomaly detection algorithms.
arXiv Detail & Related papers (2024-02-11T19:12:51Z) - Unraveling the "Anomaly" in Time Series Anomaly Detection: A
Self-supervised Tri-domain Solution [89.16750999704969]
Anomaly labels hinder traditional supervised models in time series anomaly detection.
Various SOTA deep learning techniques, such as self-supervised learning, have been introduced to tackle this issue.
We propose a novel self-supervised learning based Tri-domain Anomaly Detector (TriAD)
arXiv Detail & Related papers (2023-11-19T05:37:18Z) - Deep Learning for Time Series Anomaly Detection: A Survey [53.83593870825628]
Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare.
The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns.
This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning.
arXiv Detail & Related papers (2022-11-09T22:40:22Z) - Catching Both Gray and Black Swans: Open-set Supervised Anomaly
Detection [90.32910087103744]
A few labeled anomaly examples are often available in many real-world applications.
These anomaly examples provide valuable knowledge about the application-specific abnormality.
Those anomalies seen during training often do not illustrate every possible class of anomaly.
This paper tackles open-set supervised anomaly detection.
arXiv Detail & Related papers (2022-03-28T05:21:37Z) - Anomaly Rule Detection in Sequence Data [2.3757190901941736]
We present a new anomaly detection framework called DUOS that enables Discovery of Utility-aware Outlier Sequential rules from a set of sequences.
In this work, we incorporate both the anomalousness and utility of a group, and then introduce the concept of utility-aware outlier rule (UOSR)
arXiv Detail & Related papers (2021-11-29T23:52:31Z) - Anomaly Detection for High-Dimensional Data Using Large Deviations
Principle [0.8526086056172273]
We propose an anomaly detection algorithm that can scale to high-dimensional data using concepts from the theory of large deviations.
The proposed Large Deviations Anomaly Detection (LAD) algorithm is shown to outperform state of art anomaly detection methods on a variety of large and high-dimensional benchmark data sets.
arXiv Detail & Related papers (2021-09-28T13:13:14Z) - Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
We introduce a novel weakly-supervised anomaly detection framework to train detection models.
The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability.
Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
arXiv Detail & Related papers (2021-08-01T14:33:17Z) - Toward Deep Supervised Anomaly Detection: Reinforcement Learning from
Partially Labeled Anomaly Data [150.9270911031327]
We consider the problem of anomaly detection with a small set of partially labeled anomaly examples and a large-scale unlabeled dataset.
Existing related methods either exclusively fit the limited anomaly examples that typically do not span the entire set of anomalies, or proceed with unsupervised learning from the unlabeled data.
We propose here instead a deep reinforcement learning-based approach that enables an end-to-end optimization of the detection of both labeled and unlabeled anomalies.
arXiv Detail & Related papers (2020-09-15T03:05:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.