gen2Out: Detecting and Ranking Generalized Anomalies
- URL: http://arxiv.org/abs/2109.02704v1
- Date: Mon, 6 Sep 2021 19:29:08 GMT
- Title: gen2Out: Detecting and Ranking Generalized Anomalies
- Authors: Meng-Chieh Lee, Shubhranshu Shekhar, Christos Faloutsos, T. Noah
Hutson, Leon Iasemidis
- Abstract summary: We are the first to generalize anomaly detection in two dimensions.
gen2Out not only detects, but also ranks, anomalies in suspiciousness order.
Experiments on real-world epileptic recordings (200GB) demonstrate effectiveness of gen2Out.
- Score: 18.235699698922566
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In a cloud of m-dimensional data points, how would we spot, as well as rank,
both single-point- as well as group- anomalies? We are the first to generalize
anomaly detection in two dimensions: The first dimension is that we handle both
point-anomalies, as well as group-anomalies, under a unified view -- we shall
refer to them as generalized anomalies. The second dimension is that gen2Out
not only detects, but also ranks, anomalies in suspiciousness order. Detection,
and ranking, of anomalies has numerous applications: For example, in EEG
recordings of an epileptic patient, an anomaly may indicate a seizure; in
computer network traffic data, it may signify a power failure, or a DoS/DDoS
attack. We start by setting some reasonable axioms; surprisingly, none of the
earlier methods pass all the axioms. Our main contribution is the gen2Out
algorithm, that has the following desirable properties: (a) Principled and
Sound anomaly scoring that obeys the axioms for detectors, (b) Doubly-general
in that it detects, as well as ranks generalized anomaly -- both point- and
group-anomalies, (c) Scalable, it is fast and scalable, linear on input size.
(d) Effective, experiments on real-world epileptic recordings (200GB)
demonstrate effectiveness of gen2Out as confirmed by clinicians. Experiments on
27 real-world benchmark datasets show that gen2Out detects ground truth groups,
matches or outperforms point-anomaly baseline algorithms on accuracy, with no
competition for group-anomalies and requires about 2 minutes for 1 million data
points on a stock machine.
Related papers
- ARISE: Graph Anomaly Detection on Attributed Networks via Substructure
Awareness [70.60721571429784]
We propose a new graph anomaly detection framework on attributed networks via substructure awareness (ARISE)
ARISE focuses on the substructures in the graph to discern abnormalities.
Experiments show that ARISE greatly improves detection performance compared to state-of-the-art attributed networks anomaly detection (ANAD) algorithms.
arXiv Detail & Related papers (2022-11-28T12:17:40Z) - Are we certain it's anomalous? [57.729669157989235]
Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations.
Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD)
HypAD learns self-supervisedly to reconstruct the input signal.
arXiv Detail & Related papers (2022-11-16T21:31:39Z) - Catching Both Gray and Black Swans: Open-set Supervised Anomaly
Detection [90.32910087103744]
A few labeled anomaly examples are often available in many real-world applications.
These anomaly examples provide valuable knowledge about the application-specific abnormality.
Those anomalies seen during training often do not illustrate every possible class of anomaly.
This paper tackles open-set supervised anomaly detection.
arXiv Detail & Related papers (2022-03-28T05:21:37Z) - Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
We introduce a novel weakly-supervised anomaly detection framework to train detection models.
The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability.
Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
arXiv Detail & Related papers (2021-08-01T14:33:17Z) - New Methods and Datasets for Group Anomaly Detection From Fundamental
Physics [0.4297070083645048]
Unsupervised group anomaly detection has become a new frontier of fundamental physics.
We propose a realistic synthetic benchmark dataset (LHCO 2020) for the development of group anomaly detection algorithms.
arXiv Detail & Related papers (2021-07-06T18:00:57Z) - Understanding the Effect of Bias in Deep Anomaly Detection [15.83398707988473]
Anomaly detection presents a unique challenge in machine learning, due to the scarcity of labeled anomaly data.
Recent work attempts to mitigate such problems by augmenting training of deep anomaly detection models with additional labeled anomaly samples.
In this paper, we aim to understand the effect of a biased anomaly set on anomaly detection.
arXiv Detail & Related papers (2021-05-16T03:55:02Z) - Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networks [8.608288231153304]
We propose a hop-count based model (HCM) to detect anomalies by modeling both local and global contextual information.
To make better use of hop counts for anomaly identification, we propose to use hop counts prediction as a self-supervised task.
arXiv Detail & Related papers (2021-04-16T06:43:05Z) - Anomaly detection using principles of human perception [0.0]
Unsupervised anomaly detection algorithm is developed that is simple, real-time and parameter-free.
The idea is to assume anomalies are observations that are unexpected to occur with respect to certain groupings made by the majority of the data.
arXiv Detail & Related papers (2021-03-23T05:46:27Z) - Fast and Accurate Anomaly Detection in Dynamic Graphs with a Two-Pronged
Approach [49.25767340466445]
We propose AnomRank, an online algorithm for anomaly detection in dynamic graphs.
AnomRank uses a two-pronged approach defining two novel metrics for anomalousness.
We show theoretically and experimentally that the two-pronged approach successfully detects two common types of anomalies.
arXiv Detail & Related papers (2020-11-26T01:38:27Z) - Deep Weakly-supervised Anomaly Detection [118.55172352231381]
Pairwise Relation prediction Network (PReNet) learns pairwise relation features and anomaly scores.
PReNet can detect any seen/unseen abnormalities that fit the learned pairwise abnormal patterns.
Empirical results on 12 real-world datasets show that PReNet significantly outperforms nine competing methods in detecting seen and unseen anomalies.
arXiv Detail & Related papers (2019-10-30T00:40:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.