New Methods and Datasets for Group Anomaly Detection From Fundamental
Physics
- URL: http://arxiv.org/abs/2107.02821v1
- Date: Tue, 6 Jul 2021 18:00:57 GMT
- Title: New Methods and Datasets for Group Anomaly Detection From Fundamental
Physics
- Authors: Gregor Kasieczka, Benjamin Nachman, David Shih
- Abstract summary: Unsupervised group anomaly detection has become a new frontier of fundamental physics.
We propose a realistic synthetic benchmark dataset (LHCO 2020) for the development of group anomaly detection algorithms.
- Score: 0.4297070083645048
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The identification of anomalous overdensities in data - group or collective
anomaly detection - is a rich problem with a large number of real world
applications. However, it has received relatively little attention in the
broader ML community, as compared to point anomalies or other types of single
instance outliers. One reason for this is the lack of powerful benchmark
datasets. In this paper, we first explain how, after the Nobel-prize winning
discovery of the Higgs boson, unsupervised group anomaly detection has become a
new frontier of fundamental physics (where the motivation is to find new
particles and forces). Then we propose a realistic synthetic benchmark dataset
(LHCO2020) for the development of group anomaly detection algorithms. Finally,
we compare several existing statistically-sound techniques for unsupervised
group anomaly detection, and demonstrate their performance on the LHCO2020
dataset.
Related papers
- ARC: A Generalist Graph Anomaly Detector with In-Context Learning [62.202323209244]
ARC is a generalist GAD approach that enables a one-for-all'' GAD model to detect anomalies across various graph datasets on-the-fly.
equipped with in-context learning, ARC can directly extract dataset-specific patterns from the target dataset.
Extensive experiments on multiple benchmark datasets from various domains demonstrate the superior anomaly detection performance, efficiency, and generalizability of ARC.
arXiv Detail & Related papers (2024-05-27T02:42:33Z) - SAD: Semi-Supervised Anomaly Detection on Dynamic Graphs [11.819993729810257]
Anomaly detection aims to distinguish abnormal instances that deviate significantly from the majority of benign ones.
graph neural networks become increasingly popular in tackling the anomaly detection problem.
We present semi-supervised anomaly detection (SAD), an end-to-end framework for anomaly detection on dynamic graphs.
arXiv Detail & Related papers (2023-05-23T01:05:34Z) - Are we certain it's anomalous? [57.729669157989235]
Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations.
Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD)
HypAD learns self-supervisedly to reconstruct the input signal.
arXiv Detail & Related papers (2022-11-16T21:31:39Z) - gen2Out: Detecting and Ranking Generalized Anomalies [18.235699698922566]
We are the first to generalize anomaly detection in two dimensions.
gen2Out not only detects, but also ranks, anomalies in suspiciousness order.
Experiments on real-world epileptic recordings (200GB) demonstrate effectiveness of gen2Out.
arXiv Detail & Related papers (2021-09-06T19:29:08Z) - Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
We introduce a novel weakly-supervised anomaly detection framework to train detection models.
The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability.
Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
arXiv Detail & Related papers (2021-08-01T14:33:17Z) - Understanding the Effect of Bias in Deep Anomaly Detection [15.83398707988473]
Anomaly detection presents a unique challenge in machine learning, due to the scarcity of labeled anomaly data.
Recent work attempts to mitigate such problems by augmenting training of deep anomaly detection models with additional labeled anomaly samples.
In this paper, we aim to understand the effect of a biased anomaly set on anomaly detection.
arXiv Detail & Related papers (2021-05-16T03:55:02Z) - Anomaly detection using principles of human perception [0.0]
Unsupervised anomaly detection algorithm is developed that is simple, real-time and parameter-free.
The idea is to assume anomalies are observations that are unexpected to occur with respect to certain groupings made by the majority of the data.
arXiv Detail & Related papers (2021-03-23T05:46:27Z) - TadGAN: Time Series Anomaly Detection Using Generative Adversarial
Networks [73.01104041298031]
TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs)
To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics.
To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
arXiv Detail & Related papers (2020-09-16T15:52:04Z) - Toward Deep Supervised Anomaly Detection: Reinforcement Learning from
Partially Labeled Anomaly Data [150.9270911031327]
We consider the problem of anomaly detection with a small set of partially labeled anomaly examples and a large-scale unlabeled dataset.
Existing related methods either exclusively fit the limited anomaly examples that typically do not span the entire set of anomalies, or proceed with unsupervised learning from the unlabeled data.
We propose here instead a deep reinforcement learning-based approach that enables an end-to-end optimization of the detection of both labeled and unlabeled anomalies.
arXiv Detail & Related papers (2020-09-15T03:05:39Z) - How to find a unicorn: a novel model-free, unsupervised anomaly
detection method for time series [0.0]
We introduce a new anomaly concept called "unicorn" or unique event and present a new, model-free, unsupervised detection algorithm to detect unicorns.
The performance of our algorithm was examined in recognizing unique events on different types of simulated data sets with anomalies.
arXiv Detail & Related papers (2020-04-23T21:38:38Z) - Deep Weakly-supervised Anomaly Detection [118.55172352231381]
Pairwise Relation prediction Network (PReNet) learns pairwise relation features and anomaly scores.
PReNet can detect any seen/unseen abnormalities that fit the learned pairwise abnormal patterns.
Empirical results on 12 real-world datasets show that PReNet significantly outperforms nine competing methods in detecting seen and unseen anomalies.
arXiv Detail & Related papers (2019-10-30T00:40:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.