IPOF: An Extremely and Excitingly Simple Outlier Detection Booster via
Infinite Propagation
- URL: http://arxiv.org/abs/2108.00360v1
- Date: Sun, 1 Aug 2021 03:48:09 GMT
- Title: IPOF: An Extremely and Excitingly Simple Outlier Detection Booster via
Infinite Propagation
- Authors: Sibo Zhu, Handong Zhao, Hongfu Liu
- Abstract summary: Outlier detection is one of the most popular and continuously rising topics in the data mining field.
In this paper, we consider the score-based outlier detection category and point out that the performance of current outlier detection algorithms might be further boosted by score propagation.
Specifically, we propose Infinite propagation of Outlier Factor (iPOF) algorithm, an extremely and excitingly simple outlier detection booster via infinite propagation.
- Score: 30.91911545889579
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Outlier detection is one of the most popular and continuously rising topics
in the data mining field due to its crucial academic value and extensive
industrial applications. Among different settings, unsupervised outlier
detection is the most challenging and practical one, which attracts tremendous
efforts from diverse perspectives. In this paper, we consider the score-based
outlier detection category and point out that the performance of current
outlier detection algorithms might be further boosted by score propagation.
Specifically, we propose Infinite Propagation of Outlier Factor (iPOF)
algorithm, an extremely and excitingly simple outlier detection booster via
infinite propagation. By employing score-based outlier detectors for
initialization, iPOF updates each data point's outlier score by averaging the
outlier factors of its nearest common neighbors. Extensive experimental results
on numerous datasets in various domains demonstrate the effectiveness and
efficiency of iPOF significantly over several classical and recent
state-of-the-art methods. We also provide the parameter analysis on the number
of neighbors, the unique parameter in iPOF, and different initial outlier
detectors for general validation. It is worthy to note that iPOF brings in
positive improvements ranging from 2% to 46% on the average level, and in some
cases, iPOF boosts the performance over 3000% over the original outlier
detection algorithm.
Related papers
- Fuzzy Granule Density-Based Outlier Detection with Multi-Scale Granular Balls [65.44462297594308]
Outlier detection refers to the identification of anomalous samples that deviate significantly from the distribution of normal data.
Most unsupervised outlier detection methods are carefully designed to detect specified outliers.
We propose a fuzzy rough sets-based multi-scale outlier detection method to identify various types of outliers.
arXiv Detail & Related papers (2025-01-06T12:35:51Z) - An Efficient Outlier Detection Algorithm for Data Streaming [51.56874851156008]
Traditional outlier detection methods, such as the Local Outlier Factor (LOF) algorithm, struggle with real-time data.
We propose a novel approach to enhance the efficiency of LOF algorithms for online anomaly detection, named the Efficient Incremental LOF (EILOF) algorithm.
The EILOF algorithm not only significantly reduces computational costs, but also systematically improves detection accuracy when the number of additional points increases.
arXiv Detail & Related papers (2025-01-02T05:12:43Z) - Rethinking Unsupervised Outlier Detection via Multiple Thresholding [15.686139522490189]
We propose a multiple thresholding (Multi-T) module to advance existing scoring methods.
It generates two thresholds that isolate inliers and outliers from the unlabelled target dataset.
Experiments verify that Multi-T can significantly improve proposed outlier scoring methods.
arXiv Detail & Related papers (2024-07-07T14:09:50Z) - Diversified Outlier Exposure for Out-of-Distribution Detection via
Informative Extrapolation [110.34982764201689]
Out-of-distribution (OOD) detection is important for deploying reliable machine learning models on real-world applications.
Recent advances in outlier exposure have shown promising results on OOD detection via fine-tuning model with informatively sampled auxiliary outliers.
We propose a novel framework, namely, Diversified Outlier Exposure (DivOE), for effective OOD detection via informative extrapolation based on the given auxiliary outliers.
arXiv Detail & Related papers (2023-10-21T07:16:09Z) - Adaptive Thresholding Heuristic for KPI Anomaly Detection [1.57731592348751]
A plethora of outlier detectors have been explored in the time series domain, however, in a business sense, not all outliers are anomalies of interest.
This article proposes an Adaptive Thresholding Heuristic (ATH) to dynamically adjust the detection threshold based on the local properties of the data distribution and adapt to changes in time series patterns.
Experimental results show that ATH is efficient making it scalable for near real time anomaly detection and flexible with forecasters and outlier detectors.
arXiv Detail & Related papers (2023-08-21T06:45:28Z) - Little Help Makes a Big Difference: Leveraging Active Learning to
Improve Unsupervised Time Series Anomaly Detection [2.1684857243537334]
A large set of anomaly detection algorithms have been deployed for detecting unexpected network incidents.
Unsupervised anomaly detection algorithms often suffer from excessive false alarms.
We propose to use active learning to introduce and benefit from the feedback of operators.
arXiv Detail & Related papers (2022-01-25T13:54:19Z) - Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions.
In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data.
We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
arXiv Detail & Related papers (2022-01-11T23:01:12Z) - EAD: an ensemble approach to detect adversarial examples from the hidden
features of deep neural networks [1.3212032015497979]
We propose an Ensemble Adversarial Detector (EAD) for the identification of adversarial examples.
EAD combines multiple detectors that exploit distinct properties of the input instances in the internal representation of a pre-trained Deep Neural Network (DNN)
We show that EAD achieves the best AUROC and AUPR in the large majority of the settings and comparable performance in the others.
arXiv Detail & Related papers (2021-11-24T17:05:26Z) - Robust and Accurate Object Detection via Adversarial Learning [111.36192453882195]
This work augments the fine-tuning stage for object detectors by exploring adversarial examples.
Our approach boosts the performance of state-of-the-art EfficientDets by +1.1 mAP on the object detection benchmark.
arXiv Detail & Related papers (2021-03-23T19:45:26Z) - Discriminative Nearest Neighbor Few-Shot Intent Detection by
Transferring Natural Language Inference [150.07326223077405]
Few-shot learning is attracting much attention to mitigate data scarcity.
We present a discriminative nearest neighbor classification with deep self-attention.
We propose to boost the discriminative ability by transferring a natural language inference (NLI) model.
arXiv Detail & Related papers (2020-10-25T00:39:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.