RCC-Dual-GAN: An Efficient Approach for Outlier Detection with Few Identified Anomalies

• URL: http://arxiv.org/abs/2003.03609v1
• Date: Sat, 7 Mar 2020 17:13:52 GMT
• Title: RCC-Dual-GAN: An Efficient Approach for Outlier Detection with Few Identified Anomalies
• Authors: Zhe Li, Chunhua Sun, Chunli Liu, Xiayu Chen, Meng Wang, Yezheng Liu
• Abstract summary: Outlier detection is an important task in data mining and many technologies have been explored in various applications. We propose a novel detection model Dual-GAN, which can directly utilize the potential information in identified anomalies to detect discrete outliers simultaneously. In addition, to deal with the evaluation of Nash equilibrium and the selection of optimal model, two evaluation indicators are created and introduced into the two models to make the detection process more intelligent.
• Score: 11.02452262854759
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Outlier detection is an important task in data mining and many technologies have been explored in various applications. However, due to the default assumption that outliers are non-concentrated, unsupervised outlier detection may not correctly detect group anomalies with higher density levels. As for the supervised outlier detection, although high detection rates and optimal parameters can usually be achieved, obtaining sufficient and correct labels is a time-consuming task. To address these issues, we focus on semi-supervised outlier detection with few identified anomalies, in the hope of using limited labels to achieve high detection accuracy. First, we propose a novel detection model Dual-GAN, which can directly utilize the potential information in identified anomalies to detect discrete outliers and partially identified group anomalies simultaneously. And then, considering the instances with similar output values may not all be similar in a complex data structure, we replace the two MO-GAN components in Dual-GAN with the combination of RCC and M-GAN (RCC-Dual-GAN). In addition, to deal with the evaluation of Nash equilibrium and the selection of optimal model, two evaluation indicators are created and introduced into the two models to make the detection process more intelligent. Extensive experiments on both benchmark datasets and two practical tasks demonstrate that our proposed approaches (i.e., Dual-GAN and RCC-Dual-GAN) can significantly improve the accuracy of outlier detection even with only a few identified anomalies. Moreover, compared with the two MO-GAN components in Dual-GAN, the network structure combining RCC and M-GAN has greater stability in various situations.

Related papers

• FUN-AD: Fully Unsupervised Learning for Anomaly Detection with Noisy Training Data [1.0650780147044159]
  We propose a novel learning-based approach for fully unsupervised anomaly detection with unlabeled and potentially contaminated training data. Our method is motivated by two observations, that i) the pairwise feature distances between the normal samples are on average likely to be smaller than those between the anomaly samples or heterogeneous samples and ii) pairs of features mutually closest to each other are likely to be homogeneous pairs. Building on the first observation that nearest-neighbor distances can distinguish between confident normal samples and anomalies, we propose a pseudo-labeling strategy using an iteratively reconstructed memory bank.
  arXiv  Detail & Related papers  (2024-11-25T05:51:38Z)
• Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detection [63.93728560200819]
  Unsupervised out-of-distribution (U-OOD) detection is to identify data samples with a detector trained solely on unlabeled in-distribution (ID) data. Recent studies have developed various detectors based on DGMs to move beyond likelihood. We apply two techniques for each direction, specifically post-hoc prior and dataset entropy-mutual calibration. Experimental results demonstrate that the Resultant could be a new state-of-the-art U-OOD detector.
  arXiv  Detail & Related papers  (2024-09-05T02:58:13Z)
• Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark [101.23684938489413]
  Anomaly detection (AD) is often focused on detecting anomalies for industrial quality inspection and medical lesion examination. This work first constructs a large-scale and general-purpose COCO-AD dataset by extending COCO to the AD field. Inspired by the metrics in the segmentation field, we propose several more practical threshold-dependent AD-specific metrics.
  arXiv  Detail & Related papers  (2024-04-16T17:38:26Z)
• Open-Vocabulary Video Anomaly Detection [57.552523669351636]
  Video anomaly detection (VAD) with weak supervision has achieved remarkable performance in utilizing video-level labels to discriminate whether a video frame is normal or abnormal. Recent studies attempt to tackle a more realistic setting, open-set VAD, which aims to detect unseen anomalies given seen anomalies and normal videos. This paper takes a step further and explores open-vocabulary video anomaly detection (OVVAD), in which we aim to leverage pre-trained large models to detect and categorize seen and unseen anomalies.
  arXiv  Detail & Related papers  (2023-11-13T02:54:17Z)
• CL-Flow:Strengthening the Normalizing Flows by Contrastive Learning for Better Anomaly Detection [1.951082473090397]
  We propose a self-supervised anomaly detection approach that combines contrastive learning with 2D-Flow. Compared to mainstream unsupervised approaches, our self-supervised method demonstrates superior detection accuracy, fewer additional model parameters, and faster inference speed. Our approach showcases new state-of-the-art results, achieving a performance of 99.6% in image-level AUROC on the MVTecAD dataset and 96.8% in image-level AUROC on the BTAD dataset.
  arXiv  Detail & Related papers  (2023-11-12T10:07:03Z)
• RoSAS: Deep Semi-Supervised Anomaly Detection with Contamination-Resilient Continuous Supervision [21.393509817509464]
  This paper proposes a novel semi-supervised anomaly detection method, which devises textitcontamination-resilient continuous supervisory signals Our approach significantly outperforms state-of-the-art competitors by 20%-30% in AUC-PR.
  arXiv  Detail & Related papers  (2023-07-25T04:04:49Z)
• ESAD: End-to-end Deep Semi-supervised Anomaly Detection [85.81138474858197]
  We propose a new objective function that measures the KL-divergence between normal and anomalous data. The proposed method significantly outperforms several state-of-the-arts on multiple benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-09T08:16:35Z)
• Dual Adversarial Auto-Encoders for Clustering [152.84443014554745]
  We propose Dual Adversarial Auto-encoder (Dual-AAE) for unsupervised clustering. By performing variational inference on the objective function of Dual-AAE, we derive a new reconstruction loss which can be optimized by training a pair of Auto-encoders. Experiments on four benchmarks show that Dual-AAE achieves superior performance over state-of-the-art clustering methods.
  arXiv  Detail & Related papers  (2020-08-23T13:16:34Z)
• SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection [63.253850875265115]
  Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples. We propose a modular acceleration system, called SUOD, to address it.
  arXiv  Detail & Related papers  (2020-03-11T00:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.