Unsupervised Anomaly Detection Ensembles using Item Response Theory

• URL: http://arxiv.org/abs/2106.06243v1
• Date: Fri, 11 Jun 2021 08:51:26 GMT
• Title: Unsupervised Anomaly Detection Ensembles using Item Response Theory
• Authors: Sevvandi Kandanaarachchi
• Abstract summary: We use Item Response Theory (IRT) to construct an unsupervised anomaly detection ensemble. IRT's latent trait lends itself to anomaly detection because the latent trait can be used to uncover the hidden ground truth. We demonstrate the effectiveness of the IRT ensemble on an extensive data repository.
• Score: 0.4640835690336652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Constructing an ensemble from a heterogeneous set of unsupervised anomaly detection methods is challenging because the class labels or the ground truth is unknown. Thus, traditional ensemble techniques that use the response variable or the class labels cannot be used to construct an ensemble for unsupervised anomaly detection. We use Item Response Theory (IRT) -- a class of models used in educational psychometrics to assess student and test question characteristics -- to construct an unsupervised anomaly detection ensemble. IRT's latent trait computation lends itself to anomaly detection because the latent trait can be used to uncover the hidden ground truth. Using a novel IRT mapping to the anomaly detection problem, we construct an ensemble that can downplay noisy, non-discriminatory methods and accentuate sharper methods. We demonstrate the effectiveness of the IRT ensemble on an extensive data repository, by comparing its performance to other ensemble techniques.

Related papers

• Enhancing Anomaly Detection via Generating Diversified and Hard-to-distinguish Synthetic Anomalies [7.021105583098609]
  Recent approaches have focused on leveraging domain-specific transformations or perturbations to generate synthetic anomalies from normal samples. We introduce a novel domain-agnostic method that employs a set of conditional perturbators and a discriminator. We demonstrate the superiority of our method over state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2024-09-16T08:15:23Z)
• Ensembled Cold-Diffusion Restorations for Unsupervised Anomaly Detection [7.94529540044472]
  Unsupervised Anomaly Detection (UAD) methods aim to identify anomalies in test samples comparing them with a normative distribution learned from a dataset known to be anomaly-free. Approaches based on generative models offer interpretability by generating anomaly-free versions of test images, but are typically unable to identify subtle anomalies. We present a novel method that combines the strengths of both strategies: a generative cold-diffusion pipeline that is trained with the objective of turning synthetically-corrupted images back to their normal, original appearance.
  arXiv  Detail & Related papers  (2024-07-09T08:02:46Z)
• Towards a Unified Framework of Clustering-based Anomaly Detection [18.30208347233284]
  Unsupervised Anomaly Detection (UAD) plays a crucial role in identifying abnormal patterns within data without labeled examples. We propose a novel probabilistic mixture model for anomaly detection to establish a theoretical connection among representation learning, clustering, and anomaly detection. We have devised an improved anomaly score that more effectively harnesses the combined power of representation learning and clustering.
  arXiv  Detail & Related papers  (2024-06-01T14:30:12Z)
• Generating and Reweighting Dense Contrastive Patterns for Unsupervised Anomaly Detection [59.34318192698142]
  We introduce a prior-less anomaly generation paradigm and develop an innovative unsupervised anomaly detection framework named GRAD. PatchDiff effectively expose various types of anomaly patterns. experiments on both MVTec AD and MVTec LOCO datasets also support the aforementioned observation.
  arXiv  Detail & Related papers  (2023-12-26T07:08:06Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection [97.93062818228015]
  We propose to integrate the reconstruction-based functionality into a novel self-supervised predictive architectural building block. Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field. We demonstrate the generality of our block by integrating it into several state-of-the-art frameworks for anomaly detection on image and video.
  arXiv  Detail & Related papers  (2021-11-17T13:30:31Z)
• Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data [150.9270911031327]
  We consider the problem of anomaly detection with a small set of partially labeled anomaly examples and a large-scale unlabeled dataset. Existing related methods either exclusively fit the limited anomaly examples that typically do not span the entire set of anomalies, or proceed with unsupervised learning from the unlabeled data. We propose here instead a deep reinforcement learning-based approach that enables an end-to-end optimization of the detection of both labeled and unlabeled anomalies.
  arXiv  Detail & Related papers  (2020-09-15T03:05:39Z)
• Generalizing Fault Detection Against Domain Shifts Using Stratification-Aware Cross-Validation [4.731408120697983]
  Incipient anomalies present milder symptoms compared to severe ones. These anomalies can be easily mistaken as normal operating conditions. We show that ensemble learning methods can give improved performance on incipient anomalies.
  arXiv  Detail & Related papers  (2020-08-20T00:03:09Z)
• Using Ensemble Classifiers to Detect Incipient Anomalies [12.947364178385637]
  Incipient anomalies present milder symptoms compared to severe ones. These anomalies can be easily mistaken as normal operating conditions. We show that ensemble learning methods can give improved performance on incipient anomalies.
  arXiv  Detail & Related papers  (2020-08-20T00:00:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.