Evaluating Anomaly Detectors for Simulated Highly Imbalanced Industrial Classification Problems

• URL: http://arxiv.org/abs/2601.00005v1
• Date: Sun, 07 Dec 2025 03:49:54 GMT
• Title: Evaluating Anomaly Detectors for Simulated Highly Imbalanced Industrial Classification Problems
• Authors: Lesley Wheat, Martin v. Mohrenschildt, Saeid Habibi,
• Abstract summary: This paper presents a comprehensive evaluation of anomaly detection algorithms using a problem-agnostic simulated dataset.<n>We benchmark 14 detectors across training datasets with anomaly rates between 0.05% and 20% and training sizes between 1 000 and 10 000.<n>Our findings reveal that the best detector is highly dependant on the total number of faulty examples in the training dataset.
• Score: 1.376408511310322
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning offers potential solutions to current issues in industrial systems in areas such as quality control and predictive maintenance, but also faces unique barriers in industrial applications. An ongoing challenge is extreme class imbalance, primarily due to the limited availability of faulty data during training. This paper presents a comprehensive evaluation of anomaly detection algorithms using a problem-agnostic simulated dataset that reflects real-world engineering constraints. Using a synthetic dataset with a hyper-spherical based anomaly distribution in 2D and 10D, we benchmark 14 detectors across training datasets with anomaly rates between 0.05% and 20% and training sizes between 1 000 and 10 000 (with a testing dataset size of 40 000) to assess performance and generalization error. Our findings reveal that the best detector is highly dependant on the total number of faulty examples in the training dataset, with additional healthy examples offering insignificant benefits in most cases. With less than 20 faulty examples, unsupervised methods (kNN/LOF) dominate; but around 30-50 faulty examples, semi-supervised (XGBOD) and supervised (SVM/CatBoost) detectors, we see large performance increases. While semi-supervised methods do not show significant benefits with only two features, the improvements are evident at ten features. The study highlights the performance drop on generalization of anomaly detection methods on smaller datasets, and provides practical insights for deploying anomaly detection in industrial environments.

Related papers

• DRL-Guided Neural Batch Sampling for Semi-Supervised Pixel-Level Anomaly Detection [0.0]
  Anomaly detection in industrial visual inspection is challenging due to the scarcity of defective samples.<n>We propose a semi-supervised deep reinforcement learning framework that integrates a neural batch sampler, an autoencoder, and a predictor.
  arXiv  Detail & Related papers  (2025-11-25T12:53:53Z)
• Strengthening Anomaly Awareness [0.0]
  We present a refined version of the Anomaly Awareness framework for enhancing unsupervised anomaly detection.<n>Our approach introduces minimal supervision into Variational Autoencoders (VAEs) through a two-stage training strategy.
  arXiv  Detail & Related papers  (2025-04-15T16:52:22Z)
• Robust Distribution Alignment for Industrial Anomaly Detection under Distribution Shift [51.24522135151649]
  Anomaly detection plays a crucial role in quality control for industrial applications.<n>Existing methods attempt to address domain shifts by training generalizable models.<n>Our proposed method demonstrates superior results compared with state-of-the-art anomaly detection and domain adaptation methods.
  arXiv  Detail & Related papers  (2025-03-19T05:25:52Z)
• ISP-AD: A Large-Scale Real-World Dataset for Advancing Industrial Anomaly Detection with Synthetic and Real Defects [0.0]
  TheISP-AD is the largest publicly available industrial dataset to date, including both synthetic and real defects collected directly from the factory floor.<n>Experiments show that even a small amount of injected, weakly labeled real defects improves generalization.
  arXiv  Detail & Related papers  (2025-03-06T21:56:31Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [89.92916473403108]
  This paper proposes a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods.<n>The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics.<n>We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection [46.495442380849894]
  We propose a large-scale, Real-world, and multi-view Industrial Anomaly Detection dataset, named Real-IAD. It contains 150K high-resolution images of 30 different objects, an order of magnitude larger than existing datasets. To make the dataset closer to real application scenarios, we adopted a multi-view shooting method and proposed sample-level evaluation metrics.
  arXiv  Detail & Related papers  (2024-03-19T09:44:41Z)
• Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection [59.41026558455904]
  We focus on multi-modal anomaly detection. Specifically, we investigate early multi-modal approaches that attempted to utilize models pre-trained on large-scale visual datasets. We propose a Local-to-global Self-supervised Feature Adaptation (LSFA) method to finetune the adaptors and learn task-oriented representation toward anomaly detection.
  arXiv  Detail & Related papers  (2024-01-06T07:30:41Z)
• An Outlier Exposure Approach to Improve Visual Anomaly Detection Performance for Mobile Robots [76.36017224414523]
  We consider the problem of building visual anomaly detection systems for mobile robots. Standard anomaly detection models are trained using large datasets composed only of non-anomalous data. We tackle the problem of exploiting these data to improve the performance of a Real-NVP anomaly detection model.
  arXiv  Detail & Related papers  (2022-09-20T15:18:13Z)
• Scale-Equivalent Distillation for Semi-Supervised Object Detection [57.59525453301374]
  Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, generating hard pseudo-labels by a teacher model on unlabeled data as supervisory signals. We analyze the challenges these methods meet with the empirical experiment results. We introduce a novel approach, Scale-Equivalent Distillation (SED), which is a simple yet effective end-to-end knowledge distillation framework robust to large object size variance and class imbalance.
  arXiv  Detail & Related papers  (2022-03-23T07:33:37Z)
• Semantic Perturbations with Normalizing Flows for Improved Generalization [62.998818375912506]
  We show that perturbations in the latent space can be used to define fully unsupervised data augmentations. We find that our latent adversarial perturbations adaptive to the classifier throughout its training are most effective.
  arXiv  Detail & Related papers  (2021-08-18T03:20:00Z)
• Out-Of-Bag Anomaly Detection [0.9449650062296822]
  Data anomalies are ubiquitous in real world datasets, and can have an adverse impact on machine learning (ML) systems. We propose a novel model-based anomaly detection method, that we call Out-of-Bag anomaly detection. We show our method can improve the accuracy and reliability of an ML system as data pre-processing step via a case study on home valuation.
  arXiv  Detail & Related papers  (2020-09-20T06:01:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.