GradStop: Exploring Training Dynamics in Unsupervised Outlier Detection through Gradient Cohesion

• URL: http://arxiv.org/abs/2412.08501v1
• Date: Wed, 11 Dec 2024 16:07:58 GMT
• Title: GradStop: Exploring Training Dynamics in Unsupervised Outlier Detection through Gradient Cohesion
• Authors: Yuang Zhang, Liping Wang, Yihong Huang, Yuanxing Zheng,
• Abstract summary: Unsupervised Outlier Detection (UOD) is a critical task in data mining and machine learning, aiming to identify instances that significantly deviate from the majority. Without any label, deep UOD methods struggle with the misalignment between the model's direct optimization goal and the final performance goal of Outlier Detection task. This paper proposes an early stopping algorithm to optimize the training of deep UOD models, ensuring they perform optimally in Outlier Detection task.
• Score: 3.5510589740276424
• License:
• Abstract: Unsupervised Outlier Detection (UOD) is a critical task in data mining and machine learning, aiming to identify instances that significantly deviate from the majority. Without any label, deep UOD methods struggle with the misalignment between the model's direct optimization goal and the final performance goal of Outlier Detection (OD) task. Through the perspective of training dynamics, this paper proposes an early stopping algorithm to optimize the training of deep UOD models, ensuring they perform optimally in OD rather than overfitting the entire contaminated dataset. Inspired by UOD mechanism and inlier priority phenomenon, where intuitively models fit inliers more quickly than outliers, we propose GradStop, a sampling-based label-free algorithm to estimate model's real-time performance during training. First, a sampling method generates two sets: one likely containing more outliers and the other more inliers, then a metric based on gradient cohesion is applied to probe into current training dynamics, which reflects model's performance on OD task. Experimental results on 4 deep UOD algorithms and 47 real-world datasets and theoretical proofs demonstrate the effectiveness of our proposed early stopping algorithm in enhancing the performance of deep UOD models. Auto Encoder (AE) enhanced by GradStop achieves better performance than itself, other SOTA UOD methods, and even ensemble AEs. Our method provides a robust and effective solution to the problem of performance degradation during training, enabling deep UOD models to achieve better potential in anomaly detection tasks.

Related papers

• E2EDiff: Direct Mapping from Noise to Data for Enhanced Diffusion Models [15.270657838960114]
  Diffusion models have emerged as a powerful framework for generative modeling, achieving state-of-the-art performance across various tasks. They face several inherent limitations, including a training-sampling gap, information leakage in the progressive noising process, and the inability to incorporate advanced loss functions like perceptual and adversarial losses during training. We propose an innovative end-to-end training framework that aligns the training and sampling processes by directly optimizing the final reconstruction output.
  arXiv  Detail & Related papers  (2024-12-30T16:06:31Z)
• ALTBI: Constructing Improved Outlier Detection Models via Optimization of Inlier-Memorization Effect [2.3961612657966946]
  Outlier detection (OD) is the task of identifying unusual observations (or outliers) from a given or upcoming data. Inlier-memorization (IM) effect suggests that generative models memorize inliers before outliers in early learning stages. We propose a theoretically principled method to address UOD tasks by maximally utilizing the IM effect.
  arXiv  Detail & Related papers  (2024-08-19T08:40:53Z)
• HUWSOD: Holistic Self-training for Unified Weakly Supervised Object Detection [66.42229859018775]
  We introduce a unified, high-capacity weakly supervised object detection (WSOD) network called HUWSOD. HUWSOD incorporates a self-supervised proposal generator and an autoencoder proposal generator with a multi-rate re-supervised pyramid to replace traditional object proposals. Our findings indicate that randomly boxes, although significantly different from well-designed offline object proposals, are effective for WSOD training.
  arXiv  Detail & Related papers  (2024-06-27T17:59:49Z)
• EntropyStop: Unsupervised Deep Outlier Detection with Loss Entropy [19.154826741973277]
  We propose a zero-label entropy metric named Loss Entropy for loss distribution, enabling us to infer optimal stopping points for training without labels. We also develop an automated early-stopping algorithm, EntropyStop, which halts training when loss entropy suggests the maximum model detection capability.
  arXiv  Detail & Related papers  (2024-05-21T05:17:43Z)
• Semi-supervised Open-World Object Detection [74.95267079505145]
  We introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD) We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Our experiments on 4 datasets including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-02-25T07:12:51Z)
• Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective [142.36200080384145]
  We propose a single objective which jointly optimize a latent-space model and policy to achieve high returns while remaining self-consistent. We demonstrate that the resulting algorithm matches or improves the sample-efficiency of the best prior model-based and model-free RL methods.
  arXiv  Detail & Related papers  (2022-09-18T03:51:58Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection [63.253850875265115]
  Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples. We propose a modular acceleration system, called SUOD, to address it.
  arXiv  Detail & Related papers  (2020-03-11T00:22:50Z)
• Adversarial Distributional Training for Robust Deep Learning [53.300984501078126]
  Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples. Most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks. In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models.
  arXiv  Detail & Related papers  (2020-02-14T12:36:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.