MSS-PAE: Saving Autoencoder-based Outlier Detection from Unexpected Reconstruction

• URL: http://arxiv.org/abs/2304.00709v3
• Date: Tue, 9 Jul 2024 06:08:17 GMT
• Title: MSS-PAE: Saving Autoencoder-based Outlier Detection from Unexpected Reconstruction
• Authors: Xu Tan, Jiawei Yang, Junqi Chen, Sylwan Rahardja, Susanto Rahardja,
• Abstract summary: AutoEncoders (AEs) are commonly used for machine learning tasks due to their intrinsic learning ability. AE-based methods face the issue of overconfident decisions and unexpected reconstruction results of outliers, limiting their performance in Outlier Detection (OD) The proposed methods have the potential to advance AE's development in OD.
• Score: 25.60381244912307
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AutoEncoders (AEs) are commonly used for machine learning tasks due to their intrinsic learning ability. This unique characteristic can be capitalized for Outlier Detection (OD). However conventional AE-based methods face the issue of overconfident decisions and unexpected reconstruction results of outliers, limiting their performance in OD. To mitigate these issues, the Mean Squared Error (MSE) and Negative Logarithmic Likelihood (NLL) were firstly analyzed, and the importance of incorporating aleatoric uncertainty to AE-based OD was elucidated. Then the Weighted Negative Logarithmic Likelihood (WNLL) was proposed to adjust for the effect of uncertainty for different OD scenarios. Moreover, the Mean-Shift Scoring (MSS) method was proposed to utilize the local relationship of data to reduce the issue of false inliers caused by AE. Experiments on 32 real-world OD datasets proved the effectiveness of the proposed methods. The combination of WNLL and MSS achieved 41% relative performance improvement compared to the best baseline. In addition, MSS improved the detection performance of multiple AE-based outlier detectors by an average of 20%. The proposed methods have the potential to advance AE's development in OD.

Related papers

• DDAD: A Two-pronged Adversarial Defense Based on Distributional Discrepancy [30.502354813427523]
  Statistical adversarial data detection (SADD) detects whether an upcoming batch contains adversarial examples (AEs) In this paper, we show that minimizing distributional discrepancy can help reduce the expected loss on AEs. We propose a two-pronged adversarial defense method, named Distributional-Discrepancy-based Adversarial Defense (DDAD)
  arXiv  Detail & Related papers  (2025-03-04T01:16:21Z)
• Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems [89.35169042718739]
  collaborative inference enables end users to leverage powerful deep learning models without exposure of sensitive raw data to cloud servers. Recent studies have revealed that these intermediate features may not sufficiently preserve privacy, as information can be leaked and raw data can be reconstructed via model inversion attacks (MIAs) This work first theoretically proves that the conditional entropy of inputs given intermediate features provides a guaranteed lower bound on the reconstruction mean square error (MSE) under any MIA. Then, we derive a differentiable and solvable measure for bounding this conditional entropy based on the Gaussian mixture estimation and propose a conditional entropy algorithm to enhance the inversion robustness
  arXiv  Detail & Related papers  (2025-03-01T07:15:21Z)
• Enhanced Extractor-Selector Framework and Symmetrization Weighted Binary Cross-Entropy for Edge Detections [0.0]
  Recent advancements have demonstrated the effectiveness of the extractor-selector (E-S) framework in edge detection (ED) tasks. We propose an enhanced E-S architecture, which utilizes richer, less-loss feature representations. We introduce a novel loss function, the Symmetrization Weight Binary Cross-Entropy (SWBCE), which simultaneously emphasizes both the recall of edge pixels and the suppression of erroneous edge predictions.
  arXiv  Detail & Related papers  (2025-01-23T04:10:31Z)
• Revisiting Essential and Nonessential Settings of Evidential Deep Learning [70.82728812001807]
  Evidential Deep Learning (EDL) is an emerging method for uncertainty estimation. We propose Re-EDL, a simplified yet more effective variant of EDL.
  arXiv  Detail & Related papers  (2024-10-01T04:27:07Z)
• Margin-bounded Confidence Scores for Out-of-Distribution Detection [2.373572816573706]
  We propose a novel method called Margin bounded Confidence Scores (MaCS) to address the nontrivial OOD detection problem. MaCS enlarges the disparity between ID and OOD scores, which in turn makes the decision boundary more compact. Experiments on various benchmark datasets for image classification tasks demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2024-09-22T05:40:25Z)
• ALTBI: Constructing Improved Outlier Detection Models via Optimization of Inlier-Memorization Effect [2.3961612657966946]
  Outlier detection (OD) is the task of identifying unusual observations (or outliers) from a given or upcoming data. Inlier-memorization (IM) effect suggests that generative models memorize inliers before outliers in early learning stages. We propose a theoretically principled method to address UOD tasks by maximally utilizing the IM effect.
  arXiv  Detail & Related papers  (2024-08-19T08:40:53Z)
• Semi-supervised Open-World Object Detection [74.95267079505145]
  We introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD) We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Our experiments on 4 datasets including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-02-25T07:12:51Z)
• Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [61.94125052118442]
  Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL) In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
  arXiv  Detail & Related papers  (2023-03-03T16:12:59Z)
• Mitigating Algorithmic Bias with Limited Annotations [65.060639928772]
  When sensitive attributes are not disclosed or available, it is needed to manually annotate a small part of the training data to mitigate bias. We propose Active Penalization Of Discrimination (APOD), an interactive framework to guide the limited annotations towards maximally eliminating the effect of algorithmic bias. APOD shows comparable performance to fully annotated bias mitigation, which demonstrates that APOD could benefit real-world applications when sensitive information is limited.
  arXiv  Detail & Related papers  (2022-07-20T16:31:19Z)
• Detecting and Recovering Adversarial Examples from Extracting Non-robust and Highly Predictive Adversarial Perturbations [15.669678743693947]
  adversarial examples (AEs) are maliciously designed to fool target models. Deep neural networks (DNNs) have been shown to be vulnerable against adversarial examples. We propose a model-free AEs detection method, the whole process of which is free from querying the victim model.
  arXiv  Detail & Related papers  (2022-06-30T08:48:28Z)
• Diminishing Empirical Risk Minimization for Unsupervised Anomaly Detection [0.0]
  Empirical Risk Minimization (ERM) assumes that the performance of an algorithm on an unknown distribution can be approximated by averaging losses on the known training set. We propose a novel Diminishing Empirical Risk Minimization (DERM) framework to break through the limitations of ERM. DERM adaptively adjusts the impact of individual losses through a well-devised aggregation strategy.
  arXiv  Detail & Related papers  (2022-05-29T14:18:26Z)
• Scale-Equivalent Distillation for Semi-Supervised Object Detection [57.59525453301374]
  Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, generating hard pseudo-labels by a teacher model on unlabeled data as supervisory signals. We analyze the challenges these methods meet with the empirical experiment results. We introduce a novel approach, Scale-Equivalent Distillation (SED), which is a simple yet effective end-to-end knowledge distillation framework robust to large object size variance and class imbalance.
  arXiv  Detail & Related papers  (2022-03-23T07:33:37Z)
• Partial Identification with Noisy Covariates: A Robust Optimization Approach [94.10051154390237]
  Causal inference from observational datasets often relies on measuring and adjusting for covariates. We show that this robust optimization approach can extend a wide range of causal adjustment methods to perform partial identification. Across synthetic and real datasets, we find that this approach provides ATE bounds with a higher coverage probability than existing methods.
  arXiv  Detail & Related papers  (2022-02-22T04:24:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.