Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection

• URL: http://arxiv.org/abs/2303.01201v1
• Date: Thu, 2 Mar 2023 12:34:38 GMT
• Title: Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection
• Authors: Zhen Cheng, Fei Zhu, Xu-Yao Zhang, Cheng-Lin Liu
• Abstract summary: We find the performance of OOD detection suffers from overfitting and instability during training. We propose Average of Pruning (AoP), consisting of model averaging and pruning, to mitigate the unstable behaviors.
• Score: 37.43981354073841
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting Out-of-distribution (OOD) inputs have been a critical issue for neural networks in the open world. However, the unstable behavior of OOD detection along the optimization trajectory during training has not been explored clearly. In this paper, we first find the performance of OOD detection suffers from overfitting and instability during training: 1) the performance could decrease when the training error is near zero, and 2) the performance would vary sharply in the final stage of training. Based on our findings, we propose Average of Pruning (AoP), consisting of model averaging and pruning, to mitigate the unstable behaviors. Specifically, model averaging can help achieve a stable performance by smoothing the landscape, and pruning is certified to eliminate the overfitting by eliminating redundant features. Comprehensive experiments on various datasets and architectures are conducted to verify the effectiveness of our method.

Related papers

• Gradient Short-Circuit: Efficient Out-of-Distribution Detection via Feature Intervention [19.580332929984028]
  Out-of-Distribution (OOD) detection is critical for safely deploying deep models in open-world environments.<n>We propose an inference-stage technique to short-circuit those feature coordinates that spurious gradients exploit.<n> Experiments on standard OOD benchmarks show our approach yields substantial improvements.
  arXiv  Detail & Related papers  (2025-07-02T07:18:09Z)
• Efficient Test-time Adaptive Object Detection via Sensitivity-Guided Pruning [73.40364018029673]
  Continual test-time adaptive object detection (CTTA-OD) aims to online adapt a source pre-trained detector to ever-changing environments.<n>Our motivation stems from the observation that not all learned source features are beneficial.<n>Our method achieves superior adaptation performance while reducing computational overhead by 12% in FLOPs.
  arXiv  Detail & Related papers  (2025-06-03T05:27:56Z)
• Leveraging Perturbation Robustness to Enhance Out-of-Distribution Detection [15.184096796229115]
  We propose a post-hoc method, Perturbation-Rectified OOD detection (PRO), based on the insight that prediction confidence for OOD inputs is more susceptible to reduction under perturbation than in-distribution (IND) inputs. On a CIFAR-10 model with adversarial training, PRO effectively detects near-OOD inputs, achieving a reduction of more than 10% on FPR@95 compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-24T15:32:33Z)
• Distributionally Robust Reinforcement Learning with Human Feedback [13.509499718691016]
  We introduce a distributionally robust RLHF for fine-tuning large language models. Our goal is to ensure that a fine-tuned model retains its performance even when the distribution of prompts significantly differs. We show that our robust training improves the accuracy of the learned reward models on average, and markedly on some tasks, such as reasoning.
  arXiv  Detail & Related papers  (2025-03-01T15:43:39Z)
• DRoP: Distributionally Robust Pruning [11.930434318557156]
  We conduct the first systematic study of the impact of data pruning on classification bias of trained models. We propose DRoP, a distributionally robust approach to pruning and empirically demonstrate its performance on standard computer vision benchmarks.
  arXiv  Detail & Related papers  (2024-04-08T14:55:35Z)
• Mahalanobis-Aware Training for Out-of-Distribution Detection [0.11510009152620666]
  We present a novel loss function and recipe for training networks with improved density-based out-of-distribution sensitivity. We demonstrate the effectiveness of our method on CIFAR-10, notably reducing the false-positive rate of the relative Mahalanobis distance method on far-OOD tasks by over 50%.
  arXiv  Detail & Related papers  (2023-11-01T19:46:40Z)
• On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion [46.30241353155658]
  Generalizing deep learning models to unknown target domain distribution with low latency has motivated research into test-time training/adaptation (TTT/TTA) Many state-of-the-art methods fail to maintain the performance when the target domain is contaminated with strong out-of-distribution (OOD) data. We develop an adaptive strong OOD pruning which improves the efficacy of the self-training TTT method. We regularize self-training with distribution alignment and the combination yields the state-of-the-art performance on 5 OWTTT benchmarks.
  arXiv  Detail & Related papers  (2023-08-19T08:27:48Z)
• LINe: Out-of-Distribution Detection by Leveraging Important Neurons [15.797257361788812]
  We introduce a new aspect for analyzing the difference in model outputs between in-distribution data and OOD data. We propose a novel method, Leveraging Important Neurons (LINe), for post-hoc Out of distribution detection.
  arXiv  Detail & Related papers  (2023-03-24T13:49:05Z)
• AUTO: Adaptive Outlier Optimization for Online Test-Time OOD Detection [81.49353397201887]
  Out-of-distribution (OOD) detection is crucial to deploying machine learning models in open-world applications. We introduce a novel paradigm called test-time OOD detection, which utilizes unlabeled online data directly at test time to improve OOD detection performance. We propose adaptive outlier optimization (AUTO), which consists of an in-out-aware filter, an ID memory bank, and a semantically-consistent objective.
  arXiv  Detail & Related papers  (2023-03-22T02:28:54Z)
• To be Critical: Self-Calibrated Weakly Supervised Learning for Salient Object Detection [95.21700830273221]
  Weakly-supervised salient object detection (WSOD) aims to develop saliency models using image-level annotations. We propose a self-calibrated training strategy by explicitly establishing a mutual calibration loop between pseudo labels and network predictions. We prove that even a much smaller dataset with well-matched annotations can facilitate models to achieve better performance as well as generalizability.
  arXiv  Detail & Related papers  (2021-09-04T02:45:22Z)
• NoiER: An Approach for Training more Reliable Fine-TunedDownstream Task Models [54.184609286094044]
  We propose noise entropy regularisation (NoiER) as an efficient learning paradigm that solves the problem without auxiliary models and additional data. The proposed approach improved traditional OOD detection evaluation metrics by 55% on average compared to the original fine-tuned models.
  arXiv  Detail & Related papers  (2021-08-29T06:58:28Z)
• On the Practicality of Deterministic Epistemic Uncertainty [106.06571981780591]
  deterministic uncertainty methods (DUMs) achieve strong performance on detecting out-of-distribution data. It remains unclear whether DUMs are well calibrated and can seamlessly scale to real-world applications.
  arXiv  Detail & Related papers  (2021-07-01T17:59:07Z)
• Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
  We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs. We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
  arXiv  Detail & Related papers  (2020-03-21T17:46:28Z)
• Stability for the Training of Deep Neural Networks and Other Classifiers [0.9558392439655015]
  We formalize the notion of stability, and provide examples of instability. Our results do not depend on the algorithm used for training, as long as loss decreases with training.
  arXiv  Detail & Related papers  (2020-02-10T22:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.