ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection
- URL: http://arxiv.org/abs/2402.17888v1
- Date: Tue, 27 Feb 2024 21:02:47 GMT
- Title: ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection
- Authors: Bo Peng, Yadan Luo, Yonggang Zhang, Yixuan Li, Zhen Fang
- Abstract summary: Post-hoc out-of-distribution (OOD) detection has garnered intensive attention in reliable machine learning.
We propose a novel theoretical framework grounded in Bregman divergence to provide a unified perspective on density-based score design.
We show that our proposed textscConjNorm has established a new state-of-the-art in a variety of OOD detection setups.
- Score: 41.41164637577005
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Post-hoc out-of-distribution (OOD) detection has garnered intensive attention
in reliable machine learning. Many efforts have been dedicated to deriving
score functions based on logits, distances, or rigorous data distribution
assumptions to identify low-scoring OOD samples. Nevertheless, these estimate
scores may fail to accurately reflect the true data density or impose
impractical constraints. To provide a unified perspective on density-based
score design, we propose a novel theoretical framework grounded in Bregman
divergence, which extends distribution considerations to encompass an
exponential family of distributions. Leveraging the conjugation constraint
revealed in our theorem, we introduce a \textsc{ConjNorm} method, reframing
density function design as a search for the optimal norm coefficient $p$
against the given dataset. In light of the computational challenges of
normalization, we devise an unbiased and analytically tractable estimator of
the partition function using the Monte Carlo-based importance sampling
technique. Extensive experiments across OOD detection benchmarks empirically
demonstrate that our proposed \textsc{ConjNorm} has established a new
state-of-the-art in a variety of OOD detection setups, outperforming the
current best method by up to 13.25$\%$ and 28.19$\%$ (FPR95) on CIFAR-100 and
ImageNet-1K, respectively.
Related papers
- MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts [25.643876327918544]
Leveraging the models' outputs, specifically the logits, is a common approach to estimating the test accuracy of a pre-trained neural network on out-of-distribution samples.
Despite their ease of implementation and computational efficiency, current logit-based methods are vulnerable to overconfidence issues, leading to prediction bias.
We propose MaNo which applies a data-dependent normalization on the logits to reduce prediction bias and takes the $L_p$ norm of the matrix of normalized logits as the estimation score.
arXiv Detail & Related papers (2024-05-29T10:45:06Z) - Distributed Markov Chain Monte Carlo Sampling based on the Alternating
Direction Method of Multipliers [143.6249073384419]
In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers.
We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art.
In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods.
arXiv Detail & Related papers (2024-01-29T02:08:40Z) - Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr)
abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions.
We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
arXiv Detail & Related papers (2023-11-28T02:00:47Z) - Topology-Matching Normalizing Flows for Out-of-Distribution Detection in
Robot Learning [38.97407602443256]
A powerful approach for Out-of-Distribution (OOD) detection is based on density estimation with Normalizing Flows (NFs)
In this work, we circumvent this topological mismatch using an expressive class-conditional base distribution trained with an information-theoretic objective to match the required topology.
We demonstrate superior results in density estimation and 2D object detection benchmarks in comparison with extensive baselines.
arXiv Detail & Related papers (2023-11-11T05:09:31Z) - Out-of-distribution Object Detection through Bayesian Uncertainty
Estimation [10.985423935142832]
We propose a novel, intuitive, and scalable probabilistic object detection method for OOD detection.
Our method is able to distinguish between in-distribution (ID) data and OOD data via weight parameter sampling from proposed Gaussian distributions.
We demonstrate that our Bayesian object detector can achieve satisfactory OOD identification performance by reducing the FPR95 score by up to 8.19% and increasing the AUROC score by up to 13.94% when trained on BDD100k and VOC datasets.
arXiv Detail & Related papers (2023-10-29T19:10:52Z) - Out-of-Distribution Detection with Class Ratio Estimation [4.930817402876787]
Density-based Out-of-distribution (OOD) detection has recently been shown unreliable for the task of detecting OOD images.
We propose to unify density ratio based methods under a novel framework that builds energy-based models and employs differing base distributions.
arXiv Detail & Related papers (2022-06-08T15:20:49Z) - A Unified Framework for Multi-distribution Density Ratio Estimation [101.67420298343512]
Binary density ratio estimation (DRE) provides the foundation for many state-of-the-art machine learning algorithms.
We develop a general framework from the perspective of Bregman minimization divergence.
We show that our framework leads to methods that strictly generalize their counterparts in binary DRE.
arXiv Detail & Related papers (2021-12-07T01:23:20Z) - Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic
Uncertainty [58.144520501201995]
Bi-Lipschitz regularization of neural network layers preserve relative distances between data instances in the feature spaces of each layer.
With the use of an attentive set encoder, we propose to meta learn either diagonal or diagonal plus low-rank factors to efficiently construct task specific covariance matrices.
We also propose an inference procedure which utilizes scaled energy to achieve a final predictive distribution.
arXiv Detail & Related papers (2021-10-12T22:04:19Z) - Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection [72.35532598131176]
We propose an unsupervised method to detect OOD samples using a $k$-NN density estimate.
We leverage a recent insight about label smoothing, which we call the emphLabel Smoothed Embedding Hypothesis
We show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results.
arXiv Detail & Related papers (2021-02-09T21:04:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.