FLatS: Principled Out-of-Distribution Detection with Feature-Based Likelihood Ratio Score

• URL: http://arxiv.org/abs/2310.05083v1
• Date: Sun, 8 Oct 2023 09:16:46 GMT
• Title: FLatS: Principled Out-of-Distribution Detection with Feature-Based Likelihood Ratio Score
• Authors: Haowei Lin and Yuntian Gu
• Abstract summary: FLatS is a principled solution for OOD detection based on likelihood ratio. We demonstrate that FLatS can serve as a general framework capable of enhancing other OOD detection methods. Experiments show that FLatS establishes a new SOTA on popular benchmarks.
• Score: 2.9914612342004503
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting out-of-distribution (OOD) instances is crucial for NLP models in practical applications. Although numerous OOD detection methods exist, most of them are empirical. Backed by theoretical analysis, this paper advocates for the measurement of the "OOD-ness" of a test case $\boldsymbol{x}$ through the likelihood ratio between out-distribution $\mathcal P_{\textit{out}}$ and in-distribution $\mathcal P_{\textit{in}}$. We argue that the state-of-the-art (SOTA) feature-based OOD detection methods, such as Maha and KNN, are suboptimal since they only estimate in-distribution density $p_{\textit{in}}(\boldsymbol{x})$. To address this issue, we propose FLatS, a principled solution for OOD detection based on likelihood ratio. Moreover, we demonstrate that FLatS can serve as a general framework capable of enhancing other OOD detection methods by incorporating out-distribution density $p_{\textit{out}}(\boldsymbol{x})$ estimation. Experiments show that FLatS establishes a new SOTA on popular benchmarks. Our code is publicly available at https://github.com/linhaowei1/FLatS.

Related papers

• Scalable Ensemble Diversification for OOD Generalization and Detection [68.8982448081223]
  SED identifies hard training samples on the fly and encourages the ensemble members to disagree on these. We show how to avoid the expensive computations in existing methods of exhaustive pairwise disagreements across models. For OOD generalization, we observe large benefits from the diversification in multiple settings including output-space (classical) ensembles and weight-space ensembles (model soups)
  arXiv  Detail & Related papers  (2024-09-25T10:30:24Z)
• Rethinking Test-time Likelihood: The Likelihood Path Principle and Its Application to OOD Detection [5.747789057967598]
  We introduce the likelihood path (LPath) principle, generalizing the likelihood principle. This narrows the search for informative summary statistics down to the minimal sufficient statistics of VAEs' conditional likelihoods. The corresponding LPath algorithm demonstrates SOTA performances, even using simple and small VAEs with poor likelihood estimates.
  arXiv  Detail & Related papers  (2024-01-10T05:07:14Z)
• Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
  We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr) abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions. We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
  arXiv  Detail & Related papers  (2023-11-28T02:00:47Z)
• Falsehoods that ML researchers believe about OOD detection [0.24801933141734633]
  We list some falsehoods that machine learning researchers believe about density-based OOD detection. We propose a framework, the OOD proxy framework, to unify these methods.
  arXiv  Detail & Related papers  (2022-10-23T16:21:54Z)
• Holistic Sentence Embeddings for Better Out-of-Distribution Detection [12.640837452980332]
  We propose a simple embedding approach named Avg-Avg, which averages all token representations from each intermediate layer as the sentence embedding. Our analysis demonstrates that it indeed helps preserve general linguistic knowledge in fine-tuned PLMs and substantially benefits detecting background shifts.
  arXiv  Detail & Related papers  (2022-10-14T03:22:58Z)
• Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognition [80.07843757970923]
  We show that existing OOD detection methods suffer from significant performance degradation when the training set is long-tail distributed. We propose Partial and Asymmetric Supervised Contrastive Learning (PASCL), which explicitly encourages the model to distinguish between tail-class in-distribution samples and OOD samples. Our method outperforms previous state-of-the-art method by $1.29%$, $1.45%$, $0.69%$ anomaly detection false positive rate (FPR) and $3.24%$, $4.06%$, $7.89%$ in-distribution
  arXiv  Detail & Related papers  (2022-07-04T01:53:07Z)
• No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets [69.725266027309]
  Out-of-distribution detection is an important component of reliable ML systems. In this work, we show that none of these methods are inherently better at OOD detection than others on a standardized set of 16 pairs. We also show that a method outperforming another on a certain (ID, OOD) pair may not do so in a low-data regime.
  arXiv  Detail & Related papers  (2021-09-12T16:35:00Z)
• Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection [72.35532598131176]
  We propose an unsupervised method to detect OOD samples using a $k$-NN density estimate. We leverage a recent insight about label smoothing, which we call the emphLabel Smoothed Embedding Hypothesis We show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results.
  arXiv  Detail & Related papers  (2021-02-09T21:04:44Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.