How Good Are LLMs at Out-of-Distribution Detection?

• URL: http://arxiv.org/abs/2308.10261v4
• Date: Tue, 16 Apr 2024 11:38:35 GMT
• Title: How Good Are LLMs at Out-of-Distribution Detection?
• Authors: Bo Liu, Liming Zhan, Zexin Lu, Yujie Feng, Lei Xue, Xiao-Ming Wu,
• Abstract summary: Out-of-distribution (OOD) detection plays a vital role in enhancing the reliability of machine learning (ML) models. This paper embarks on a pioneering empirical investigation of OOD detection in the domain of large language models (LLMs)
• Score: 13.35571704613836
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Out-of-distribution (OOD) detection plays a vital role in enhancing the reliability of machine learning (ML) models. The emergence of large language models (LLMs) has catalyzed a paradigm shift within the ML community, showcasing their exceptional capabilities across diverse natural language processing tasks. While existing research has probed OOD detection with relative small-scale Transformers like BERT, RoBERTa and GPT-2, the stark differences in scales, pre-training objectives, and inference paradigms call into question the applicability of these findings to LLMs. This paper embarks on a pioneering empirical investigation of OOD detection in the domain of LLMs, focusing on LLaMA series ranging from 7B to 65B in size. We thoroughly evaluate commonly-used OOD detectors, scrutinizing their performance in both zero-grad and fine-tuning scenarios. Notably, we alter previous discriminative in-distribution fine-tuning into generative fine-tuning, aligning the pre-training objective of LLMs with downstream tasks. Our findings unveil that a simple cosine distance OOD detector demonstrates superior efficacy, outperforming other OOD detectors. We provide an intriguing explanation for this phenomenon by highlighting the isotropic nature of the embedding spaces of LLMs, which distinctly contrasts with the anisotropic property observed in smaller BERT family models. The new insight enhances our understanding of how LLMs detect OOD data, thereby enhancing their adaptability and reliability in dynamic environments. We have released the source code at \url{https://github.com/Awenbocc/LLM-OOD} for other researchers to reproduce our results.

Related papers

• Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement [51.601916604301685]
  Large language models (LLMs) generate content that can undermine trust in online discourse. Current methods often focus on binary classification, failing to address the complexities of real-world scenarios like human-AI collaboration. To move beyond binary classification and address these challenges, we propose a new paradigm for detecting LLM-generated content.
  arXiv  Detail & Related papers  (2024-10-18T08:14:10Z)
• Large Language Models for Anomaly and Out-of-Distribution Detection: A Survey [18.570066068280212]
  Large Language Models (LLMs) have demonstrated their effectiveness not only in natural language processing but also in broader applications. This survey focuses on the problem of anomaly and OOD detection under the context of LLMs. We propose a new taxonomy to categorize existing approaches into two classes based on the role played by LLMs.
  arXiv  Detail & Related papers  (2024-09-03T15:22:41Z)
• Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector [17.305076703258813]
  We revisit the likelihood ratio between a pretrained large language model (LLM) and its finetuned variant as a criterion for out-of-distribution (OOD) detection. We show that, for the first time, the likelihood ratio can serve as an effective OOD detector.
  arXiv  Detail & Related papers  (2024-04-07T10:32:49Z)
• PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics [51.17512229589]
  PoLLMgraph is a model-based white-box detection and forecasting approach for large language models. We show that hallucination can be effectively detected by analyzing the LLM's internal state transition dynamics. Our work paves a new way for model-based white-box analysis of LLMs, motivating the research community to further explore, understand, and refine the intricate dynamics of LLM behaviors.
  arXiv  Detail & Related papers  (2024-04-06T20:02:20Z)
• Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension [63.330262740414646]
  We study how to characterize and predict the truthfulness of texts generated from large language models (LLMs) We suggest investigating internal activations and quantifying LLM's truthfulness using the local intrinsic dimension (LID) of model activations.
  arXiv  Detail & Related papers  (2024-02-28T04:56:21Z)
• Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection [34.135738700682055]
  This paper conducts a comprehensive evaluation of large language models (LLMs) represented by ChatGPT. We find that LLMs exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource.
  arXiv  Detail & Related papers  (2024-02-27T07:02:10Z)
• Assessing the Reliability of Large Language Model Knowledge [78.38870272050106]
  Large language models (LLMs) have been treated as knowledge bases due to their strong performance in knowledge probing tasks. How do we evaluate the capabilities of LLMs to consistently produce factually correct answers? We propose MOdel kNowledge relIabiliTy scORe (MONITOR), a novel metric designed to directly measure LLMs' factual reliability.
  arXiv  Detail & Related papers  (2023-10-15T12:40:30Z)
• Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection [67.68030805755679]
  Large language models (LLMs) encode a wealth of world knowledge and can be prompted to generate descriptive features for each class. In this paper, we propose to apply world knowledge to enhance OOD detection performance through selective generation from LLMs.
  arXiv  Detail & Related papers  (2023-10-12T04:14:28Z)
• Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations [111.88727295707454]
  This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We propose a benchmark construction protocol that ensures clear differentiation and challenging distribution shifts. We conduct experiments on pre-trained language models for analysis and evaluation of OOD robustness.
  arXiv  Detail & Related papers  (2023-06-07T17:47:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.