HalluCounter: Reference-free LLM Hallucination Detection in the Wild!

• URL: http://arxiv.org/abs/2503.04615v1
• Date: Thu, 06 Mar 2025 16:59:18 GMT
• Title: HalluCounter: Reference-free LLM Hallucination Detection in the Wild!
• Authors: Ashok Urlana, Gopichand Kanumolu, Charaka Vinayak Kumar, Bala Mallikarjunarao Garlapati, Rahul Mishra,
• Abstract summary: HalluCounter is a reference-free hallucination detection method that utilizes both response-response and query-response consistency and alignment patterns.<n>Our method outperforms state-of-the-art approaches by a significant margin, achieving over 90% average confidence in hallucination detection across datasets.
• Score: 6.5037356041929675
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Response consistency-based, reference-free hallucination detection (RFHD) methods do not depend on internal model states, such as generation probabilities or gradients, which Grey-box models typically rely on but are inaccessible in closed-source LLMs. However, their inability to capture query-response alignment patterns often results in lower detection accuracy. Additionally, the lack of large-scale benchmark datasets spanning diverse domains remains a challenge, as most existing datasets are limited in size and scope. To this end, we propose HalluCounter, a novel reference-free hallucination detection method that utilizes both response-response and query-response consistency and alignment patterns. This enables the training of a classifier that detects hallucinations and provides a confidence score and an optimal response for user queries. Furthermore, we introduce HalluCounterEval, a benchmark dataset comprising both synthetically generated and human-curated samples across multiple domains. Our method outperforms state-of-the-art approaches by a significant margin, achieving over 90\% average confidence in hallucination detection across datasets.

Related papers

• Enhancing Hallucination Detection through Noise Injection [9.582929634879932]
  Large Language Models (LLMs) are prone to generating plausible yet incorrect responses, known as hallucinations. We show that detection can be improved significantly by taking into account model uncertainty in the Bayesian sense. We propose a very simple and efficient approach that perturbs an appropriate subset of model parameters, or equivalently hidden unit activations, during sampling.
  arXiv  Detail & Related papers  (2025-02-06T06:02:20Z)
• Cost-Effective Hallucination Detection for LLMs [11.58436181159839]
  Large language models (LLMs) can be prone to hallucinations - generating unreliable outputs that are unfaithful to their inputs, external facts or internally inconsistent. Our pipeline for hallucination detection entails: first, producing a confidence score representing the likelihood that a generated answer is a hallucination; second, calibrating the score conditional on attributes of the inputs and candidate response; third, performing detection by thresholding the score.
  arXiv  Detail & Related papers  (2024-07-31T08:19:06Z)
• Mitigating LLM Hallucinations via Conformal Abstention [70.83870602967625]
  We develop a principled procedure for determining when a large language model should abstain from responding in a general domain. We leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate) Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets.
  arXiv  Detail & Related papers  (2024-04-04T11:32:03Z)
• HypoTermQA: Hypothetical Terms Dataset for Benchmarking Hallucination Tendency of LLMs [0.0]
  Hallucinations pose a significant challenge to the reliability and alignment of Large Language Models (LLMs) This paper introduces an automated scalable framework that combines benchmarking LLMs' hallucination tendencies with efficient hallucination detection. The framework is domain-agnostic, allowing the use of any language model for benchmark creation or evaluation in any domain.
  arXiv  Detail & Related papers  (2024-02-25T22:23:37Z)
• Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus [99.33091772494751]
  Large Language Models (LLMs) have gained significant popularity for their impressive performance across diverse fields. LLMs are prone to hallucinate untruthful or nonsensical outputs that fail to meet user expectations. We propose a novel reference-free, uncertainty-based method for detecting hallucinations in LLMs.
  arXiv  Detail & Related papers  (2023-11-22T08:39:17Z)
• Chainpoll: A high efficacy method for LLM hallucination detection [0.0]
  We introduce ChainPoll, an innovative hallucination detection method that excels compared to its counterparts. We also unveil RealHall, a refined collection of benchmark datasets to assess hallucination detection metrics from recent studies.
  arXiv  Detail & Related papers  (2023-10-22T14:45:14Z)
• ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks [91.55895047448249]
  This paper presents ReEval, an LLM-based framework using prompt chaining to perturb the original evidence for generating new test cases. We implement ReEval using ChatGPT and evaluate the resulting variants of two popular open-domain QA datasets. Our generated data is human-readable and useful to trigger hallucination in large language models.
  arXiv  Detail & Related papers  (2023-10-19T06:37:32Z)
• A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection [63.56136319976554]
  Large Language Models (LLMs) generate hallucinations, which can cause significant damage when deployed for mission-critical tasks. We propose a self-check approach based on reverse validation to detect factual errors automatically in a zero-resource fashion. We empirically evaluate our method and existing zero-resource detection methods on two datasets.
  arXiv  Detail & Related papers  (2023-10-10T10:14:59Z)
• AutoHall: Automated Hallucination Dataset Generation for Large Language Models [56.92068213969036]
  This paper introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. We also propose a zero-resource and black-box hallucination detection method based on self-contradiction.
  arXiv  Detail & Related papers  (2023-09-30T05:20:02Z)
• Detecting Hallucinated Content in Conditional Neural Sequence Generation [165.68948078624499]
  We propose a task to predict whether each token in the output sequence is hallucinated (not contained in the input) We also introduce a method for learning to detect hallucinations using pretrained language models fine tuned on synthetic data.
  arXiv  Detail & Related papers  (2020-11-05T00:18:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.