Zero-source LLM Hallucination Detection with Human-like Criteria Probing

• URL: http://arxiv.org/abs/2606.12900v1
• Date: Thu, 11 Jun 2026 04:58:05 GMT
• Title: Zero-source LLM Hallucination Detection with Human-like Criteria Probing
• Abstract summary: Large language models (LLMs) often hallucinate by generating factually incorrect or unfaithful content.<n>We propose Human-like Criteria Probing for Hallucination Detection (HCPD)<n>HCPD is a paradigm that emulates the multi-faceted reasoning of human evaluators.
• Score: 43.90784659213977
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) often hallucinate by generating factually incorrect or unfaithful content, posing significant risks to their safe use. Detecting such hallucinations is particularly challenging under the zero-source constraint, where no model internals or external references are available, and detection must rely solely on the textual query-answer pair. In this paper, we propose Human-like Criteria Probing for Hallucination Detection (HCPD), a paradigm that emulates the multi-faceted reasoning of human evaluators. Its core is a Human-like Criteria Probing (HCP) mechanism, in which a LLM agent adaptively decomposes its judgment into a weighted set of interpretable criteria and aggregates criterion-specific scores into a final truthfulness measure. To achieve this adaptive capability, we introduce a reward-based alignment scheme using only weak supervision from semantic consistency. At inference, we employ a multi-sampling aggregation strategy to ensure robust decisions while preserving full interpretability. We further provide theoretical analysis supporting the reliability of our approach. Extensive experiments show that HCPD consistently outperforms state-of-the-art baselines, offering an effective and explainable solution for zero-source hallucination detection. Code is available at https://github.com/TRISKEL10N/HCPD.

Related papers

• Chain-based Adaptive Reconfiguration Over Lattices for Hallucination Reduction [0.0]
  CAROL is a probabilistic framework for test-time hallucination reduction in large language models.<n>CAROL defines a semantic uncertainty measure based on the consistency between generated responses and a trusted context.<n>CAROL unifies hallucination detection and mitigation within a single framework.
  arXiv  Detail & Related papers  (2026-05-26T21:28:51Z)
• Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models [53.15468578562038]
  We propose an Uncertainty-aware Exploratory Direct Preference Optimization (UE-DPO) method for Multimodal Large Language Models (MLLMs)<n>We first quantify the uncertainty from the model's failure to ground token predictions in the given image.<n>We then encourage more learning pressure on visually deficient tokens in preferred samples, and alleviate the over-penalization of beneficial knowledge in dispreferred samples.
  arXiv  Detail & Related papers  (2026-05-06T13:08:12Z)
• Neural Uncertainty Principle: A Unified View of Adversarial Fragility and LLM Hallucination [60.197429875410286]
  Adversarial vulnerability in vision and hallucination in large language models are conventionally viewed as separate problems.<n>Formalizing a Neural Uncertainty Principle (NUP) under a loss-induced state, we find that in near-bound regimes, further compression must be accompanied by increased sensitivity dispersion.<n>In vision, masking highly coupled components improves robustness without costly adversarial training.<n>In language, the same prefill-stage probe detects hallucination risk before generating any answer tokens.
  arXiv  Detail & Related papers  (2026-03-20T02:07:10Z)
• Lowest Span Confidence: A Zero-Shot Metric for Efficient and Black-Box Hallucination Detection in LLMs [24.471653720056803]
  Hallucinations in Large Language Models (LLMs) generate plausible but non-factual content.<n>We propose a novel efficient zero-shot metric called Lowest Span Confidence (LSC) for hallucination detection under minimal resource assumptions.<n>LSC consistently outperforms existing zero-shot baselines, delivering strong detection performance even under resource-constrained conditions.
  arXiv  Detail & Related papers  (2026-01-07T12:48:33Z)
• SeSE: A Structural Information-Guided Uncertainty Quantification Framework for Hallucination Detection in LLMs [43.76748192880071]
  This paper presents a principled UQ framework that quantifies the inherent semantic uncertainty of large language models.<n>We develop an adaptively sparsified directed semantic graph construction algorithm that captures directional semantic dependencies.<n>We then exploit latent semantic structural information through hierarchical abstraction.
  arXiv  Detail & Related papers  (2025-11-20T11:54:12Z)
• Revisiting Hallucination Detection with Effective Rank-based Uncertainty [10.775061161282053]
  We propose a simple yet powerful method that quantifies uncertainty by measuring the effective rank of hidden states.<n>Grounded in the spectral analysis of representations, our approach provides interpretable insights into the model's internal reasoning process.<n>Our method effectively detects hallucinations and generalizes robustly across various scenarios.
  arXiv  Detail & Related papers  (2025-10-09T16:12:12Z)
• Semantic Energy: Detecting LLM Hallucination Beyond Entropy [106.92072182161712]
  Large Language Models (LLMs) are being increasingly deployed in real-world applications, but they remain susceptible to hallucinations.<n>Uncertainty estimation is a feasible approach to detect such hallucinations.<n>We introduce Semantic Energy, a novel uncertainty estimation framework.
  arXiv  Detail & Related papers  (2025-08-20T07:33:50Z)
• MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them [52.764019220214344]
  Hallucinations pose critical risks for large language model (LLM)-based agents.<n>We present MIRAGE-Bench, the first unified benchmark for eliciting and evaluating hallucinations in interactive environments.
  arXiv  Detail & Related papers  (2025-07-28T17:38:29Z)
• INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection [39.52923659121416]
  We propose to explore the dense semantic information retained within textbfINternal textbfStates for halluctextbfInation textbfDEtection. A simple yet effective textbfEigenScore metric is proposed to better evaluate responses' self-consistency. A test time feature clipping approach is explored to truncate extreme activations in the internal states.
  arXiv  Detail & Related papers  (2024-02-06T06:23:12Z)
• A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection [63.56136319976554]
  Large Language Models (LLMs) generate hallucinations, which can cause significant damage when deployed for mission-critical tasks. We propose a self-check approach based on reverse validation to detect factual errors automatically in a zero-resource fashion. We empirically evaluate our method and existing zero-resource detection methods on two datasets.
  arXiv  Detail & Related papers  (2023-10-10T10:14:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.