Kernel Token Contradiction: a Fast and Principled Approach for LLM Claim Uncertainty Quantification

• URL: http://arxiv.org/abs/2608.22506v1
• Date: Sun, 23 Aug 2026 16:55:02 GMT
• Title: Kernel Token Contradiction: a Fast and Principled Approach for LLM Claim Uncertainty Quantification
• Abstract summary: Claim-level Uncertainty Quantification (UQ) aims to mitigate the lack of reliability of Large Language Models (LLMs)<n>We introduce Kernel Token Contradiction (KTC), a lightweight approach to compute claim-level UQ under realistic white-box conditions.<n>Although CPU-only, our approach achieves over an 8.2x speedup compared to state-of-the-art GPU-accelerated methods.
• Score: 1.6916040234975798
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Claim-level Uncertainty Quantification (UQ) aims to mitigate the lack of reliability of Large Language Models (LLMs) by evaluating the factuality of each claim in their outputs. We introduce Kernel Token Contradiction (KTC), a lightweight approach to compute claim-level UQ under realistic white-box conditions. KTC represents the candidate tokens involved in LLM generation as a positive semi-definite kernel that integrates both the LLM's conditional distribution and a token contradiction score. We then use the Von Neumann entropy to quantify the uncertainty of this kernel. To estimate token contradiction, we develop a new approach based on frequency statistics from the Wikipedia corpus. Although CPU-only, our approach achieves over an 8.2x speedup compared to state-of-the-art GPU-accelerated methods based on cross-encoders, and over a 65x speedup compared to CPU-only methods with comparable performance. Our evaluation spans two benchmarks across four European languages and 16 different models. KTC not only matches the average performance of existing methods but also outperforms them in high-precision regimes. This combination of computational efficiency and accuracy makes real-time monitoring of LLM outputs practical in production.

Related papers

• CauTion: Knowing When to Trust LLMs for Ensemble Causal Discovery [51.07538881798502]
  Large language models (LLMs) offer a promising source of domain knowledge to complement statistical inference.<n>We propose CauTion, a framework that reliably integrates LLM domain knowledge into an ensemble of statistical causal discovery algorithms.<n>CauTion consistently outperforms both data-centric and LLM-augmented baselines.
  arXiv  Detail & Related papers  (2026-06-02T13:07:43Z)
• Fast-dLLM++: Fréchet Profile Decoding for Faster Diffusion LLM Inference [45.385819280687514]
  We propose textbfFast-dLLM++, a training-free extension that introduces emphFréchet profile decoding.<n>Fast-dLLM++ leaves the model, diffusion process, and cache implementation entirely unchanged.<n>Experiments on GSM8K, MATH, HumanEval, and MBPP with the LLaDA-8B model show that the theoretical improvement translates directly into empirical gains.
  arXiv  Detail & Related papers  (2026-06-01T23:18:59Z)
• OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond [50.440302567029654]
  Multi-modal intelligence has made the Key-Value cache a dominant memory bottleneck for efficient deployment.<n>In this work, we revisit the inherent limitations of the per-channel quantization paradigm.<n>We propose OScaR, an accurate and lightweight KV cache compression framework for X-LLMs.
  arXiv  Detail & Related papers  (2026-05-19T10:53:03Z)
• A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints [4.636275494777427]
  We introduce the first queueing-theoretic framework that explicitly incorporates both computation and GPU memory constraints.<n>We derive rigorous stability and instability conditions that determine whether an LLM inference service can sustain incoming demand without unbounded queue growth.<n>Our results show that the predicted stability conditions are highly accurate, with deviations typically within 10%.
  arXiv  Detail & Related papers  (2026-05-06T07:42:26Z)
• Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models [96.0074341403456]
  Inference-time compute has re-emerged as a practical way to improve LLM reasoning.<n>Most test-time scaling (TTS) algorithms rely on autoregressive decoding.<n>We propose Prism, an efficient TTS framework for dLLMs.
  arXiv  Detail & Related papers  (2026-02-02T09:14:51Z)
• Residual Context Diffusion Language Models [90.07635240595926]
  Residual Context Diffusion (RCD) is a module that converts discarded token representations into contextual residuals and injects them back for the next denoising step.<n>RCD consistently improves frontier dLLMs by 5-10 points in accuracy with minimal extra computation overhead.
  arXiv  Detail & Related papers  (2026-01-30T13:16:32Z)
• Counting Without Running: Evaluating LLMs' Reasoning About Code Complexity [2.7389338551082605]
  We develop a benchmark to test Large Language Models (LLMs) for anticipating performance bottlenecks.<n>FLOPBench predicts single and double-precision FLOP counts for 577 kernels.<n>Our results positionFLOPBench as a focused testbed for developing LLM tooling.
  arXiv  Detail & Related papers  (2025-12-04T01:03:20Z)
• MUCH: A Multilingual Claim Hallucination Benchmark [5.6001617185032595]
  Much is the first claim-level UQ benchmark designed for fair and reproducible evaluation of future methods.<n>It includes 4,873 samples across four European languages (English, French, Spanish, and German) and four instruction-tuned open-weight LLMs.
  arXiv  Detail & Related papers  (2025-11-21T09:37:16Z)
• Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling [90.86991492288487]
  evaluating constraint on every token can be prohibitively expensive.<n> LCD can distort the global distribution over strings, sampling tokens based only on local information.<n>We show that our approach is superior to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-07T18:30:18Z)
• Provable Scaling Laws for the Test-Time Compute of Large Language Models [84.00141420901038]
  We propose two algorithms that enjoy provable scaling laws for the test-time compute of large language models.<n>One is a two-stage knockout-style algorithm, where each candidate is evaluated by its average win rate against multiple opponents.<n>The other is a two-stage league-style algorithm, where each candidate is evaluated by its average win rate against multiple opponents.
  arXiv  Detail & Related papers  (2024-11-29T05:29:47Z)
• Supervised Kernel Thinning [6.6157730528755065]
  kernel thinning algorithm of Dwivedi & Mackey (2024) provides a better-than-i.i.d. compression of a generic set of points.<n>We generalize the KT algorithm to speed up supervised learning problems involving kernel methods.
  arXiv  Detail & Related papers  (2024-10-17T16:48:51Z)
• Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph [83.90988015005934]
  Uncertainty quantification is a key element of machine learning applications.<n>We introduce a novel benchmark that implements a collection of state-of-the-art UQ baselines.<n>We conduct a large-scale empirical investigation of UQ and normalization techniques across eleven tasks, identifying the most effective approaches.
  arXiv  Detail & Related papers  (2024-06-21T20:06:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.