ASK-NN: An Asymmetric Nearest-Neighbor Test that detects Distribution Drifts in Natural Language

• URL: http://arxiv.org/abs/2607.15607v1
• Date: Fri, 17 Jul 2026 04:06:22 GMT
• Title: ASK-NN: An Asymmetric Nearest-Neighbor Test that detects Distribution Drifts in Natural Language
• Abstract summary: Hallucinations and artificial text in LLM-generated outputs often appear as distributional deviations between prompt and response hidden-state distributions.<n>We consider an asymmetric two-sample test ASK-NN based on the k-nearest-neighbor graph.
• Score: 8.729988313517252
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hallucinations and artificial text in LLM-generated outputs often appear as distributional deviations between prompt and response hidden-state distributions. Since prompts or retrieved contexts typically serve as reference samples and responses as query samples, with major differences in length, these asymmetries motivate the use of change test statistics that treat the two samples differently. We consider an asymmetric two-sample test ASK-NN based on the directed k-nearest-neighbor graph. Our statistic counts reference points whose nearest neighbor in the pooled sample is also a reference point. Under the permutation null, it admits an exact finite-sample conditional mean and variance; we further establish asymptotic normality and consistency under fixed alternatives. ASK-NN is computationally effective and easy to implement. Empirically, it is competitive with kernel and graph-based baselines on synthetic benchmarks, artificial-text detection, and LLM hallucination detection from token-level hidden states.

Related papers

• Marginal Coordinate Test for Fréchet Regression with Random Objects [15.929485404318557]
  We develop a test for regression with Euclidean predictors and a random-object response in a separable metric space.<n>We establish a weighted centered chi-square null limit, wild bootstrap validity, and consistency against fixed detectable alternatives.<n>For simultaneous inference, truncated p-to-e calibration combined with e-BH provides false discovery rate control under general dependence.
  arXiv  Detail & Related papers  (2026-08-31T11:48:12Z)
• Zero-Flow Two-Sample Tests [47.04254380987003]
  We propose a new approach to two-sample testing for deciding whether two sets of samples are drawn from the same distribution.<n>The test is built on a statistical discrepancy based on the zero-flow criterion, termed zero-flow discrepancy (ZFD)<n>We prove the validity of ZFD and propose a practical testing procedure, termed the zero-flow two-sample test (ZF2ST)
  arXiv  Detail & Related papers  (2026-07-23T17:27:23Z)
• Novelty detection on path space [13.042396048879242]
  We frame novelty detection on path space as a hypothesis testing problem with signature-based test statistics.<n>We derive exact formulae for smooth surrogates of conditional value-at-risk (CVaR) in terms of expected signatures.<n>We evaluate numerically the type-$mathrmI$ error and statistical power of signature-based test statistic.
  arXiv  Detail & Related papers  (2025-12-02T21:25:03Z)
• A Sample Efficient Conditional Independence Test in the Presence of Discretization [54.047334792855345]
  Conditional Independence (CI) tests directly to discretized data can lead to incorrect conclusions.<n>Recent advancements have sought to infer the correct CI relationship between the latent variables through binarizing observed data.<n>Motivated by this, this paper introduces a sample-efficient CI test that does not rely on the binarization process.
  arXiv  Detail & Related papers  (2025-06-10T12:41:26Z)
• Statistical Hypothesis Testing for Auditing Robustness in Language Models [49.1574468325115]
  We introduce distribution-based perturbation analysis, a framework that reformulates perturbation analysis as a frequentist hypothesis testing problem.<n>We construct empirical null and alternative output distributions within a low-dimensional semantic similarity space via Monte Carlo sampling.<n>We show how we can quantify response changes, measure true/false positive rates, and evaluate alignment with reference models.
  arXiv  Detail & Related papers  (2025-06-09T17:11:07Z)
• Doubly Robust Conditional Independence Testing with Generative Neural Networks [8.323172773256449]
  This article addresses the problem of testing the conditional independence of two generic random vectors $X$ and $Y$ given a third random vector $Z$. We propose a new non-parametric testing procedure that avoids explicitly estimating any conditional distributions.
  arXiv  Detail & Related papers  (2024-07-25T01:28:59Z)
• Collaborative non-parametric two-sample testing [55.98760097296213]
  The goal is to identify nodes where the null hypothesis $p_v = q_v$ should be rejected. We propose the non-parametric collaborative two-sample testing (CTST) framework that efficiently leverages the graph structure. Our methodology integrates elements from f-divergence estimation, Kernel Methods, and Multitask Learning.
  arXiv  Detail & Related papers  (2024-02-08T14:43:56Z)
• Nearest-Neighbor Sampling Based Conditional Independence Testing [15.478671471695794]
  Conditional randomization test (CRT) was recently proposed to test whether two random variables X and Y are conditionally independent given random variables Z. The aim of this paper is to develop a novel alternative of CRT by using nearest-neighbor sampling without assuming the exact form of the distribution of X given Z.
  arXiv  Detail & Related papers  (2023-04-09T07:54:36Z)
• Active Sequential Two-Sample Testing [18.99517340397671]
  We consider the two-sample testing problem in a new scenario where sample measurements are inexpensive to access. We devise the first emphactiveNIST-sample testing framework that not only sequentially but also emphactively queries. In practice, we introduce an instantiation of our framework and evaluate it using several experiments.
  arXiv  Detail & Related papers  (2023-01-30T02:23:49Z)
• Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observations [44.71254888821376]
  We provide the first type-I-error and expected-rejection-time guarantees under general non-data generating processes. We show how to apply our results to inference on parameters defined by estimating equations, such as average treatment effects.
  arXiv  Detail & Related papers  (2022-12-29T18:37:08Z)
• Learning versus Refutation in Noninteractive Local Differential Privacy [133.80204506727526]
  We study two basic statistical tasks in non-interactive local differential privacy (LDP): learning and refutation. Our main result is a complete characterization of the sample complexity of PAC learning for non-interactive LDP protocols.
  arXiv  Detail & Related papers  (2022-10-26T03:19:24Z)
• Optimal Clustering with Bandit Feedback [57.672609011609886]
  This paper considers the problem of online clustering with bandit feedback. It includes a novel stopping rule for sequential testing that circumvents the need to solve any NP-hard weighted clustering problem as its subroutines. We show through extensive simulations on synthetic and real-world datasets that BOC's performance matches the lower boundally, and significantly outperforms a non-adaptive baseline algorithm.
  arXiv  Detail & Related papers  (2022-02-09T06:05:05Z)
• On Matched Filtering for Statistical Change Point Detection [13.64446865914411]
  Non-parametric randomness and distribution-free two-sample tests have been the foundation of many change point detection algorithms. We address these issues by deriving and applying filters matched to the expected temporal signatures of a change. Our method allows for the localization of change points without the use of ad-hoc post-processing.
  arXiv  Detail & Related papers  (2020-06-09T22:45:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.