Regularized $f$-Divergence Kernel Tests

• URL: http://arxiv.org/abs/2601.19755v1
• Date: Tue, 27 Jan 2026 16:15:48 GMT
• Title: Regularized $f$-Divergence Kernel Tests
• Authors: Mónica Ribero, Antonin Schrab, Arthur Gretton,
• Abstract summary: We propose a framework to construct practical kernel-based two-sample tests from the family of $f$-divergences.<n>We provide theoretical guarantees for statistical test power across our family of $f$-divergence estimates.<n>For machine unlearning, we propose a relative test that distinguishes true unlearning failures from safe distributional variations.
• Score: 24.182732872327183
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a framework to construct practical kernel-based two-sample tests from the family of $f$-divergences. The test statistic is computed from the witness function of a regularized variational representation of the divergence, which we estimate using kernel methods. The proposed test is adaptive over hyperparameters such as the kernel bandwidth and the regularization parameter. We provide theoretical guarantees for statistical test power across our family of $f$-divergence estimates. While our test covers a variety of $f$-divergences, we bring particular focus to the Hockey-Stick divergence, motivated by its applications to differential privacy auditing and machine unlearning evaluation. For two-sample testing, experiments demonstrate that different $f$-divergences are sensitive to different localized differences, illustrating the importance of leveraging diverse statistics. For machine unlearning, we propose a relative test that distinguishes true unlearning failures from safe distributional variations.

Related papers

• DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing [21.083713063070586]
  We propose an aggregated statistic that explicitly incorporates kernel diversity based on the covariance between different kernels.<n>This motivates a testing framework with selection inference, which leverages information from the training phase to select kernels with strong individual performance.
  arXiv  Detail & Related papers  (2025-10-13T08:30:42Z)
• Kernel conditional tests from learning-theoretic bounds [16.813275168865953]
  We propose a framework for hypothesis testing on conditional probability distributions.<n>We then use to construct statistical tests of functionals of conditional distributions.<n>Our results establish a comprehensive foundation for conditional testing on functionals.
  arXiv  Detail & Related papers  (2025-06-04T12:53:13Z)
• Learning Representations for Independence Testing [13.842061060076004]
  We show how to construct powerful tests with finite-sample validity using variational estimators of mutual information.<n>Second, we establish a close connection between these variational mutual information-based tests and tests based on the Hilbert-Schmidt Independence Criterion (HSIC)<n>Finally, we show how to, rather than selecting a representation to maximize the statistic itself, select a representation which can maximize the power of a test.
  arXiv  Detail & Related papers  (2024-09-10T22:18:07Z)
• Collaborative non-parametric two-sample testing [55.98760097296213]
  The goal is to identify nodes where the null hypothesis $p_v = q_v$ should be rejected. We propose the non-parametric collaborative two-sample testing (CTST) framework that efficiently leverages the graph structure. Our methodology integrates elements from f-divergence estimation, Kernel Methods, and Multitask Learning.
  arXiv  Detail & Related papers  (2024-02-08T14:43:56Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• Active Sequential Two-Sample Testing [18.99517340397671]
  We consider the two-sample testing problem in a new scenario where sample measurements are inexpensive to access. We devise the first emphactiveNIST-sample testing framework that not only sequentially but also emphactively queries. In practice, we introduce an instantiation of our framework and evaluate it using several experiments.
  arXiv  Detail & Related papers  (2023-01-30T02:23:49Z)
• Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observations [44.71254888821376]
  We provide the first type-I-error and expected-rejection-time guarantees under general non-data generating processes. We show how to apply our results to inference on parameters defined by estimating equations, such as average treatment effects.
  arXiv  Detail & Related papers  (2022-12-29T18:37:08Z)
• Sequential Permutation Testing of Random Forest Variable Importance Measures [68.8204255655161]
  It is proposed here to use sequential permutation tests and sequential p-value estimation to reduce the high computational costs associated with conventional permutation tests. The results of simulation studies confirm that the theoretical properties of the sequential tests apply. The numerical stability of the methods is investigated in two additional application studies.
  arXiv  Detail & Related papers  (2022-06-02T20:16:50Z)
• Nonparametric Conditional Local Independence Testing [69.31200003384122]
  Conditional local independence is an independence relation among continuous time processes. No nonparametric test of conditional local independence has been available. We propose such a nonparametric test based on double machine learning.
  arXiv  Detail & Related papers  (2022-03-25T10:31:02Z)
• Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control [67.52000805944924]
  Learn then Test (LTT) is a framework for calibrating machine learning models. Our main insight is to reframe the risk-control problem as multiple hypothesis testing. We use our framework to provide new calibration methods for several core machine learning tasks with detailed worked examples in computer vision.
  arXiv  Detail & Related papers  (2021-10-03T17:42:03Z)
• An Optimal Witness Function for Two-Sample Testing [13.159512679346685]
  We propose data-dependent test statistics based on a one-dimensional witness function, which we call witness two-sample tests (WiTS) We show that the WiTS test based on a characteristic kernel is consistent against any fixed alternative.
  arXiv  Detail & Related papers  (2021-02-10T17:13:21Z)
• Learning Deep Kernels for Non-Parametric Two-Sample Tests [50.92621794426821]
  We propose a class of kernel-based two-sample tests, which aim to determine whether two sets of samples are drawn from the same distribution. Our tests are constructed from kernels parameterized by deep neural nets, trained to maximize test power.
  arXiv  Detail & Related papers  (2020-02-21T03:54:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.