Measuring Differences between Conditional Distributions using Kernel Embeddings

• URL: http://arxiv.org/abs/2605.02260v1
• Date: Mon, 04 May 2026 06:16:00 GMT
• Title: Measuring Differences between Conditional Distributions using Kernel Embeddings
• Abstract summary: This paper establishes a coherent framework for studying kernel-based methods to measure divergence between conditional distributions.<n>We introduce a novel doubly robust estimator for the conditional maximum mean discrepancy (CMMD) that maintains consistency provided at least one of the underlying models is correctly specified.
• Score: 17.88693838540775
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Comparing conditional distributions is a fundamental challenge in statistics and machine learning, with applications across a wide range of domains. While proposed methods for measuring discrepancies using kernel embeddings of distributions in a reproducing kernel Hilbert space (RKHS) provide powerful non-parametric techniques, the existing literature remains fragmented and lacks a unified theoretical treatment. This paper addresses this gap by establishing a coherent framework for studying kernel-based methods to measure divergence between conditional distributions through what we refer to as conditional maximum mean discrepancy (CMMD). The CMMD consists of a family of metrics which we call levels, with three special cases each using a different type of RKHS embedding: CMMD$_0$ (conditional mean operators), CMMD$_1$ (conditional mean embeddings), and CMMD$_2$ (joint mean embeddings). We additionally introduce a general level $s$ CMMD, clarifying the required assumptions, and establishing mathematical connections between the levels through the lens of operator-based smoothing. In addition to reviewing previously proposed estimators, we introduce a novel doubly robust estimator for the CMMD that maintains consistency provided at least one of the underlying models is correctly specified. We provide numerical experiments demonstrating that the CMMD effectively captures complex conditional dependencies for statistical testing.

Related papers

• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Scalable Kernel-Based Distances for Statistical Inference and Integration [0.8122270502556375]
  Maximum mean discrepancy (MMD) is a kernel-based distance constructed by comparing Hilbert space mean functions.<n>MMD has received significant attention due to its computational tractability and is favoured by practitioners.<n>In this thesis, we conduct a thorough study of kernel-based distances with a focus on efficient computation.
  arXiv  Detail & Related papers  (2026-02-25T12:25:34Z)
• MMDCP: A Distribution-free Approach to Outlier Detection and Classification with Coverage Guarantees and SCW-FDR Control [6.429952624399788]
  We propose a unified framework for multi-class classification and outlier detection under label shift.<n>The Modified Mahalanobis Distance Conformal Prediction (MMDCP) combines class-specific distance measures with full conformal prediction to construct a score function.<n>We provide the first theoretical characterization of the gap between oracle and empirical conformal $p$-values, which ensures valid coverage and effective control of the class-wise false discovery rate (CW-FDR)
  arXiv  Detail & Related papers  (2025-11-15T03:48:44Z)
• A Kernel Distribution Closeness Testing [6.903548360333296]
  The distribution closeness testing (DCT) assesses whether the distance between a distribution pair is at least $epsilon$-far.<n>Existing DCT methods mainly measure discrepancies between a distribution pair defined on discrete one-dimensional spaces.<n>We introduce maximum mean discrepancy (MMD), a powerful measurement of the distributional discrepancy between two complex distributions.
  arXiv  Detail & Related papers  (2025-07-17T07:08:54Z)
• Model-free Methods for Event History Analysis and Efficient Adjustment (PhD Thesis) [55.2480439325792]
  This thesis is a series of independent contributions to statistics unified by a model-free perspective.<n>The first chapter elaborates on how a model-free perspective can be used to formulate flexible methods that leverage prediction techniques from machine learning.<n>The second chapter studies the concept of local independence, which describes whether the evolution of one process is directly influenced by another.
  arXiv  Detail & Related papers  (2025-02-11T19:24:09Z)
• Computing Low-Entropy Couplings for Large-Support Distributions [53.00113867130712]
  Minimum-entropy coupling has applications in areas such as causality and steganography. Existing algorithms are either computationally intractable for large-support distributions or limited to specific distribution types. This work addresses these limitations by unifying a prior family of iterative MEC approaches into a generalized partition-based formalism.
  arXiv  Detail & Related papers  (2024-05-29T21:54:51Z)
• A Uniform Concentration Inequality for Kernel-Based Two-Sample Statistics [4.757470449749877]
  We show that these metrics can be unified under a general framework of kernel-based two-sample statistics.<n>This paper establishes a novel uniform concentration inequality for the aforementioned kernel-based statistics.<n>As illustrative applications, we demonstrate how these bounds facilitate the component of error bounds for procedures such as distance covariance-based dimension reduction.
  arXiv  Detail & Related papers  (2024-05-22T22:41:56Z)
• Targeted Separation and Convergence with Kernel Discrepancies [61.973643031360254]
  kernel-based discrepancy measures are required to (i) separate a target P from other probability measures or (ii) control weak convergence to P.<n>In this article we derive new sufficient and necessary conditions to ensure (i) and (ii)<n>For MMDs on separable metric spaces, we characterize those kernels that separate Bochner embeddable measures and introduce simple conditions for separating all measures with unbounded kernels.
  arXiv  Detail & Related papers  (2022-09-26T16:41:16Z)
• A Unified Framework for Multi-distribution Density Ratio Estimation [101.67420298343512]
  Binary density ratio estimation (DRE) provides the foundation for many state-of-the-art machine learning algorithms. We develop a general framework from the perspective of Bregman minimization divergence. We show that our framework leads to methods that strictly generalize their counterparts in binary DRE.
  arXiv  Detail & Related papers  (2021-12-07T01:23:20Z)
• Generalized Sliced Distances for Probability Distributions [47.543990188697734]
  We introduce a broad family of probability metrics, coined as Generalized Sliced Probability Metrics (GSPMs) GSPMs are rooted in the generalized Radon transform and come with a unique geometric interpretation. We consider GSPM-based gradient flows for generative modeling applications and show that under mild assumptions, the gradient flow converges to the global optimum.
  arXiv  Detail & Related papers  (2020-02-28T04:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.