A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings

• URL: http://arxiv.org/abs/2002.03689v8
• Date: Fri, 8 Jan 2021 14:55:23 GMT
• Title: A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings
• Authors: Junhyung Park and Krikamol Muandet
• Abstract summary: We present an operator-free, measure-theoretic approach to the conditional mean embedding. We derive a natural regression interpretation to obtain empirical estimates. As natural by-products, we obtain the conditional analogues of the mean discrepancy and Hilbert-Schmidt independence criterion.
• Score: 14.71280987722701
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present an operator-free, measure-theoretic approach to the conditional mean embedding (CME) as a random variable taking values in a reproducing kernel Hilbert space. While the kernel mean embedding of unconditional distributions has been defined rigorously, the existing operator-based approach of the conditional version depends on stringent assumptions that hinder its analysis. We overcome this limitation via a measure-theoretic treatment of CMEs. We derive a natural regression interpretation to obtain empirical estimates, and provide a thorough theoretical analysis thereof, including universal consistency. As natural by-products, we obtain the conditional analogues of the maximum mean discrepancy and Hilbert-Schmidt independence criterion, and demonstrate their behaviour via simulations.

Related papers

• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Q-Learning for Stochastic Control under General Information Structures and Non-Markovian Environments [1.90365714903665]
  We present a convergence theorem for iterations, and iterate in particular, Q-learnings under a general, possibly non-Markovian, environment. We discuss the implications and applications of this theorem to a variety of control problems with non-Markovian environments.
  arXiv  Detail & Related papers  (2023-10-31T19:53:16Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• On the Importance of Gradient Norm in PAC-Bayesian Bounds [92.82627080794491]
  We propose a new generalization bound that exploits the contractivity of the log-Sobolev inequalities. We empirically analyze the effect of this new loss-gradient norm term on different neural architectures.
  arXiv  Detail & Related papers  (2022-10-12T12:49:20Z)
• Learning Counterfactually Invariant Predictors [11.682403472580162]
  We propose a model-agnostic framework, called Counterfactually Invariant Prediction (CIP) Our experimental results demonstrate the effectiveness of CIP in enforcing counterfactual invariance across various simulated and real-world datasets.
  arXiv  Detail & Related papers  (2022-07-20T09:23:35Z)
• Optimal variance-reduced stochastic approximation in Banach spaces [114.8734960258221]
  We study the problem of estimating the fixed point of a contractive operator defined on a separable Banach space. We establish non-asymptotic bounds for both the operator defect and the estimation error.
  arXiv  Detail & Related papers  (2022-01-21T02:46:57Z)
• Nearest neighbor empirical processes [7.034466417392574]
  An empirical measure based on the responses from the nearest neighbors to a given point $x$ is introduced and studied as a central statistical quantity. A uniform non-asymptotic bound is established under a well-known condition, often referred to as Vapnik-Chervonenkis, on the uniform entropy numbers. This suggests the possibility of using standard formulas to estimate the variance by using only the nearest neighbors instead of the full data.
  arXiv  Detail & Related papers  (2021-10-27T08:15:20Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Conditional Distributional Treatment Effect with Kernel Conditional Mean Embeddings and U-Statistic Regression [20.544239209511982]
  conditional distributional treatment effect (CoDiTE) CoDiTE encodes a treatment's distributional aspects beyond the mean. Experiments on synthetic, semi-synthetic and real datasets demonstrate the merits of our approach.
  arXiv  Detail & Related papers  (2021-02-16T15:09:23Z)
• The Variational Method of Moments [65.91730154730905]
  conditional moment problem is a powerful formulation for describing structural causal parameters in terms of observables. Motivated by a variational minimax reformulation of OWGMM, we define a very general class of estimators for the conditional moment problem. We provide algorithms for valid statistical inference based on the same kind of variational reformulations.
  arXiv  Detail & Related papers  (2020-12-17T07:21:06Z)
• Kernel Autocovariance Operators of Stationary Processes: Estimation and Convergence [0.5505634045241288]
  We consider autocovariance operators of a stationary process on a Polish space embedded into a kernel reproducing Hilbert space. We investigate how empirical estimates of these operators converge along realizations of the process under various conditions. We provide applications of our theory in terms of consistency results for kernel PCA with dependent data and the conditional mean embedding of transition probabilities.
  arXiv  Detail & Related papers  (2020-04-02T09:17:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.