Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curves

• URL: http://arxiv.org/abs/2010.04855v6
• Date: Tue, 23 Aug 2022 15:21:51 GMT
• Title: Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curves
• Authors: Rahul Singh, Liyuan Xu, Arthur Gretton
• Abstract summary: We prove uniform consistency with improved finite sample rates via original analysis of generalized kernel ridge regression. We extend our main results to counterfactual distributions and to causal functions identified by front and back door criteria.
• Score: 26.880628841819004
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose estimators based on kernel ridge regression for nonparametric causal functions such as dose, heterogeneous, and incremental response curves. Treatment and covariates may be discrete or continuous in general spaces. Due to a decomposition property specific to the RKHS, our estimators have simple closed form solutions. We prove uniform consistency with improved finite sample rates, via original analysis of generalized kernel ridge regression. We extend our main results to counterfactual distributions and to causal functions identified by front and back door criteria. In nonlinear simulations with many covariates, we achieve state-of-the-art performance.

Related papers

• High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization [83.06112052443233]
  This paper studies kernel ridge regression in high dimensions under covariate shifts. By a bias-variance decomposition, we theoretically demonstrate that the re-weighting strategy allows for decreasing the variance. For bias, we analyze the regularization of the arbitrary or well-chosen scale, showing that the bias can behave very differently under different regularization scales.
  arXiv  Detail & Related papers  (2024-06-05T12:03:27Z)
• Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum [6.749750044497731]
  We prove the phenomena of tempered overfitting and catastrophic overfitting under the sub-Gaussian design assumption. We also identify that the independence of the features plays an important role in guaranteeing tempered overfitting.
  arXiv  Detail & Related papers  (2024-02-02T10:36:53Z)
• Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds [77.4346324549323]
  We show that a step size agnostic to the curvature of the manifold achieves a curvature-independent and linear last-iterate convergence rate. To the best of our knowledge, the possibility of curvature-independent rates and/or last-iterate convergence has not been considered before.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Ridgeless Regression with Random Features [23.41536146432726]
  We investigate the statistical properties of ridgeless regression with random features and gradient descent. We propose a tunable kernel algorithm that optimize the spectral density of kernel during training.
  arXiv  Detail & Related papers  (2022-05-01T14:25:08Z)
• Generalized Kernel Ridge Regression for Causal Inference with Missing-at-Random Sample Selection [3.398662563413433]
  I propose kernel ridge regression estimators for nonparametric dose response curves and semiparametric treatment effects. For the discrete treatment case, I prove root-n consistency, Gaussian approximation, and semiparametric efficiency.
  arXiv  Detail & Related papers  (2021-11-09T17:10:49Z)
• Sequential Kernel Embedding for Mediated and Time-Varying Dose Response Curves [26.880628841819004]
  We propose simple nonparametric estimators for mediated and time-varying dose response curves based on kernel ridge regression. Our key innovation is a reproducing kernel Hilbert space technique called sequential kernel embedding.
  arXiv  Detail & Related papers  (2021-11-06T19:51:39Z)
• On the Double Descent of Random Features Models Trained with SGD [78.0918823643911]
  We study properties of random features (RF) regression in high dimensions optimized by gradient descent (SGD) We derive precise non-asymptotic error bounds of RF regression under both constant and adaptive step-size SGD setting. We observe the double descent phenomenon both theoretically and empirically.
  arXiv  Detail & Related papers  (2021-10-13T17:47:39Z)
• Benign Overfitting of Constant-Stepsize SGD for Linear Regression [122.70478935214128]
  inductive biases are central in preventing overfitting empirically. This work considers this issue in arguably the most basic setting: constant-stepsize SGD for linear regression. We reflect on a number of notable differences between the algorithmic regularization afforded by (unregularized) SGD in comparison to ordinary least squares.
  arXiv  Detail & Related papers  (2021-03-23T17:15:53Z)
• Adversarial Estimation of Riesz Representers [21.510036777607397]
  We propose an adversarial framework to estimate the Riesz representer using general function spaces. We prove a nonasymptotic mean square rate in terms of an abstract quantity called the critical radius, then specialize it for neural networks, random forests, and reproducing kernel Hilbert spaces as leading cases.
  arXiv  Detail & Related papers  (2020-12-30T19:46:57Z)
• Approximation Schemes for ReLU Regression [80.33702497406632]
  We consider the fundamental problem of ReLU regression. The goal is to output the best fitting ReLU with respect to square loss given to draws from some unknown distribution.
  arXiv  Detail & Related papers  (2020-05-26T16:26:17Z)
• SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives [86.01677297601624]
  We propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior.
  arXiv  Detail & Related papers  (2020-03-05T14:33:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.