Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity

• URL: http://arxiv.org/abs/2605.27526v1
• Date: Tue, 26 May 2026 18:01:09 GMT
• Title: Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity
• Abstract summary: We develop semiparametrically efficient inference for kernel measures of noise heterogeneity in additive noise models.<n>We construct a novel one-step Hilbert-valued one-step estimator of the kernel cooperator between cocalibrates and residuals.<n>We show improved calibration and power relative to naive plug-in residual methods.
• Score: 20.281748930003737
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop semiparametrically efficient inference for kernel measures of noise heterogeneity in additive noise models. In many applications, the regression function is estimated using flexible machine learning methods. Downstream procedures based on the resulting residuals can then inherit first-stage bias: regression error may induce spurious dependence between covariates and residuals, invalidating the assumptions needed for standard analysis. We construct a novel Hilbert-valued one-step estimator of the kernel covariance operator between covariates and residuals. Our estimator yields bootstrap-calibrated tests for residual independence and goodness of fit in additive noise models, while also providing asymptotically efficient confidence intervals for the kernel dependence measure under noise heterogeneity. The framework extends to settings with additional covariates, enabling inference on distributional heterogeneity of residual noise across treatment groups. Simulations show improved calibration and power relative to naive plug-in residual methods.

Related papers

• Robust Regression with Adaptive Contamination in Response: Optimal Rates and Computational Barriers [65.58071581164043]
  We study robust regression under a contamination model in which covariates are clean while the responses may be corrupted in an adaptive manner.<n>We show that the additional information can be carefully exploited to construct an estimator that a better estimation rate than achieves that attainable under Huber contamination.<n>Despite the improvement over the Huber model from an information-theoretic perspective, we provide formal evidence that the problem exhibits strong information-computation gaps.
  arXiv  Detail & Related papers  (2026-04-05T19:07:24Z)
• Kernel Single-Index Bandits: Estimation, Inference, and Learning [3.1109381260263853]
  We study contextual bandits with finitely many actions in which the reward of each arm follows a single-index model.<n>We propose a kernelized $varepsilon$-greedy algorithm that combines Stein-based estimation of index parameters with inverse-propensity-weighted ridge regression.
  arXiv  Detail & Related papers  (2026-03-19T14:15:16Z)
• A Stable Neural Statistical Dependence Estimator for Autoencoder Feature Analysis [6.697324698653934]
  dependence measures like mutual information are ideal for analyzing autoencoders, but they can be ill-posed for deterministic, static, noise-free networks.<n>We adopt the variational (Gaussian) formulation that makes dependence among inputs, latents, and reconstructions measurable.<n>We propose a stable neural dependence estimator based on an orthonormal density-ratio decomposition.
  arXiv  Detail & Related papers  (2026-03-12T01:37:19Z)
• GaussDetect-LiNGAM:Causal Direction Identification without Gaussianity test [11.675718405602792]
  GaussDetect-LiNGAM eliminates the need for explicit Gaussianity tests.<n>We prove that the Gaussianity of the forward-model noise is equivalent to the independence between the regressor and residual in the reverse model.
  arXiv  Detail & Related papers  (2025-12-03T04:14:57Z)
• Optimal kernel regression bounds under energy-bounded noise [3.2093811507874768]
  We derive a tight, non-asymptotic uncertainty bound for kernel-based estimation.<n>We show its effectiveness in returning tight and easy-to-compute bounds for kernel-based estimates.
  arXiv  Detail & Related papers  (2025-05-28T11:11:24Z)
• Variance-Reducing Couplings for Random Features [57.73648780299374]
  Random features (RFs) are a popular technique to scale up kernel methods in machine learning. We find couplings to improve RFs defined on both Euclidean and discrete input spaces. We reach surprising conclusions about the benefits and limitations of variance reduction as a paradigm.
  arXiv  Detail & Related papers  (2024-05-26T12:25:09Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise [51.31435087414348]
  It is essential to theoretically guarantee that algorithms provide small objective residual with high probability. Existing methods for non-smooth convex optimization have complexity bounds with dependence on confidence level. We propose novel stepsize rules for two methods with gradient clipping.
  arXiv  Detail & Related papers  (2021-06-10T17:54:21Z)
• Detecting Label Noise via Leave-One-Out Cross Validation [0.0]
  We present a simple algorithm for identifying and correcting real-valued noisy labels from a mixture of clean and corrupted samples. A heteroscedastic noise model is employed, in which additive Gaussian noise terms with independent variances are associated with each and all of the observed labels. We show that the presented method can pinpoint corrupted samples and lead to better regression models when trained on synthetic and real-world scientific data sets.
  arXiv  Detail & Related papers  (2021-03-21T10:02:50Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.