Adaptive deep nonparametric regression from dependent data under covariate shift

• URL: http://arxiv.org/abs/2607.20309v1
• Date: Wed, 22 Jul 2026 15:55:46 GMT
• Title: Adaptive deep nonparametric regression from dependent data under covariate shift
• Abstract summary: We deal with a generalized Bernstein-type inequality that is satisfied by many classical models.<n>We propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data.<n>For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Covariate shift often occurs because, in many real applications, the source and the target observations may be generated from different distributions. In this case, the standard metric under the source distribution is not appropriate. This paper considers deep neural network estimators for nonparametric quantile and Huber regression under covariate shift and from dependent observations. We deal with a generalized Bernstein-type inequality that is satisfied by many classical models, including i.i.d. observations, $φ$-mixing, strong mixing, and $\mathcal{C}$-mixing processes. To perform the covariate shift phenomenon, we propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data. When the density ratio (between the source and target distributions of the covariate) is unknown, a two steps pre-training procedure is carried out: the first step is devoted to the construction of a least squares SPDNN estimator of the density ratio; which is used in the second step to perform a pre-training reweighted SPDNN estimator of the regression function. For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established in the class of Hölder smooth functions. These estimators can adaptively attain (up to a logarithmic factor) the minimax optimal convergence rate from i.i.d. data as well as from several classical time series models.

Related papers

• FlowSDR: Sufficient Dimension Reduction via Conditional Normalizing Flows [6.1441848595952]
  FlowSDR is a likelihood-based framework that jointly learns the projection and the conditional density.<n>We introduce the neural Gaussian SDR, a heteroscedastic conditional Gaussian model whose mean and variance are parameterized by shared neural-network functions.
  arXiv  Detail & Related papers  (2026-05-31T16:54:56Z)
• Deep regression learning from dependent observations with minimum error entropy principle [0.0]
  We study two estimators: the non-penalized deep neural network (NPDNN) and the sparse-penalized deep neural network (SPDNN) predictors.
  arXiv  Detail & Related papers  (2026-03-11T16:28:06Z)
• Neural Optimal Transport Meets Multivariate Conformal Prediction [58.43397908730771]
  We propose a framework for conditional vectorile regression (CVQR)<n>CVQR combines neural optimal transport with quantized optimization, and apply it to predictions.
  arXiv  Detail & Related papers  (2025-09-29T19:50:19Z)
• Chaos into Order: Neural Framework for Expected Value Estimation of Stochastic Partial Differential Equations [0.9944647907864256]
  We propose a physics-informed neural framework designed to approximate the expected value of linear partial differential equations (SPDEs)<n>By leveraging randomized sampling of both space-time coordinates and noise realizations during training, LEC trains standard feedforward neural networks to minimize residual loss across multiple samples.<n>We show that the model consistently learns accurate approximations of the expected value of the solution in lower dimensions and a predictable decrease in robustness with increased spatial dimensions.
  arXiv  Detail & Related papers  (2025-02-05T23:27:28Z)
• Gradient Descent Finds Over-Parameterized Neural Networks with Sharp Generalization for Nonparametric Regression [15.975065054204753]
  We show that if the neural network is trained by GD with early stopping, then the trained network renders a sharp rate of nonparametric regression risk.<n>$mathcalO(epsilon_n2)$ is known to be minimax optimal for specific cases.
  arXiv  Detail & Related papers  (2024-11-05T08:43:54Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate.<n>We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise [51.87307904567702]
  Quantile regression is a leading approach for obtaining such intervals via the empirical estimation of quantiles in the distribution of outputs.<n>We propose Relaxed Quantile Regression (RQR), a direct alternative to quantile regression based interval construction that removes this arbitrary constraint.<n>We demonstrate that this added flexibility results in intervals with an improvement in desirable qualities.
  arXiv  Detail & Related papers  (2024-06-05T13:36:38Z)
• Semi-Supervised Deep Sobolev Regression: Estimation and Variable Selection by ReQU Neural Network [3.4623717820849476]
  We propose SDORE, a Semi-supervised Deep Sobolev Regressor, for the nonparametric estimation of the underlying regression function and its gradient.<n>Our study includes a thorough analysis of the convergence rates of SDORE in $L2$-norm, achieving the minimax optimality.
  arXiv  Detail & Related papers  (2024-01-09T13:10:30Z)
• On the Double Descent of Random Features Models Trained with SGD [78.0918823643911]
  We study properties of random features (RF) regression in high dimensions optimized by gradient descent (SGD) We derive precise non-asymptotic error bounds of RF regression under both constant and adaptive step-size SGD setting. We observe the double descent phenomenon both theoretically and empirically.
  arXiv  Detail & Related papers  (2021-10-13T17:47:39Z)
• Optimization Variance: Exploring Generalization Properties of DNNs [83.78477167211315]
  The test error of a deep neural network (DNN) often demonstrates double descent. We propose a novel metric, optimization variance (OV), to measure the diversity of model updates.
  arXiv  Detail & Related papers  (2021-06-03T09:34:17Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.