LASER: A new method for locally adaptive nonparametric regression

• URL: http://arxiv.org/abs/2412.19802v1
• Date: Fri, 27 Dec 2024 18:59:03 GMT
• Title: LASER: A new method for locally adaptive nonparametric regression
• Authors: Sabyasachi Chatterjee, Subhajit Goswami, Soumendu Sundar Mukherjee,
• Abstract summary: We introduce textsfLASER (Locally Adaptive Smoothing Estimator for Regression), a computationally efficient locally adaptive nonparametric regression method. We prove that it adapts (near-)optimally to the local H"older exponent of the underlying regression function textttsimultaneously at all points in its domain.
• Score: 5.926203312586109
• License:
• Abstract: In this article, we introduce \textsf{LASER} (Locally Adaptive Smoothing Estimator for Regression), a computationally efficient locally adaptive nonparametric regression method that performs variable bandwidth local polynomial regression. We prove that it adapts (near-)optimally to the local H\"{o}lder exponent of the underlying regression function \texttt{simultaneously} at all points in its domain. Furthermore, we show that there is a single ideal choice of a global tuning parameter under which the above mentioned local adaptivity holds. Despite the vast literature on nonparametric regression, instances of practicable methods with provable guarantees of such a strong notion of local adaptivity are rare. The proposed method achieves excellent performance across a broad range of numerical experiments in comparison to popular alternative locally adaptive methods.

Related papers

• GeneralizeFormer: Layer-Adaptive Model Generation across Test-Time Distribution Shifts [58.95913531746308]
  We consider the problem of test-time domain generalization, where a model is trained on several source domains and adjusted on target domains never seen during training. We propose to generate multiple layer parameters on the fly during inference by a lightweight meta-learned transformer, which we call textitGeneralizeFormer.
  arXiv  Detail & Related papers  (2025-02-15T10:10:49Z)
• Robust Local Polynomial Regression with Similarity Kernels [0.0]
  Local Polynomial Regression (LPR) is a widely used nonparametric method for modeling complex relationships. It estimates a regression function by fitting low-degree weights to localized subsets of the data, weighted by proximity. Traditional LPR is sensitive to outliers and high-leverage points, which can significantly affect estimation accuracy. This paper proposes a novel framework that incorporates both predictor and response variables in the weighting mechanism.
  arXiv  Detail & Related papers  (2025-01-18T11:21:26Z)
• Adaptive Conformal Inference by Betting [51.272991377903274]
  We consider the problem of adaptive conformal inference without any assumptions about the data generating process. Existing approaches for adaptive conformal inference are based on optimizing the pinball loss using variants of online gradient descent. We propose a different approach for adaptive conformal inference that leverages parameter-free online convex optimization techniques.
  arXiv  Detail & Related papers  (2024-12-26T18:42:08Z)
• Minmax Trend Filtering: A Locally Adaptive Nonparametric Regression Method via Pointwise Min Max Optimization [4.07926531936425]
  There seems to be no unanimously agreed upon definition of local adaptivity in the literature. We first derive a new pointwise formula for the Fused Lasso estimator in terms of min-max/max-min optimization of penalized local averages. We then propose higher order versions of Fused Lasso which are defined pointwise in terms of min-max/max-min optimization of penalized local regressions.
  arXiv  Detail & Related papers  (2024-10-03T23:15:35Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate. We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup [90.26270347459915]
  We propose a novel momentum-based algorithm via utilizing the global descent locally adaptive. textitLADA could greatly reduce the communication rounds and achieves higher accuracy than several baselines.
  arXiv  Detail & Related papers  (2023-07-30T14:53:21Z)
• Benign overfitting and adaptive nonparametric regression [71.70323672531606]
  We construct an estimator which is a continuous function interpolating the data points with high probability. We attain minimax optimal rates under mean squared risk on the scale of H"older classes adaptively to the unknown smoothness.
  arXiv  Detail & Related papers  (2022-06-27T14:50:14Z)
• Communication-Efficient Distributed Quantile Regression with Optimal Statistical Guarantees [2.064612766965483]
  We address the problem of how to achieve optimal inference in distributed quantile regression without stringent scaling conditions. The difficulties are resolved through a double-smoothing approach that is applied to the local (at each data source) and global objective functions. Despite the reliance on a delicate combination of local and global smoothing parameters, the quantile regression model is fully parametric.
  arXiv  Detail & Related papers  (2021-10-25T17:09:59Z)
• AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methods [7.486132958737807]
  We present an adaptive variance reduced method with an implicit approach for adaptivity. We provide convergence guarantees for finite-sum minimization problems and show a faster convergence than SARAH can be achieved if local geometry permits. This algorithm implicitly computes step-size and efficiently estimates local Lipschitz smoothness of functions.
  arXiv  Detail & Related papers  (2021-02-19T01:17:15Z)
• Support estimation in high-dimensional heteroscedastic mean regression [2.07180164747172]
  We consider a linear mean regression model with random design and potentially heteroscedastic, heavy-tailed errors. We use a strictly convex, smooth variant of the Huber loss function with tuning parameter depending on the parameters of the problem. For the resulting estimator we show sign-consistency and optimal rates of convergence in the $ell_infty$ norm.
  arXiv  Detail & Related papers  (2020-11-03T09:46:31Z)
• On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization [80.03647903934723]
  We prove adaptive gradient methods in expectation of gradient convergence methods. Our analyses shed light on better adaptive gradient methods in optimizing non understanding gradient bounds.
  arXiv  Detail & Related papers  (2018-08-16T20:25:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.