Outlier Robust and Sparse Estimation of Linear Regression Coefficients

• URL: http://arxiv.org/abs/2208.11592v5
• Date: Fri, 24 May 2024 06:26:48 GMT
• Title: Outlier Robust and Sparse Estimation of Linear Regression Coefficients
• Authors: Takeyuki Sasai, Hironori Fujisawa,
• Abstract summary: We consider outlier-robust and sparse estimation of linear regression coefficients. Our results present sharper error bounds under weaker assumptions than prior studies that share similar interests with this study.
• Score: 2.0257616108612373
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider outlier-robust and sparse estimation of linear regression coefficients, when the covariates and the noises are contaminated by adversarial outliers and noises are sampled from a heavy-tailed distribution. Our results present sharper error bounds under weaker assumptions than prior studies that share similar interests with this study. Our analysis relies on some sharp concentration inequalities resulting from generic chaining.

Related papers

• Risk and cross validation in ridge regression with correlated samples [72.59731158970894]
  We provide training examples for the in- and out-of-sample risks of ridge regression when the data points have arbitrary correlations. We further extend our analysis to the case where the test point has non-trivial correlations with the training set, setting often encountered in time series forecasting. We validate our theory across a variety of high dimensional data.
  arXiv  Detail & Related papers  (2024-08-08T17:27:29Z)
• Sparse Linear Regression when Noises and Covariates are Heavy-Tailed and Contaminated by Outliers [2.0257616108612373]
  We investigate a problem estimating coefficients of linear regression under sparsity assumption. We consider the situation where not only covariates and noises are sampled from heavy tailed distributions but also contaminated by outliers. Our estimators can be computed efficiently, and exhibit sharp error bounds.
  arXiv  Detail & Related papers  (2024-08-02T15:33:04Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• Robust and Sparse Estimation of Linear Regression Coefficients with Heavy-tailed Noises and Covariates [0.0]
  Our estimator can be computed efficiently. Further, our estimation error bound is sharp. The situation addressed in this paper is that co variables and noises are sampled from heavy-tailed distributions, and the co variables and noises are contaminated by malicious outliers.
  arXiv  Detail & Related papers  (2022-06-15T15:23:24Z)
• The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks [51.1848572349154]
  neural network models that perfectly fit noisy data can generalize well to unseen test data. We consider interpolating two-layer linear neural networks trained with gradient flow on the squared loss and derive bounds on the excess risk.
  arXiv  Detail & Related papers  (2021-08-25T22:01:01Z)
• Heavy-tailed Streaming Statistical Estimation [58.70341336199497]
  We consider the task of heavy-tailed statistical estimation given streaming $p$ samples. We design a clipped gradient descent and provide an improved analysis under a more nuanced condition on the noise of gradients.
  arXiv  Detail & Related papers  (2021-08-25T21:30:27Z)
• Adversarial robust weighted Huber regression [2.0257616108612373]
  We consider a robust estimation of linear regression coefficients. We derive an estimation error bound, which depends on the stable rank and the condition number of the covariance matrix.
  arXiv  Detail & Related papers  (2021-02-22T15:50:34Z)
• Outlier Robust Mean Estimation with Subgaussian Rates via Stability [46.03021473600576]
  We study the problem of robust outlier high-dimensional mean estimation. We obtain first computationally efficient rate with subgaussian for outlier-robust mean estimation.
  arXiv  Detail & Related papers  (2020-07-30T17:33:03Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)
• Compressing Large Sample Data for Discriminant Analysis [78.12073412066698]
  We consider the computational issues due to large sample size within the discriminant analysis framework. We propose a new compression approach for reducing the number of training samples for linear and quadratic discriminant analysis.
  arXiv  Detail & Related papers  (2020-05-08T05:09:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.