Outlier-robust Estimation of a Sparse Linear Model Using Invexity

• URL: http://arxiv.org/abs/2306.12678v1
• Date: Thu, 22 Jun 2023 05:48:25 GMT
• Title: Outlier-robust Estimation of a Sparse Linear Model Using Invexity
• Authors: Adarsh Barik and Jean Honorio
• Abstract summary: In this paper, we study problem of estimating a sparse regression vector with correct support in the presence of outlier samples. We propose a version of outlier-robust lasso which also identifies clean samples. We also provide a novel invex relaxation for the problem and provide provable theoretical guarantees for this relaxation.
• Score: 31.061339148448006
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we study problem of estimating a sparse regression vector with correct support in the presence of outlier samples. The inconsistency of lasso-type methods is well known in this scenario. We propose a combinatorial version of outlier-robust lasso which also identifies clean samples. Subsequently, we use these clean samples to make a good estimation. We also provide a novel invex relaxation for the combinatorial problem and provide provable theoretical guarantees for this relaxation. Finally, we conduct experiments to validate our theory and compare our results against standard lasso.

Related papers

• Distribution Estimation under the Infinity Norm [19.997465098927858]
  We present novel bounds for estimating discrete probability distributions under the $ell_infty$ norm. Our data-dependent convergence guarantees for the maximum likelihood estimator significantly improve upon the currently known results.
  arXiv  Detail & Related papers  (2024-02-13T12:49:50Z)
• Which Pretrain Samples to Rehearse when Finetuning Pretrained Models? [60.59376487151964]
  Fine-tuning pretrained models on specific tasks is now the de facto approach for text and vision tasks. A known pitfall of this approach is the forgetting of pretraining knowledge that happens during finetuning. We propose a novel sampling scheme, mix-cd, that identifies and prioritizes samples that actually face forgetting.
  arXiv  Detail & Related papers  (2024-02-12T22:32:12Z)
• Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling [156.7248383178991]
  Collaborative Metric Learning (CML) paradigm has aroused wide interest in the area of recommendation systems (RS) We find that negative sampling would lead to a biased estimation of the generalization error. Motivated by this, we propose an efficient alternative without negative sampling for CML named textitSampling-Free Collaborative Metric Learning (SFCML)
  arXiv  Detail & Related papers  (2022-06-23T08:50:22Z)
• Bayes Classification using an approximation to the Joint Probability Distribution of the Attributes [1.0660480034605242]
  We propose an approach that estimates conditional probabilities using information in the neighbourhood of the test sample. We illustrate the performance of the proposed approach on a wide range of datasets taken from the University of California at Irvine (UCI) Machine Learning Repository.
  arXiv  Detail & Related papers  (2022-05-29T22:24:02Z)
• An Analysis of Ensemble Sampling [28.18592417451813]
  Ensemble sampling serves as a practical approximation to Thompson sampling when maintaining an exact posterior distribution over model parameters is computationally intractable. We establish a Bayesian regret bound that ensures desirable behavior when ensemble sampling is applied to the linear bandit problem.
  arXiv  Detail & Related papers  (2022-03-02T18:41:22Z)
• The Adaptive Doubly Robust Estimator for Policy Evaluation in Adaptive Experiments and a Paradox Concerning Logging Policy [13.772109618082382]
  We propose a doubly robust (DR) estimator for dependent samples obtained from adaptive experiments. We report an empirical paradox that our proposed DR estimator tends to show better performances compared to other estimators.
  arXiv  Detail & Related papers  (2020-10-08T06:42:48Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)
• Efficiently Sampling Functions from Gaussian Process Posteriors [76.94808614373609]
  We propose an easy-to-use and general-purpose approach for fast posterior sampling. We demonstrate how decoupled sample paths accurately represent Gaussian process posteriors at a fraction of the usual cost.
  arXiv  Detail & Related papers  (2020-02-21T14:03:16Z)
• Estimating Gradients for Discrete Random Variables by Sampling without Replacement [93.09326095997336]
  We derive an unbiased estimator for expectations over discrete random variables based on sampling without replacement. We show that our estimator can be derived as the Rao-Blackwellization of three different estimators.
  arXiv  Detail & Related papers  (2020-02-14T14:15:18Z)
• On Contrastive Learning for Likelihood-free Inference [20.49671736540948]
  Likelihood-free methods perform parameter inference in simulator models where evaluating the likelihood is intractable. One class of methods for this likelihood-free problem uses a classifier to distinguish between pairs of parameter-observation samples. Another popular class of methods fits a conditional distribution to the parameter posterior directly, and a particular recent variant allows for the use of flexible neural density estimators.
  arXiv  Detail & Related papers  (2020-02-10T13:14:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.