An integrated perspective of robustness in regression through the lens of the bias-variance trade-off

• URL: http://arxiv.org/abs/2407.10418v1
• Date: Mon, 15 Jul 2024 03:47:16 GMT
• Title: An integrated perspective of robustness in regression through the lens of the bias-variance trade-off
• Authors: Akifumi Okuno,
• Abstract summary: We examine the relationship between traditional outlier-resistant robust estimation and robust optimization, which focuses on parameter estimation resistant to imaginary dataset-perturbations. While both are commonly regarded as robust methods, these concepts demonstrate a bias-variance trade-off, indicating that they follow roughly converse strategies.
• Score: 3.0277213703725767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents an integrated perspective on robustness in regression. Specifically, we examine the relationship between traditional outlier-resistant robust estimation and robust optimization, which focuses on parameter estimation resistant to imaginary dataset-perturbations. While both are commonly regarded as robust methods, these concepts demonstrate a bias-variance trade-off, indicating that they follow roughly converse strategies.

Related papers

• Generalization Certificates for Adversarially Robust Bayesian Linear Regression [16.3368950151084]
  Adversarial robustness of machine learning models is critical to ensuring reliable performance under data perturbations. Recent progress has been on point estimators, and this paper considers distributional predictors. Experiments on real and synthetic datasets demonstrate the superior robustness of the derived adversarially robust posterior over Bayes posterior.
  arXiv  Detail & Related papers  (2025-02-20T06:25:30Z)
• ACCon: Angle-Compensated Contrastive Regularizer for Deep Regression [28.491074229136014]
  In deep regression, capturing the relationship among continuous labels in feature space is a fundamental challenge that has attracted increasing interest. Existing approaches often rely on order-aware representation learning or distance-based weighting. We propose an angle-compensated contrastive regularizer for deep regression, which adjusts the cosine distance between anchor and negative samples.
  arXiv  Detail & Related papers  (2025-01-13T03:55:59Z)
• Debiased Nonparametric Regression for Statistical Inference and Distributionally Robustness [10.470114319701576]
  We introduce a model-free debiasing method for smooth nonparametric estimators derived from any nonparametric regression approach. We obtain a debiased estimator with proven pointwise normality and uniform convergence.
  arXiv  Detail & Related papers  (2024-12-28T15:01:19Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Errors-in-variables Fr\'echet Regression with Low-rank Covariate Approximation [2.1756081703276]
  Fr'echet regression has emerged as a promising approach for regression analysis involving non-Euclidean response variables. Our proposed framework combines the concepts of global Fr'echet regression and principal component regression, aiming to improve the efficiency and accuracy of the regression estimator.
  arXiv  Detail & Related papers  (2023-05-16T08:37:54Z)
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition [109.62614226793833]
  The trade-off between robustness and accuracy has been widely studied in the adversarial literature. We find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance. By definition, SCORE facilitates the reconciliation between robustness and accuracy, while still handling the worst-case uncertainty.
  arXiv  Detail & Related papers  (2022-02-21T10:36:09Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Adversarially Robust Estimate and Risk Analysis in Linear Regression [17.931533943788335]
  Adversarially robust learning aims to design algorithms that are robust to small adversarial perturbations on input variables. By discovering the statistical minimax rate of convergence of adversarially robust estimators, we emphasize the importance of incorporating model information. We propose a straightforward two-stage adversarial learning framework, which facilitates to utilize model structure information to improve adversarial robustness.
  arXiv  Detail & Related papers  (2020-12-18T14:55:55Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• Counterfactual Representation Learning with Balancing Weights [74.67296491574318]
  Key to causal inference with observational data is achieving balance in predictive features associated with each treatment type. Recent literature has explored representation learning to achieve this goal. We develop an algorithm for flexible, scalable and accurate estimation of causal effects.
  arXiv  Detail & Related papers  (2020-10-23T19:06:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.