An integrated perspective of robustness in regression through the lens of the bias-variance trade-off

• URL: http://arxiv.org/abs/2407.10418v1
• Date: Mon, 15 Jul 2024 03:47:16 GMT
• Title: An integrated perspective of robustness in regression through the lens of the bias-variance trade-off
• Authors: Akifumi Okuno,
• Abstract summary: We examine the relationship between traditional outlier-resistant robust estimation and robust optimization, which focuses on parameter estimation resistant to imaginary dataset-perturbations. While both are commonly regarded as robust methods, these concepts demonstrate a bias-variance trade-off, indicating that they follow roughly converse strategies.
• Score: 3.0277213703725767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents an integrated perspective on robustness in regression. Specifically, we examine the relationship between traditional outlier-resistant robust estimation and robust optimization, which focuses on parameter estimation resistant to imaginary dataset-perturbations. While both are commonly regarded as robust methods, these concepts demonstrate a bias-variance trade-off, indicating that they follow roughly converse strategies.

Related papers

• Contextual Optimization under Covariate Shift: A Robust Approach by Intersecting Wasserstein Balls [18.047245099229325]
  We propose a distributionally robust approach that uses an ambiguity set by the intersection of two Wasserstein balls. We demonstrate the strong empirical performance of our proposed models.
  arXiv  Detail & Related papers  (2024-06-04T15:46:41Z)
• Generating Less Certain Adversarial Examples Improves Robust Generalization [22.00283527210342]
  This paper revisits the robust overfitting phenomenon of adversarial training. We argue that overconfidence in predicting adversarial examples is a potential cause. We propose a formal definition of adversarial certainty that captures the variance of the model's predicted logits on adversarial examples.
  arXiv  Detail & Related papers  (2023-10-06T19:06:13Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Errors-in-variables Fr\'echet Regression with Low-rank Covariate Approximation [2.1756081703276]
  Fr'echet regression has emerged as a promising approach for regression analysis involving non-Euclidean response variables. Our proposed framework combines the concepts of global Fr'echet regression and principal component regression, aiming to improve the efficiency and accuracy of the regression estimator.
  arXiv  Detail & Related papers  (2023-05-16T08:37:54Z)
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition [109.62614226793833]
  The trade-off between robustness and accuracy has been widely studied in the adversarial literature. We find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance. By definition, SCORE facilitates the reconciliation between robustness and accuracy, while still handling the worst-case uncertainty.
  arXiv  Detail & Related papers  (2022-02-21T10:36:09Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Adversarially Robust Estimate and Risk Analysis in Linear Regression [17.931533943788335]
  Adversarially robust learning aims to design algorithms that are robust to small adversarial perturbations on input variables. By discovering the statistical minimax rate of convergence of adversarially robust estimators, we emphasize the importance of incorporating model information. We propose a straightforward two-stage adversarial learning framework, which facilitates to utilize model structure information to improve adversarial robustness.
  arXiv  Detail & Related papers  (2020-12-18T14:55:55Z)
• Combination of interval-valued belief structures based on belief entropy [5.221097007424518]
  The paper investigates the issues of combination and normalization of interval-valued belief structures within the framework of Dempster-Shafer theory of evidence. A new optimality approach based on uncertainty measure is developed, where the problem of combining interval-valued belief structures degenerates into combining basic probability assignments.
  arXiv  Detail & Related papers  (2020-11-27T10:09:52Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• Counterfactual Representation Learning with Balancing Weights [74.67296491574318]
  Key to causal inference with observational data is achieving balance in predictive features associated with each treatment type. Recent literature has explored representation learning to achieve this goal. We develop an algorithm for flexible, scalable and accurate estimation of causal effects.
  arXiv  Detail & Related papers  (2020-10-23T19:06:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.