Adversarial Estimators

• URL: http://arxiv.org/abs/2204.10495v1
• Date: Fri, 22 Apr 2022 04:39:44 GMT
• Title: Adversarial Estimators
• Authors: Jonas Metzger
• Abstract summary: We develop an theory of adversarial estimators (A-estimators') We present results characterizing the convergence rates of A-estimators under both point-wise and partial identification. Our theory also yields the normality of general functionals of neural network M-estimators.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We develop an asymptotic theory of adversarial estimators (`A-estimators'). Like maximum-likelihood-type estimators (`M-estimators'), both the estimator and estimand are defined as the critical points of a sample and population average respectively. A-estimators generalize M-estimators, as their objective is maximized by one set of parameters and minimized by another. The continuous-updating Generalized Method of Moments estimator, popular in econometrics and causal inference, is among the earliest members of this class which distinctly falls outside the M-estimation framework. Since the recent success of Generative Adversarial Networks, A-estimators received considerable attention in both machine learning and causal inference contexts, where a flexible adversary can remove the need for researchers to manually specify which features of a problem are important. We present general results characterizing the convergence rates of A-estimators under both point-wise and partial identification, and derive the asymptotic root-n normality for plug-in estimates of smooth functionals of their parameters. All unknown parameters may contain functions which are approximated via sieves. While the results apply generally, we provide easily verifiable, low-level conditions for the case where the sieves correspond to (deep) neural networks. Our theory also yields the asymptotic normality of general functionals of neural network M-estimators (as a special case), overcoming technical issues previously identified by the literature. We examine a variety of A-estimators proposed across econometrics and machine learning and use our theory to derive novel statistical results for each of them. Embedding distinct A-estimators into the same framework, we notice interesting connections among them, providing intuition and formal justification for their recent success in practical applications.

Related papers

• Causal K-Means Clustering [5.087519744951637]
  Causal k-Means Clustering harnesses the widely-used k-means clustering algorithm to uncover the unknown subgroup structure. We present a plug-in estimator which is simple and readily implementable using off-the-shelf algorithms. Our proposed methods are especially useful for modern outcome-wide studies with multiple treatment levels.
  arXiv  Detail & Related papers  (2024-05-05T23:59:51Z)
• The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness [13.120373493503772]
  We prove a surprising result that even if the ground truth itself is robust to adversarial examples, the benignly overfitted model is benign in terms of the standard'' out-of-sample risk objective. Our finding provides theoretical insights into the puzzling phenomenon observed in practice, where the true target function (e.g., human) is robust against adverasrial attack, while beginly overfitted neural networks lead to models that are not robust.
  arXiv  Detail & Related papers  (2024-01-19T15:40:46Z)
• Fine-grained analysis of non-parametric estimation for pairwise learning [9.676007573960383]
  We are concerned with the generalization performance of non-parametric estimation for pairwise learning. Our results can be used to handle a wide range of pairwise learning problems including ranking, AUC, pairwise regression and metric and similarity learning.
  arXiv  Detail & Related papers  (2023-05-31T08:13:14Z)
• A Statistical Model for Predicting Generalization in Few-Shot Classification [6.158812834002346]
  We introduce a Gaussian model of the feature distribution to predict the generalization error. We show that our approach outperforms alternatives such as the leave-one-out cross-validation strategy.
  arXiv  Detail & Related papers  (2022-12-13T10:21:15Z)
• Predicting Unreliable Predictions by Shattering a Neural Network [145.3823991041987]
  Piecewise linear neural networks can be split into subfunctions. Subfunctions have their own activation pattern, domain, and empirical error. Empirical error for the full network can be written as an expectation over subfunctions.
  arXiv  Detail & Related papers  (2021-06-15T18:34:41Z)
• Counterfactual Maximum Likelihood Estimation for Training Deep Networks [83.44219640437657]
  Deep learning models are prone to learning spurious correlations that should not be learned as predictive clues. We propose a causality-based training framework to reduce the spurious correlations caused by observable confounders. We conduct experiments on two real-world tasks: Natural Language Inference (NLI) and Image Captioning.
  arXiv  Detail & Related papers  (2021-06-07T17:47:16Z)
• Scalable Personalised Item Ranking through Parametric Density Estimation [53.44830012414444]
  Learning from implicit feedback is challenging because of the difficult nature of the one-class problem. Most conventional methods use a pairwise ranking approach and negative samplers to cope with the one-class problem. We propose a learning-to-rank approach, which achieves convergence speed comparable to the pointwise counterpart.
  arXiv  Detail & Related papers  (2021-05-11T03:38:16Z)
• Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approach [84.29777236590674]
  We study the estimation of causal parameters when not all confounders are observed and instead negative controls are available. Recent work has shown how these can enable identification and efficient estimation via two so-called bridge functions.
  arXiv  Detail & Related papers  (2021-03-25T17:59:19Z)
• Performance metrics for intervention-triggering prediction models do not reflect an expected reduction in outcomes from using the model [71.9860741092209]
  Clinical researchers often select among and evaluate risk prediction models. Standard metrics calculated from retrospective data are only related to model utility under certain assumptions. When predictions are delivered repeatedly throughout time, the relationship between standard metrics and utility is further complicated.
  arXiv  Detail & Related papers  (2020-06-02T16:26:49Z)
• Machine learning for causal inference: on the use of cross-fit estimators [77.34726150561087]
  Doubly-robust cross-fit estimators have been proposed to yield better statistical properties. We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE) When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
  arXiv  Detail & Related papers  (2020-04-21T23:09:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.