Statistical Analysis of Wasserstein Distributionally Robust Estimators
- URL: http://arxiv.org/abs/2108.02120v1
- Date: Wed, 4 Aug 2021 15:45:47 GMT
- Title: Statistical Analysis of Wasserstein Distributionally Robust Estimators
- Authors: Jose Blanchet and Karthyek Murthy and Viet Anh Nguyen
- Abstract summary: We consider statistical methods which invoke a min-max distributionally robust formulation to extract good out-of-sample performance in data-driven optimization and learning problems.
The resulting Distributionally Robust Optimization (DRO) formulations are specified using optimal transportation phenomena.
This tutorial is devoted to insights into the nature of the adversarials selected by the min-max formulations and additional applications of optimal transport projections.
- Score: 9.208007322096535
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We consider statistical methods which invoke a min-max distributionally
robust formulation to extract good out-of-sample performance in data-driven
optimization and learning problems. Acknowledging the distributional
uncertainty in learning from limited samples, the min-max formulations
introduce an adversarial inner player to explore unseen covariate data. The
resulting Distributionally Robust Optimization (DRO) formulations, which
include Wasserstein DRO formulations (our main focus), are specified using
optimal transportation phenomena. Upon describing how these
infinite-dimensional min-max problems can be approached via a
finite-dimensional dual reformulation, the tutorial moves into its main
component, namely, explaining a generic recipe for optimally selecting the size
of the adversary's budget. This is achieved by studying the limit behavior of
an optimal transport projection formulation arising from an inquiry on the
smallest confidence region that includes the unknown population risk minimizer.
Incidentally, this systematic prescription coincides with those in specific
examples in high-dimensional statistics and results in error bounds that are
free from the curse of dimensions. Equipped with this prescription, we present
a central limit theorem for the DRO estimator and provide a recipe for
constructing compatible confidence regions that are useful for uncertainty
quantification. The rest of the tutorial is devoted to insights into the nature
of the optimizers selected by the min-max formulations and additional
applications of optimal transport projections.
Related papers
- Distributionally Robust Optimization as a Scalable Framework to Characterize Extreme Value Distributions [22.765095010254118]
The goal of this paper is to develop distributionally robust optimization (DRO) estimators, specifically for multidimensional Extreme Value Theory (EVT) statistics.
In order to mitigate over-conservative estimates while enhancing out-of-sample performance, we study DRO estimators informed by semi-parametric max-stable constraints in the space of point processes.
Both approaches are validated using synthetically generated data, recovering prescribed characteristics, and verifying the efficacy of the proposed techniques.
arXiv Detail & Related papers (2024-07-31T19:45:27Z) - Wasserstein Distributionally Robust Estimation in High Dimensions:
Performance Analysis and Optimal Hyperparameter Tuning [0.0]
We propose a Wasserstein distributionally robust estimation framework to estimate an unknown parameter from noisy linear measurements.
We focus on the task of analyzing the squared error performance of such estimators.
We show that the squared error can be recovered as the solution of a convex-concave optimization problem.
arXiv Detail & Related papers (2022-06-27T13:02:59Z) - A Dimensionality Reduction Method for Finding Least Favorable Priors
with a Focus on Bregman Divergence [108.28566246421742]
This paper develops a dimensionality reduction method that allows us to move the optimization to a finite-dimensional setting with an explicit bound on the dimension.
In order to make progress on the problem, we restrict ourselves to Bayesian risks induced by a relatively large class of loss functions, namely Bregman divergences.
arXiv Detail & Related papers (2022-02-23T16:22:28Z) - Keep it Tighter -- A Story on Analytical Mean Embeddings [0.6445605125467574]
Kernel techniques are among the most popular and flexible approaches in data science.
Mean embedding gives rise to a divergence measure referred to as maximum mean discrepancy (MMD)
In this paper we focus on the problem of MMD estimation when the mean embedding of one of the underlying distributions is available analytically.
arXiv Detail & Related papers (2021-10-15T21:29:27Z) - Outlier-Robust Sparse Estimation via Non-Convex Optimization [73.18654719887205]
We explore the connection between high-dimensional statistics and non-robust optimization in the presence of sparsity constraints.
We develop novel and simple optimization formulations for these problems.
As a corollary, we obtain that any first-order method that efficiently converges to station yields an efficient algorithm for these tasks.
arXiv Detail & Related papers (2021-09-23T17:38:24Z) - Data-Driven Combinatorial Optimization with Incomplete Information: a
Distributionally Robust Optimization Approach [0.0]
We analyze linear optimization problems where the cost vector is not known a priori, but is only observable through a finite data set.
The goal is to find a procedure that transforms the data set into an estimate of the expected value of the objective function.
arXiv Detail & Related papers (2021-05-28T23:17:35Z) - Modeling the Second Player in Distributionally Robust Optimization [90.25995710696425]
We argue for the use of neural generative models to characterize the worst-case distribution.
This approach poses a number of implementation and optimization challenges.
We find that the proposed approach yields models that are more robust than comparable baselines.
arXiv Detail & Related papers (2021-03-18T14:26:26Z) - Residuals-based distributionally robust optimization with covariate
information [0.0]
We consider data-driven approaches that integrate a machine learning prediction model within distributionally robust optimization (DRO)
Our framework is flexible in the sense that it can accommodate a variety of learning setups and DRO ambiguity sets.
arXiv Detail & Related papers (2020-12-02T11:21:34Z) - Amortized Conditional Normalized Maximum Likelihood: Reliable Out of
Distribution Uncertainty Estimation [99.92568326314667]
We propose the amortized conditional normalized maximum likelihood (ACNML) method as a scalable general-purpose approach for uncertainty estimation.
Our algorithm builds on the conditional normalized maximum likelihood (CNML) coding scheme, which has minimax optimal properties according to the minimum description length principle.
We demonstrate that ACNML compares favorably to a number of prior techniques for uncertainty estimation in terms of calibration on out-of-distribution inputs.
arXiv Detail & Related papers (2020-11-05T08:04:34Z) - On the minmax regret for statistical manifolds: the role of curvature [68.8204255655161]
Two-part codes and the minimum description length have been successful in delivering procedures to single out the best models.
We derive a sharper expression than the standard one given by the complexity, where the scalar curvature of the Fisher information metric plays a dominant role.
arXiv Detail & Related papers (2020-07-06T17:28:19Z) - On Projection Robust Optimal Transport: Sample Complexity and Model
Misspecification [101.0377583883137]
Projection robust (PR) OT seeks to maximize the OT cost between two measures by choosing a $k$-dimensional subspace onto which they can be projected.
Our first contribution is to establish several fundamental statistical properties of PR Wasserstein distances.
Next, we propose the integral PR Wasserstein (IPRW) distance as an alternative to the PRW distance, by averaging rather than optimizing on subspaces.
arXiv Detail & Related papers (2020-06-22T14:35:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.