Outlier-Robust Wasserstein DRO

• URL: http://arxiv.org/abs/2311.05573v1
• Date: Thu, 9 Nov 2023 18:32:00 GMT
• Title: Outlier-Robust Wasserstein DRO
• Authors: Sloan Nietert, Ziv Goldfeld, Soroosh Shafiee
• Abstract summary: Distributionally robust optimization (DRO) is an effective approach for data-driven decision-making in the presence of uncertainty. We propose a novel outlier-robust WDRO framework for decision-making under both geometric (Wasserstein) perturbations and non-geometric (TV) contamination. We prove a strong duality result that enables tractable convex reformulations and efficient computation of our outlier-robust WDRO problem.
• Score: 19.355450629316486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Distributionally robust optimization (DRO) is an effective approach for data-driven decision-making in the presence of uncertainty. Geometric uncertainty due to sampling or localized perturbations of data points is captured by Wasserstein DRO (WDRO), which seeks to learn a model that performs uniformly well over a Wasserstein ball centered around the observed data distribution. However, WDRO fails to account for non-geometric perturbations such as adversarial outliers, which can greatly distort the Wasserstein distance measurement and impede the learned model. We address this gap by proposing a novel outlier-robust WDRO framework for decision-making under both geometric (Wasserstein) perturbations and non-geometric (total variation (TV)) contamination that allows an $\varepsilon$-fraction of data to be arbitrarily corrupted. We design an uncertainty set using a certain robust Wasserstein ball that accounts for both perturbation types and derive minimax optimal excess risk bounds for this procedure that explicitly capture the Wasserstein and TV risks. We prove a strong duality result that enables tractable convex reformulations and efficient computation of our outlier-robust WDRO problem. When the loss function depends only on low-dimensional features of the data, we eliminate certain dimension dependencies from the risk bounds that are unavoidable in the general setting. Finally, we present experiments validating our theory on standard regression and classification tasks.

Related papers

• Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity [54.145730036889496]
  This paper deals with Gradient learning (FL) in the presence of malicious attacks Byzantine data. A novel Average Algorithm (RAGA) is proposed, which leverages robustness aggregation and can select a dataset.
  arXiv  Detail & Related papers  (2024-03-20T08:15:08Z)
• Geometry-Calibrated DRO: Combating Over-Pessimism with Free Energy Implications [31.3535638804615]
  Machine learning algorithms minimize average risk are susceptible to distributional shifts. distributionally robust optimization (DRO) addresses this issue by optimizing the worst-case risk within an uncertainty set. DRO suffers from over-pessimism, leading to low-confidence predictions, poor parameter estimations as well as poor generalization. In this work, we conduct a theoretical analysis of a probable root cause of over-pessimism: excessive focus on noisy samples.
  arXiv  Detail & Related papers  (2023-11-08T23:33:39Z)
• On the Size and Approximation Error of Distilled Sets [57.61696480305911]
  We take a theoretical view on kernel ridge regression based methods of dataset distillation such as Kernel Inducing Points. We prove that a small set of instances exists in the original input space such that its solution in the RFF space coincides with the solution of the original data. A KRR solution can be generated using this distilled set of instances which gives an approximation towards the KRR solution optimized on the full input data.
  arXiv  Detail & Related papers  (2023-05-23T14:37:43Z)
• Mutual Wasserstein Discrepancy Minimization for Sequential Recommendation [82.0801585843835]
  We propose a novel self-supervised learning framework based on Mutual WasserStein discrepancy minimization MStein for the sequential recommendation. We also propose a novel contrastive learning loss based on Wasserstein Discrepancy Measurement.
  arXiv  Detail & Related papers  (2023-01-28T13:38:48Z)
• Wasserstein Distributionally Robust Estimation in High Dimensions: Performance Analysis and Optimal Hyperparameter Tuning [0.0]
  We propose a Wasserstein distributionally robust estimation framework to estimate an unknown parameter from noisy linear measurements. We focus on the task of analyzing the squared error performance of such estimators. We show that the squared error can be recovered as the solution of a convex-concave optimization problem.
  arXiv  Detail & Related papers  (2022-06-27T13:02:59Z)
• Projected Sliced Wasserstein Autoencoder-based Hyperspectral Images Anomaly Detection [42.585075865267946]
  We propose the Projected Sliced Wasserstein (PSW) autoencoder-based anomaly detection method. In particular, the computation-friendly eigen-decomposition method is leveraged to find the principal component for slicing the high-dimensional data. Comprehensive experiments conducted on various real-world hyperspectral anomaly detection benchmarks demonstrate the superior performance of the proposed method.
  arXiv  Detail & Related papers  (2021-12-20T09:21:02Z)
• DORO: Distributional and Outlier Robust Optimization [98.44757325531631]
  We propose the framework of DORO, for Distributional and Outlier Robust Optimization. At the core of this approach is a refined risk function which prevents DRO from overfitting to potential outliers. We theoretically prove the effectiveness of the proposed method, and empirically show that DORO improves the performance and stability of DRO with experiments on large modern datasets.
  arXiv  Detail & Related papers  (2021-06-11T02:59:54Z)
• Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss [111.8807588392563]
  Boundary discontinuity and its inconsistency to the final detection metric have been the bottleneck for rotating detection regression loss design. We propose a novel regression loss based on Gaussian Wasserstein distance as a fundamental approach to solve the problem.
  arXiv  Detail & Related papers  (2021-01-28T12:04:35Z)
• Finite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionality [5.888646114353371]
  Wasserstein distributionally robust optimization aims to find robust and generalizable solutions by hedging against data perturbations in Wasserstein distance. Existing performance guarantees for generic loss functions are either overly conservative due to the curse of dimensionality, or plausible only in large sample complexitys. We develop a non-asymptotic framework for analyzing the out-of-sample performance for Wasserstein robust learning and the generalization bound for its related Lipschitz and gradient regularization problems.
  arXiv  Detail & Related papers  (2020-09-09T16:02:57Z)
• Localization Uncertainty Estimation for Anchor-Free Object Detection [48.931731695431374]
  There are several limitations of the existing uncertainty estimation methods for anchor-based object detection. We propose a new localization uncertainty estimation method called UAD for anchor-free object detection. Our method captures the uncertainty in four directions of box offsets that are homogeneous, so that it can tell which direction is uncertain.
  arXiv  Detail & Related papers  (2020-06-28T13:49:30Z)
• Principled learning method for Wasserstein distributionally robust optimization with local perturbations [21.611525306059985]
  Wasserstein distributionally robust optimization (WDRO) attempts to learn a model that minimizes the local worst-case risk in the vicinity of the empirical data distribution. We propose a minimizer based on a novel approximation theorem and provide the corresponding risk consistency results. Our results show that the proposed method achieves significantly higher accuracy than baseline models on noisy datasets.
  arXiv  Detail & Related papers  (2020-06-05T09:32:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.