Distributionally Robust Learning in Heterogeneous Contexts

• URL: http://arxiv.org/abs/2105.08532v1
• Date: Tue, 18 May 2021 14:00:34 GMT
• Title: Distributionally Robust Learning in Heterogeneous Contexts
• Authors: Muhammad Osama, Dave Zachariah, Petre Stoica
• Abstract summary: We consider the problem of learning from training data obtained in different contexts, where the test data is subject to distributional shifts. We develop a distributionally robust method that focuses on excess risks and achieves a more appropriate trade-off between performance and robustness than the conventional and overly conservative minimax approach.
• Score: 29.60681287631439
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the problem of learning from training data obtained in different contexts, where the test data is subject to distributional shifts. We develop a distributionally robust method that focuses on excess risks and achieves a more appropriate trade-off between performance and robustness than the conventional and overly conservative minimax approach. The proposed method is computationally feasible and provides statistical guarantees. We demonstrate its performance using both real and synthetic data.

Related papers

• Achievable Fairness on Your Data With Utility Guarantees [16.78730663293352]
  In machine learning fairness, training models that minimize disparity across different sensitive groups often leads to diminished accuracy. We present a computationally efficient approach to approximate the fairness-accuracy trade-off curve tailored to individual datasets. We introduce a novel methodology for quantifying uncertainty in our estimates, thereby providing practitioners with a robust framework for auditing model fairness.
  arXiv  Detail & Related papers  (2024-02-27T00:59:32Z)
• Distributionally Robust Model-based Reinforcement Learning with Large State Spaces [55.14361269378122]
  Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment. We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets. We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
  arXiv  Detail & Related papers  (2023-09-05T13:42:11Z)
• Distributionally Robust Learning with Stable Adversarial Training [34.74504615726101]
  Machine learning algorithms with empirical risk minimization are vulnerable under distributional shifts. We propose a novel Stable Adversarial Learning (SAL) algorithm that leverages heterogeneous data sources to construct a more practical uncertainty set.
  arXiv  Detail & Related papers  (2021-06-30T03:05:45Z)
• Deep Stable Learning for Out-Of-Distribution Generalization [27.437046504902938]
  Approaches based on deep neural networks have achieved striking performance when testing data and training data share similar distribution. Eliminating the impact of distribution shifts between training and testing data is crucial for building performance-promising deep models. We propose to address this problem by removing the dependencies between features via learning weights for training samples.
  arXiv  Detail & Related papers  (2021-04-16T03:54:21Z)
• Learning from Similarity-Confidence Data [94.94650350944377]
  We investigate a novel weakly supervised learning problem of learning from similarity-confidence (Sconf) data. We propose an unbiased estimator of the classification risk that can be calculated from only Sconf data and show that the estimation error bound achieves the optimal convergence rate.
  arXiv  Detail & Related papers  (2021-02-13T07:31:16Z)
• Fair Densities via Boosting the Sufficient Statistics of Exponential Families [72.34223801798422]
  We introduce a boosting algorithm to pre-process data for fairness. Our approach shifts towards better data fitting while still ensuring a minimal fairness guarantee. Empirical results are present to display the quality of result on real-world data.
  arXiv  Detail & Related papers  (2020-12-01T00:49:17Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)
• Stable Adversarial Learning under Distributional Shifts [46.98655899839784]
  Machine learning algorithms with empirical risk minimization are vulnerable under distributional shifts. We propose Stable Adversarial Learning (SAL) algorithm that leverages heterogeneous data sources to construct a more practical uncertainty set.
  arXiv  Detail & Related papers  (2020-06-08T08:42:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.