Distributionally Robust Losses for Latent Covariate Mixtures

• URL: http://arxiv.org/abs/2007.13982v2
• Date: Wed, 10 Aug 2022 18:58:19 GMT
• Title: Distributionally Robust Losses for Latent Covariate Mixtures
• Authors: John Duchi, Tatsunori Hashimoto, Hongseok Namkoong
• Abstract summary: We propose a convex procedure that controls the worst-case performance over all subpopulations of a given size. We observe on lexical similarity, wine quality, and recidivism prediction tasks that our worst-case procedure learns models that do well against unseen subpopulations.
• Score: 28.407773942857148
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While modern large-scale datasets often consist of heterogeneous subpopulations -- for example, multiple demographic groups or multiple text corpora -- the standard practice of minimizing average loss fails to guarantee uniformly low losses across all subpopulations. We propose a convex procedure that controls the worst-case performance over all subpopulations of a given size. Our procedure comes with finite-sample (nonparametric) convergence guarantees on the worst-off subpopulation. Empirically, we observe on lexical similarity, wine quality, and recidivism prediction tasks that our worst-case procedure learns models that do well against unseen subpopulations.

Related papers

• Theoretical Guarantees of Data Augmented Last Layer Retraining Methods [5.352699766206809]
  Linear last layer retraining strategies have been shown to achieve state-of-the-art performance for worst-group accuracy. We present the optimal worst-group accuracy when modeling the distribution of the latent representations. We evaluate and verify our results for both synthetic and large publicly available datasets.
  arXiv  Detail & Related papers  (2024-05-09T17:16:54Z)
• Minimax Regret Learning for Data with Heterogeneous Subgroups [12.253779655660571]
  We develop a min-max-regret (MMR) learning framework for general supervised learning, which targets to minimize the worst-group regret. We demonstrate the effectiveness of our method through extensive simulation studies and an application to kidney transplantation data from hundreds of transplant centers.
  arXiv  Detail & Related papers  (2024-05-02T20:06:41Z)
• Estimating Unknown Population Sizes Using the Hypergeometric Distribution [1.03590082373586]
  We tackle the challenge of estimating discrete distributions when both the total population size and the sizes of its constituent categories are unknown. We develop our approach to account for a data generating process where the ground-truth is a mixture of distributions conditional on a continuous latent variable. Empirical data simulation demonstrates that our method outperforms other likelihood functions used to model count data.
  arXiv  Detail & Related papers  (2024-02-22T01:53:56Z)
• Optimal Multi-Distribution Learning [88.3008613028333]
  Multi-distribution learning seeks to learn a shared model that minimizes the worst-case risk across $k$ distinct data distributions. We propose a novel algorithm that yields an varepsilon-optimal randomized hypothesis with a sample complexity on the order of (d+k)/varepsilon2.
  arXiv  Detail & Related papers  (2023-12-08T16:06:29Z)
• Reweighted Mixup for Subpopulation Shift [63.1315456651771]
  Subpopulation shift exists in many real-world applications, which refers to the training and test distributions that contain the same subpopulation groups but with different subpopulation proportions. Importance reweighting is a classical and effective way to handle the subpopulation shift. We propose a simple yet practical framework, called reweighted mixup, to mitigate the overfitting issue.
  arXiv  Detail & Related papers  (2023-04-09T03:44:50Z)
• UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixup [44.0372420908258]
  Subpopulation shift wildly exists in many real-world machine learning applications. Importance reweighting is a normal way to handle the subpopulation shift issue. We propose uncertainty-aware mixup (Umix) to mitigate the overfitting issue.
  arXiv  Detail & Related papers  (2022-09-19T11:22:28Z)
• Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling [156.7248383178991]
  Collaborative Metric Learning (CML) paradigm has aroused wide interest in the area of recommendation systems (RS) We find that negative sampling would lead to a biased estimation of the generalization error. Motivated by this, we propose an efficient alternative without negative sampling for CML named textitSampling-Free Collaborative Metric Learning (SFCML)
  arXiv  Detail & Related papers  (2022-06-23T08:50:22Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• GANs with Variational Entropy Regularizers: Applications in Mitigating the Mode-Collapse Issue [95.23775347605923]
  Building on the success of deep learning, Generative Adversarial Networks (GANs) provide a modern approach to learn a probability distribution from observed samples. GANs often suffer from the mode collapse issue where the generator fails to capture all existing modes of the input distribution. We take an information-theoretic approach and maximize a variational lower bound on the entropy of the generated samples to increase their diversity.
  arXiv  Detail & Related papers  (2020-09-24T19:34:37Z)
• Cumulative deviation of a subpopulation from the full population [0.0]
  Assessing equity in treatment of a subpopulation often involves assigning numerical "scores" to all individuals in the full population. Given such scores, individuals with similar scores may or may not attain similar outcomes independent of the individuals' memberships in the subpopulation. The cumulative plots encode subpopulation deviation directly as the slopes of secant lines for the graphs.
  arXiv  Detail & Related papers  (2020-08-04T19:30:02Z)
• Survival Cluster Analysis [93.50540270973927]
  There is an unmet need in survival analysis for identifying subpopulations with distinct risk profiles. An approach that addresses this need is likely to improve characterization of individual outcomes.
  arXiv  Detail & Related papers  (2020-02-29T22:41:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.