Group Distributionally Robust Machine Learning under Group Level Distributional Uncertainty

• URL: http://arxiv.org/abs/2509.08942v1
• Date: Wed, 10 Sep 2025 19:08:17 GMT
• Title: Group Distributionally Robust Machine Learning under Group Level Distributional Uncertainty
• Authors: Xenia Konti, Yi Shen, Zifan Wang, Karl Henrik Johansson, Michael J. Pencina, Nicoleta J. Economou-Zavlanos, Michael M. Zavlanos,
• Abstract summary: We propose a novel framework that relies on Wasserstein-based distributionally robust optimization (DRO) to account for the distributional uncertainty within each group.<n>We develop a gradient descent-ascent algorithm to solve the proposed DRO problem and provide convergence results.
• Score: 14.693433974739213
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The performance of machine learning (ML) models critically depends on the quality and representativeness of the training data. In applications with multiple heterogeneous data generating sources, standard ML methods often learn spurious correlations that perform well on average but degrade performance for atypical or underrepresented groups. Prior work addresses this issue by optimizing the worst-group performance. However, these approaches typically assume that the underlying data distributions for each group can be accurately estimated using the training data, a condition that is frequently violated in noisy, non-stationary, and evolving environments. In this work, we propose a novel framework that relies on Wasserstein-based distributionally robust optimization (DRO) to account for the distributional uncertainty within each group, while simultaneously preserving the objective of improving the worst-group performance. We develop a gradient descent-ascent algorithm to solve the proposed DRO problem and provide convergence results. Finally, we validate the effectiveness of our method on real-world data.

Related papers

• Robust Mixture Models for Algorithmic Fairness Under Latent Heterogeneity [8.425890077048374]
  We introduce ROME, a framework that learns latent group structure from data while optimizing for worst-group performance.<n>ROME significantly improves algorithmic fairness compared to standard methods while maintaining competitive average performance.<n>Our method requires no predefined group labels, making it practical when sources of disparities are unknown or evolving.
  arXiv  Detail & Related papers  (2025-09-22T07:03:33Z)
• Distributionally Robust Optimization with Adversarial Data Contamination [49.89480853499918]
  We focus on optimizing Wasserstein-1 DRO objectives for generalized linear models with convex Lipschitz loss functions.<n>Our primary contribution lies in a novel modeling framework that integrates robustness against training data contamination with robustness against distributional shifts.<n>This work establishes the first rigorous guarantees, supported by efficient computation, for learning under the dual challenges of data contamination and distributional shifts.
  arXiv  Detail & Related papers  (2025-07-14T18:34:10Z)
• Uncertainty Aware Learning for Language Model Alignment [97.36361196793929]
  We propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios. We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples. Experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-06-07T11:37:45Z)
• Mitigating Group Bias in Federated Learning for Heterogeneous Devices [1.181206257787103]
  Federated Learning is emerging as a privacy-preserving model training approach in distributed edge applications. Our work proposes a group-fair FL framework that minimizes group-bias while preserving privacy and without resource utilization overhead.
  arXiv  Detail & Related papers  (2023-09-13T16:53:48Z)
• Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization [61.39201891894024]
  Group distributionally robust optimization (group DRO) can minimize the worst-case loss over pre-defined groups. We reformulate the group DRO framework by proposing Q-Diversity. Characterized by an interactive training mode, Q-Diversity relaxes the group identification from annotation into direct parameterization.
  arXiv  Detail & Related papers  (2023-05-20T07:02:27Z)
• Ranking & Reweighting Improves Group Distributional Robustness [14.021069321266516]
  We propose a ranking-based training method called Discounted Rank Upweighting (DRU) to learn models that exhibit strong OOD performance on the test data. Results on several synthetic and real-world datasets highlight the superior ability of our group-ranking-based (akin to soft-minimax) approach in selecting and learning models that are robust to group distributional shifts.
  arXiv  Detail & Related papers  (2023-05-09T20:37:16Z)
• Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach [72.19525160912943]
  We first theoretically demonstrate the inherent connection between distribution shift, data perturbation, and model weight perturbation. We then analyze the sufficient conditions to guarantee fairness for the target dataset. Motivated by these sufficient conditions, we propose robust fairness regularization (RFR)
  arXiv  Detail & Related papers  (2023-03-06T17:19:23Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation [1.370633147306388]
  Machine learning systems based on minimizing average error have been shown to perform inconsistently across notable subsets of the data. In social and economic applications, where data represent people, this can lead to discrimination underrepresented gender and ethnic groups.
  arXiv  Detail & Related papers  (2021-06-17T13:18:03Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)
• Dynamic Federated Learning [57.14673504239551]
  Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
  arXiv  Detail & Related papers  (2020-02-20T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.