Robust Mixture Models for Algorithmic Fairness Under Latent Heterogeneity

• URL: http://arxiv.org/abs/2509.17411v1
• Date: Mon, 22 Sep 2025 07:03:33 GMT
• Title: Robust Mixture Models for Algorithmic Fairness Under Latent Heterogeneity
• Authors: Siqi Li, Molei Liu, Ziye Tian, Chuan Hong, Nan Liu,
• Abstract summary: We introduce ROME, a framework that learns latent group structure from data while optimizing for worst-group performance.<n>ROME significantly improves algorithmic fairness compared to standard methods while maintaining competitive average performance.<n>Our method requires no predefined group labels, making it practical when sources of disparities are unknown or evolving.
• Score: 8.425890077048374
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Standard machine learning models optimized for average performance often fail on minority subgroups and lack robustness to distribution shifts. This challenge worsens when subgroups are latent and affected by complex interactions among continuous and discrete features. We introduce ROME (RObust Mixture Ensemble), a framework that learns latent group structure from data while optimizing for worst-group performance. ROME employs two approaches: an Expectation-Maximization algorithm for linear models and a neural Mixture-of-Experts for nonlinear settings. Through simulations and experiments on real-world datasets, we demonstrate that ROME significantly improves algorithmic fairness compared to standard methods while maintaining competitive average performance. Importantly, our method requires no predefined group labels, making it practical when sources of disparities are unknown or evolving.

Related papers

• Crowdsourcing Without People: Modelling Clustering Algorithms as Experts [0.0]
  mixsemble is an ensemble method that adapts the Dawid-Skene model to aggregate predictions from multiple model-based clustering algorithms.<n>Unlike traditional crowdsourcing, which relies on human labels, the framework models the outputs of clustering algorithms as noisy annotations.
  arXiv  Detail & Related papers  (2025-09-29T18:52:37Z)
• Group Distributionally Robust Machine Learning under Group Level Distributional Uncertainty [14.693433974739213]
  We propose a novel framework that relies on Wasserstein-based distributionally robust optimization (DRO) to account for the distributional uncertainty within each group.<n>We develop a gradient descent-ascent algorithm to solve the proposed DRO problem and provide convergence results.
  arXiv  Detail & Related papers  (2025-09-10T19:08:17Z)
• Regularized Neural Ensemblers [55.15643209328513]
  In this study, we explore employing regularized neural networks as ensemble methods.<n>Motivated by the risk of learning low-diversity ensembles, we propose regularizing the ensembling model by randomly dropping base model predictions.<n>We demonstrate this approach provides lower bounds for the diversity within the ensemble, reducing overfitting and improving generalization capabilities.
  arXiv  Detail & Related papers  (2024-10-06T15:25:39Z)
• Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models [83.02797560769285]
  Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data.<n>Current methods often overlook the heterogeneity among pre-trained models, which leads to performance degradation due to task conflicts.
  arXiv  Detail & Related papers  (2024-05-26T13:11:55Z)
• Mitigating Group Bias in Federated Learning for Heterogeneous Devices [1.181206257787103]
  Federated Learning is emerging as a privacy-preserving model training approach in distributed edge applications. Our work proposes a group-fair FL framework that minimizes group-bias while preserving privacy and without resource utilization overhead.
  arXiv  Detail & Related papers  (2023-09-13T16:53:48Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization [61.39201891894024]
  Group distributionally robust optimization (group DRO) can minimize the worst-case loss over pre-defined groups. We reformulate the group DRO framework by proposing Q-Diversity. Characterized by an interactive training mode, Q-Diversity relaxes the group identification from annotation into direct parameterization.
  arXiv  Detail & Related papers  (2023-05-20T07:02:27Z)
• Ranking & Reweighting Improves Group Distributional Robustness [14.021069321266516]
  We propose a ranking-based training method called Discounted Rank Upweighting (DRU) to learn models that exhibit strong OOD performance on the test data. Results on several synthetic and real-world datasets highlight the superior ability of our group-ranking-based (akin to soft-minimax) approach in selecting and learning models that are robust to group distributional shifts.
  arXiv  Detail & Related papers  (2023-05-09T20:37:16Z)
• Take One Gram of Neural Features, Get Enhanced Group Robustness [23.541213868620837]
  Predictive performance of machine learning models trained with empirical risk minimization can degrade considerably under distribution shifts. We propose to partition the training dataset into groups based on Gram matrices of features extracted by an identification'' model. Our approach not only improves group robustness over ERM but also outperforms all recent baselines.
  arXiv  Detail & Related papers  (2022-08-26T12:34:55Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.