Fairness-Aware Mixture-of-Experts via Subgroup Reweighting and Gate Regularization

• URL: http://arxiv.org/abs/2608.22820v1
• Date: Mon, 24 Aug 2026 05:34:25 GMT
• Title: Fairness-Aware Mixture-of-Experts via Subgroup Reweighting and Gate Regularization
• Abstract summary: We propose an end-to-end Mixture-of-Experts (MoE) framework that corrects subgroup imbalance.<n>We apply subgroup reweighting to correct data imbalance, and introduce gate entropy regularization to prevent routing from collapsing onto subgroup attributes.<n> Experimental results demonstrate that the proposed approach improves fairness while maintaining competitive predictive performance.
• Score: 2.6245086561385276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning models often produce performance disparities across demographic groups, due to the training data imbalance with respect to sensitive attributes such as gender or age. To address this problem, existing work has explored fair representation learning, data re-sampling, and adversarial training, which can be broadly categorized into two main approaches. Single-stage methods typically learn a shared representation for fairness, but often struggle to handle heterogeneous subgroup distributions. Two-stage methods learn representations separately from the final prediction task, which can lead to misalignment between fairness objectives and downstream predictions. We identify routing-induced bias, a failure mode in which subgroup imbalance drives the gating network to route subgroups onto a few experts, and propose an end-to-end Mixture-of-Experts (MoE) framework that corrects it. Specifically, we apply subgroup reweighting to correct data imbalance, and introduce gate entropy regularization to prevent routing from collapsing onto subgroup attributes, keeping expert utilization both balanced and interpretable. Beyond improving fairness, the routing distribution offers an interpretable view of how subgroups are allocated across experts. Experimental results demonstrate that the proposed approach improves fairness while maintaining competitive predictive performance.

Related papers

• Procedural Fairness via Group Counterfactual Explanation [0.0]
  Group Counterfactual Integrated Gradients (GCIG) is an in-processing regularization framework that enforces explanation invariance across groups.<n>We show that GCIG substantially reduces cross-group explanation disparity while maintaining competitive predictive performance.<n>Our results also show that aligning model reasoning across groups offers a principled and practical avenue for advancing fairness beyond outcome parity.
  arXiv  Detail & Related papers  (2026-03-11T16:38:59Z)
• Class-Conditional Distribution Balancing for Group Robust Classification [11.525201208566925]
  Spurious correlations that lead models to correct predictions for the wrong reasons pose a critical challenge for robust real-world generalization.<n>We offer a novel perspective by reframing the spurious correlations as imbalances or mismatches in class-conditional distributions.<n>We propose a simple yet effective robust learning method that eliminates the need for both bias annotations and predictions.
  arXiv  Detail & Related papers  (2025-04-24T07:15:53Z)
• Fairness of Deep Ensembles: On the interplay between per-group task difficulty and under-representation [9.11104048176204]
  Ensembling is commonly regarded as an effective way to improve the general performance of models in machine learning.<n>We show how a simple and straightforward method is able to mitigate disparities, particularly benefiting under-performing subgroups.<n>We analyzed the interplay between two factors which may result in biases: sub-group under-representation and the inherent difficulty of the task for each group.
  arXiv  Detail & Related papers  (2025-01-24T14:54:01Z)
• Learning Informative Representation for Fairness-aware Multivariate Time-series Forecasting: A Group-based Perspective [50.093280002375984]
  Performance unfairness among variables widely exists in multivariate time series (MTS) forecasting models. We propose a novel framework, named FairFor, for fairness-aware MTS forecasting.
  arXiv  Detail & Related papers  (2023-01-27T04:54:12Z)
• Domain Adaptation meets Individual Fairness. And they get along [48.95808607591299]
  We show that algorithmic fairness interventions can help machine learning models overcome distribution shifts. In particular, we show that enforcing suitable notions of individual fairness (IF) can improve the out-of-distribution accuracy of ML models.
  arXiv  Detail & Related papers  (2022-05-01T16:19:55Z)
• Addressing Missing Sources with Adversarial Support-Matching [8.53946780558779]
  We investigate a scenario in which the absence of certain data is linked to the second level of a two-level hierarchy in the data. Inspired by the idea of protected groups from algorithmic fairness, we refer to the partitions carved by this second level as "subgroups" We make use of an additional, diverse but unlabeled dataset, called the "deployment set", to learn a representation that is invariant to subgroup.
  arXiv  Detail & Related papers  (2022-03-24T16:19:19Z)
• Fair Group-Shared Representations with Normalizing Flows [68.29997072804537]
  We develop a fair representation learning algorithm which is able to map individuals belonging to different groups in a single group. We show experimentally that our methodology is competitive with other fair representation learning algorithms.
  arXiv  Detail & Related papers  (2022-01-17T10:49:49Z)
• Contrastive Learning for Fair Representations [50.95604482330149]
  Trained classification models can unintentionally lead to biased representations and predictions. Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise. We propose a method for mitigating bias by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations.
  arXiv  Detail & Related papers  (2021-09-22T10:47:51Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)
• Fair Mixup: Fairness via Interpolation [28.508444261249423]
  We propose fair mixup, a new data augmentation strategy for imposing the fairness constraint. We show that fairness can be achieved by regularizing the models on paths of interpolated samples between the groups. We empirically show that it ensures a better generalization for both accuracy and fairness measurement in benchmarks.
  arXiv  Detail & Related papers  (2021-03-11T06:57:26Z)
• Global Distance-distributions Separation for Unsupervised Person Re-identification [93.39253443415392]
  Existing unsupervised ReID approaches often fail in correctly identifying the positive samples and negative samples through the distance-based matching/ranking. We introduce a global distance-distributions separation constraint over the two distributions to encourage the clear separation of positive and negative samples from a global view. We show that our method leads to significant improvement over the baselines and achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-06-01T07:05:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.