Enhancing Fairness through Reweighting: A Path to Attain the Sufficiency Rule

• URL: http://arxiv.org/abs/2408.14126v2
• Date: Tue, 1 Oct 2024 13:18:35 GMT
• Title: Enhancing Fairness through Reweighting: A Path to Attain the Sufficiency Rule
• Authors: Xuan Zhao, Klaus Broelemann, Salvatore Ruggieri, Gjergji Kasneci,
• Abstract summary: We introduce an innovative approach to enhancing the empirical risk minimization process in model training. This scheme aims to uphold the sufficiency rule in fairness by ensuring that optimal predictors maintain consistency across diverse sub-groups.
• Score: 23.335423207588466
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce an innovative approach to enhancing the empirical risk minimization (ERM) process in model training through a refined reweighting scheme of the training data to enhance fairness. This scheme aims to uphold the sufficiency rule in fairness by ensuring that optimal predictors maintain consistency across diverse sub-groups. We employ a bilevel formulation to address this challenge, wherein we explore sample reweighting strategies. Unlike conventional methods that hinge on model size, our formulation bases generalization complexity on the space of sample weights. We discretize the weights to improve training speed. Empirical validation of our method showcases its effectiveness and robustness, revealing a consistent improvement in the balance between prediction performance and fairness metrics across various experiments.

Related papers

• Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining [55.262510814326035]
  Existing reweighting strategies primarily focus on group-level data importance. We introduce novel algorithms for dynamic, instance-level data reweighting. Our framework allows us to devise reweighting strategies deprioritizing redundant or uninformative data.
  arXiv  Detail & Related papers  (2025-02-10T17:57:15Z)
• Balance-aware Sequence Sampling Makes Multi-modal Learning Better [0.5439020425819]
  We propose Balance-aware Sequence Sampling (BSS) to enhance the robustness of MML. Via a multi-perspective measurer, we first define a multi-perspective measurer to evaluate the balance degree of each sample. We employ a scheduler based on curriculum learning (CL) that incrementally provides training subsets, progressing from balanced to imbalanced samples to rebalance MML.
  arXiv  Detail & Related papers  (2025-01-01T06:19:55Z)
• Model-Based Reward Shaping for Adversarial Inverse Reinforcement Learning in Stochastic Environments [11.088387316161064]
  We tackle the limitation of the Adrial Inverse Reinforcement Learning (AIRL) method in environments where theoretical results cannot hold and performance is degraded. We propose a novel method which infuses the dynamics information into the reward shaping with the theoretical guarantee for the induced optimal policy in the environments. We present a novel Model-Enhanced AIRL framework, which integrates transition model estimation directly into reward shaping.
  arXiv  Detail & Related papers  (2024-10-04T18:27:37Z)
• Preference-Based Multi-Agent Reinforcement Learning: Data Coverage and Algorithmic Techniques [65.55451717632317]
  We study Preference-Based Multi-Agent Reinforcement Learning (PbMARL) We identify the Nash equilibrium from a preference-only offline dataset in general-sum games. Our findings underscore the multifaceted approach required for PbMARL.
  arXiv  Detail & Related papers  (2024-09-01T13:14:41Z)
• Relaxed Contrastive Learning for Federated Learning [48.96253206661268]
  We propose a novel contrastive learning framework to address the challenges of data heterogeneity in federated learning. Our framework outperforms all existing federated learning approaches by huge margins on the standard benchmarks.
  arXiv  Detail & Related papers  (2024-01-10T04:55:24Z)
• MAST: Model-Agnostic Sparsified Training [4.962431253126472]
  We introduce a novel optimization problem formulation that departs from the conventional way of minimizing machine learning model loss as a black-box function. Unlike traditional formulations, the proposed approach explicitly incorporates an initially pre-trained model and random sketch operators. We present several variants of the Gradient Descent (SGD) method adapted to the new problem formulation.
  arXiv  Detail & Related papers  (2023-11-27T18:56:03Z)
• Aggregation Weighting of Federated Learning via Generalization Bound Estimation [65.8630966842025]
  Federated Learning (FL) typically aggregates client model parameters using a weighting approach determined by sample proportions. We replace the aforementioned weighting method with a new strategy that considers the generalization bounds of each local model.
  arXiv  Detail & Related papers  (2023-11-10T08:50:28Z)
• Stratified Learning: A General-Purpose Statistical Method for Improved Learning under Covariate Shift [1.1470070927586016]
  We propose a simple, statistically principled, and theoretically justified method to improve supervised learning when the training set is not representative. We build upon a well-established methodology in causal inference, and show that the effects of covariate shift can be reduced or eliminated by conditioning on propensity scores. We demonstrate the effectiveness of our general-purpose method on two contemporary research questions in cosmology, outperforming state-of-the-art importance weighting methods.
  arXiv  Detail & Related papers  (2021-06-21T15:53:20Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)
• Learning the Truth From Only One Side of the Story [58.65439277460011]
  We focus on generalized linear models and show that without adjusting for this sampling bias, the model may converge suboptimally or even fail to converge to the optimal solution. We propose an adaptive approach that comes with theoretical guarantees and show that it outperforms several existing methods empirically.
  arXiv  Detail & Related papers  (2020-06-08T18:20:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.