The Fair Game: Auditing & Debiasing AI Algorithms Over Time

• URL: http://arxiv.org/abs/2508.06443v1
• Date: Fri, 08 Aug 2025 16:36:16 GMT
• Title: The Fair Game: Auditing & Debiasing AI Algorithms Over Time
• Authors: Debabrota Basu, Udvas Das,
• Abstract summary: "Fair Game" puts together an Auditor and a Debiasing algorithm in a loop around an ML algorithm.<n>"Fair Game" provides a unique framework where the fairness goals can be adapted over time.<n>This allows us to develop a flexible and adaptive-over-time framework to build Fair ML systems pre- and post-deployment.
• Score: 8.784438985280094
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: An emerging field of AI, namely Fair Machine Learning (ML), aims to quantify different types of bias (also known as unfairness) exhibited in the predictions of ML algorithms, and to design new algorithms to mitigate them. Often, the definitions of bias used in the literature are observational, i.e. they use the input and output of a pre-trained algorithm to quantify a bias under concern. In reality,these definitions are often conflicting in nature and can only be deployed if either the ground truth is known or only in retrospect after deploying the algorithm. Thus,there is a gap between what we want Fair ML to achieve and what it does in a dynamic social environment. Hence, we propose an alternative dynamic mechanism,"Fair Game",to assure fairness in the predictions of an ML algorithm and to adapt its predictions as the society interacts with the algorithm over time. "Fair Game" puts together an Auditor and a Debiasing algorithm in a loop around an ML algorithm. The "Fair Game" puts these two components in a loop by leveraging Reinforcement Learning (RL). RL algorithms interact with an environment to take decisions, which yields new observations (also known as data/feedback) from the environment and in turn, adapts future decisions. RL is already used in algorithms with pre-fixed long-term fairness goals. "Fair Game" provides a unique framework where the fairness goals can be adapted over time by only modifying the auditor and the different biases it quantifies. Thus,"Fair Game" aims to simulate the evolution of ethical and legal frameworks in the society by creating an auditor which sends feedback to a debiasing algorithm deployed around an ML system. This allows us to develop a flexible and adaptive-over-time framework to build Fair ML systems pre- and post-deployment.

Related papers

• Lookahead Counterfactual Fairness [13.803918985391451]
  Concerns have arisen that machine learning algorithms may be biased against certain social groups.<n>We introduce textitlook counterfactual (LCF), a notion accounting for the downstream effects of ML models.
  arXiv  Detail & Related papers  (2024-12-02T02:53:14Z)
• Provable Scaling Laws for the Test-Time Compute of Large Language Models [84.00141420901038]
  We propose two algorithms that enjoy provable scaling laws for the test-time compute of large language models.<n>One is a two-stage knockout-style algorithm, where each candidate is evaluated by its average win rate against multiple opponents.<n>The other is a two-stage league-style algorithm, where each candidate is evaluated by its average win rate against multiple opponents.
  arXiv  Detail & Related papers  (2024-11-29T05:29:47Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• Understanding Unfairness in Fraud Detection through Model and Data Bias Interactions [4.159343412286401]
  We argue that algorithmic unfairness stems from interactions between models and biases in the data. We study a set of hypotheses regarding the fairness-accuracy trade-offs that fairness-blind ML algorithms exhibit under different data bias settings.
  arXiv  Detail & Related papers  (2022-07-13T15:18:30Z)
• ABCinML: Anticipatory Bias Correction in Machine Learning Applications [9.978142416219294]
  We propose an anticipatory dynamic learning approach for correcting the algorithm to mitigate bias before it occurs. Results from experiments over multiple real-world datasets suggest that this approach has promise for anticipatory bias correction.
  arXiv  Detail & Related papers  (2022-06-14T16:26:10Z)
• Domain Adaptation meets Individual Fairness. And they get along [48.95808607591299]
  We show that algorithmic fairness interventions can help machine learning models overcome distribution shifts. In particular, we show that enforcing suitable notions of individual fairness (IF) can improve the out-of-distribution accuracy of ML models.
  arXiv  Detail & Related papers  (2022-05-01T16:19:55Z)
• Are There Exceptions to Goodhart's Law? On the Moral Justification of Fairness-Aware Machine Learning [14.428360876120333]
  We argue that fairness measures are particularly sensitive to Goodhart's law. We present a framework for moral reasoning about the justification of fairness metrics.
  arXiv  Detail & Related papers  (2022-02-17T09:26:39Z)
• Fair Group-Shared Representations with Normalizing Flows [68.29997072804537]
  We develop a fair representation learning algorithm which is able to map individuals belonging to different groups in a single group. We show experimentally that our methodology is competitive with other fair representation learning algorithms.
  arXiv  Detail & Related papers  (2022-01-17T10:49:49Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• Metrics and methods for a systematic comparison of fairness-aware machine learning algorithms [0.0]
  This study is the most comprehensive of its kind. It considers fairness, predictive-performance, calibration quality, and speed of 28 different modelling pipelines. We also found that fairness-aware algorithms can induce fairness without material drops in predictive power.
  arXiv  Detail & Related papers  (2020-10-08T13:58:09Z)
• Discovering Reinforcement Learning Algorithms [53.72358280495428]
  Reinforcement learning algorithms update an agent's parameters according to one of several possible rules. This paper introduces a new meta-learning approach that discovers an entire update rule. It includes both 'what to predict' (e.g. value functions) and 'how to learn from it' by interacting with a set of environments.
  arXiv  Detail & Related papers  (2020-07-17T07:38:39Z)
• Transparency Tools for Fairness in AI (Luskin) [12.158766675246337]
  We propose new tools for assessing and correcting fairness and bias in AI algorithms. The three tools are: - A new definition of fairness called "controlled fairness" with respect to choices of protected features and filters. They are useful for understanding various dimensions of bias, and that in practice the algorithms are effective in starkly reducing a given observed bias when tested on new data.
  arXiv  Detail & Related papers  (2020-07-09T00:21:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.