Revisiting Pre-processing Group Fairness: A Modular Benchmarking Framework

• URL: http://arxiv.org/abs/2508.15193v1
• Date: Thu, 21 Aug 2025 03:04:30 GMT
• Title: Revisiting Pre-processing Group Fairness: A Modular Benchmarking Framework
• Authors: Brodie Oldfield, Ziqi Xu, Sevvandi Kandanaarachchi,
• Abstract summary: Methods to mitigate bias fall into three categories: pre-processing, in-processing, and post-processing.<n>FairPrep is a benchmarking framework designed to evaluate fairness-aware pre-processing techniques on datasets.
• Score: 3.035039100561926
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As machine learning systems become increasingly integrated into high-stakes decision-making processes, ensuring fairness in algorithmic outcomes has become a critical concern. Methods to mitigate bias typically fall into three categories: pre-processing, in-processing, and post-processing. While significant attention has been devoted to the latter two, pre-processing methods, which operate at the data level and offer advantages such as model-agnosticism and improved privacy compliance, have received comparatively less focus and lack standardised evaluation tools. In this work, we introduce FairPrep, an extensible and modular benchmarking framework designed to evaluate fairness-aware pre-processing techniques on tabular datasets. Built on the AIF360 platform, FairPrep allows seamless integration of datasets, fairness interventions, and predictive models. It features a batch-processing interface that enables efficient experimentation and automatic reporting of fairness and utility metrics. By offering standardised pipelines and supporting reproducible evaluations, FairPrep fills a critical gap in the fairness benchmarking landscape and provides a practical foundation for advancing data-level fairness research.

Related papers

• Pushing the Boundaries of Interpretability: Incremental Enhancements to the Explainable Boosting Machine [1.2461503242570642]
  This paper aims at improving the Explainable Boosting Machine (EBM), a state-of-the-art glassbox model that delivers both high accuracy and complete transparency.<n>The work is positioned as a critical step toward developing machine learning systems that are robust, equitable, and transparent.
  arXiv  Detail & Related papers  (2025-11-29T15:46:13Z)
• Reliable and Reproducible Demographic Inference for Fairness in Face Analysis [63.46525489354455]
  We propose a fully reproducible DAI pipeline that replaces conventional end-to-end training with a modular transfer learning approach.<n>We audit this pipeline across three dimensions: accuracy, fairness, and a newly introduced notion of robustness, defined via intra-identity consistency.<n>Our results show that the proposed method outperforms strong baselines, particularly on ethnicity, which is the more challenging attribute.
  arXiv  Detail & Related papers  (2025-10-23T12:22:02Z)
• General Post-Processing Framework for Fairness Adjustment of Machine Learning Models [0.0]
  This paper introduces a novel framework for fairness adjustments that applies to diverse machine learning tasks.<n>By decoupling fairness adjustments from the model training process, our framework preserves model performance on average.<n>We demonstrate the effectiveness of this approach by comparing it to Adversarial Debiasing, showing that our framework achieves a comparable fairness/accuracy tradeoff on real-world datasets.
  arXiv  Detail & Related papers  (2025-04-22T20:06:59Z)
• Data Preparation for Fairness-Performance Trade-Offs: A Practitioner-Friendly Alternative? [11.172805305320592]
  Pre-processing techniques, which mitigate bias before training, are effective but may impact model performance and pose integration difficulties.<n>This report proposes an empirical evaluation of how optimally selected fairness-aware practices, applied in early ML lifecycle stages, can enhance both fairness and performance.<n>Using FATE, we will analyze the fairness-performance trade-off, comparing pipelines selected by FATE with results by pre-processing bias mitigation techniques.
  arXiv  Detail & Related papers  (2024-12-20T14:12:39Z)
• Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark [95.0484665958967]
  Pre-trained vision models (PVMs) have demonstrated remarkable adaptability across a wide range of downstream vision tasks.<n>As these models scale to billions or even trillions of parameters, conventional full fine-tuning has become increasingly impractical due to its high computational and storage demands.<n> parameter-efficient fine-tuning (PEFT) has emerged as a promising alternative, aiming to achieve performance comparable to full fine-tuning while making minimal adjustments to the model parameters.
  arXiv  Detail & Related papers  (2024-02-03T19:12:20Z)
• FRAPPE: A Group Fairness Framework for Post-Processing Everything [48.57876348370417]
  We propose a framework that turns any regularized in-processing method into a post-processing approach. We show theoretically and through experiments that our framework preserves the good fairness-error trade-offs achieved with in-processing.
  arXiv  Detail & Related papers  (2023-12-05T09:09:21Z)
• FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods [84.1077756698332]
  This paper introduces the Fair Fairness Benchmark (textsfFFB), a benchmarking framework for in-processing group fairness methods. We provide a comprehensive analysis of state-of-the-art methods to ensure different notions of group fairness.
  arXiv  Detail & Related papers  (2023-06-15T19:51:28Z)
• Improving Fair Training under Correlation Shifts [33.385118640843416]
  In particular, when the bias between labels and sensitive groups changes, the fairness of the trained model is directly influenced and can worsen. We analytically show that existing in-processing fair algorithms have fundamental limits in accuracy and group fairness. We propose a novel pre-processing step that samples the input data to reduce correlation shifts.
  arXiv  Detail & Related papers  (2023-02-05T07:23:35Z)
• An Operational Perspective to Fairness Interventions: Where and How to Intervene [9.833760837977222]
  We present a holistic framework for evaluating and contextualizing fairness interventions. We demonstrate our framework with a case study on predictive parity. We find predictive parity is difficult to achieve without using group data.
  arXiv  Detail & Related papers  (2023-02-03T07:04:33Z)
• fairlib: A Unified Framework for Assessing and Improving Classification Fairness [66.27822109651757]
  fairlib is an open-source framework for assessing and improving classification fairness. We implement 14 debiasing methods, including pre-processing, at-training-time, and post-processing approaches. The built-in metrics cover the most commonly used fairness criterion and can be further generalized and customized for fairness evaluation.
  arXiv  Detail & Related papers  (2022-05-04T03:50:23Z)
• Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking Fairness and Algorithm Utility [54.179859639868646]
  Bipartite ranking aims to learn a scoring function that ranks positive individuals higher than negative ones from labeled data. There have been rising concerns on whether the learned scoring function can cause systematic disparity across different protected groups. We propose a model post-processing framework for balancing them in the bipartite ranking scenario.
  arXiv  Detail & Related papers  (2020-06-15T10:08:39Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.