The Statistical Fairness-Accuracy Frontier

• URL: http://arxiv.org/abs/2508.17622v1
• Date: Mon, 25 Aug 2025 03:01:35 GMT
• Title: The Statistical Fairness-Accuracy Frontier
• Authors: Alireza Fallah, Michael I. Jordan, Annie Ulichney,
• Abstract summary: Machine learning models must balance accuracy and fairness, but these goals often conflict.<n>A useful tool for understanding this trade-off is the fairness-accuracy frontier, which characterizes the set of models that cannot be simultaneously improved in both fairness and accuracy.<n>We study the FA frontier in the finite-sample regime, showing how it deviates from its population counterpart and quantifying the worst-case gap between them.
• Score: 50.323024516295725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning models must balance accuracy and fairness, but these goals often conflict, particularly when data come from multiple demographic groups. A useful tool for understanding this trade-off is the fairness-accuracy (FA) frontier, which characterizes the set of models that cannot be simultaneously improved in both fairness and accuracy. Prior analyses of the FA frontier provide a full characterization under the assumption of complete knowledge of population distributions -- an unrealistic ideal. We study the FA frontier in the finite-sample regime, showing how it deviates from its population counterpart and quantifying the worst-case gap between them. In particular, we derive minimax-optimal estimators that depend on the designer's knowledge of the covariate distribution. For each estimator, we characterize how finite-sample effects asymmetrically impact each group's risk, and identify optimal sample allocation strategies. Our results transform the FA frontier from a theoretical construct into a practical tool for policymakers and practitioners who must often design algorithms with limited data.

Related papers

• Reliable and Reproducible Demographic Inference for Fairness in Face Analysis [63.46525489354455]
  We propose a fully reproducible DAI pipeline that replaces conventional end-to-end training with a modular transfer learning approach.<n>We audit this pipeline across three dimensions: accuracy, fairness, and a newly introduced notion of robustness, defined via intra-identity consistency.<n>Our results show that the proposed method outperforms strong baselines, particularly on ethnicity, which is the more challenging attribute.
  arXiv  Detail & Related papers  (2025-10-23T12:22:02Z)
• Judging with Confidence: Calibrating Autoraters to Preference Distributions [56.17041629492863]
  We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population.<n>We present two learning methods tailored to different data conditions.<n>Our results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution.
  arXiv  Detail & Related papers  (2025-09-30T20:36:41Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Achievable Fairness on Your Data With Utility Guarantees [16.78730663293352]
  In machine learning fairness, training models that minimize disparity across different sensitive groups often leads to diminished accuracy. We present a computationally efficient approach to approximate the fairness-accuracy trade-off curve tailored to individual datasets. We introduce a novel methodology for quantifying uncertainty in our estimates, thereby providing practitioners with a robust framework for auditing model fairness.
  arXiv  Detail & Related papers  (2024-02-27T00:59:32Z)
• Fair Multivariate Adaptive Regression Splines for Ensuring Equity and Transparency [1.124958340749622]
  We propose a fair predictive model based on MARS that incorporates fairness measures in the learning process. MARS is a non-parametric regression model that performs feature selection, handles non-linear relationships, generates interpretable decision rules, and derives optimal splitting criteria on the variables. We apply our fairMARS model to real-world data and demonstrate its effectiveness in terms of accuracy and equity.
  arXiv  Detail & Related papers  (2024-02-23T19:02:24Z)
• Marginal Debiased Network for Fair Visual Recognition [59.05212866862219]
  We propose a novel marginal debiased network (MDN) to learn debiased representations. Our MDN can achieve a remarkable performance on under-represented samples.
  arXiv  Detail & Related papers  (2024-01-04T08:57:09Z)
• Fair and Optimal Classification via Post-Processing [10.163721748735801]
  This paper provides a complete characterization of the inherent tradeoff of demographic parity on classification problems. We show that the minimum error rate achievable by randomized and attribute-aware fair classifiers is given by the optimal value of a Wasserstein-barycenter problem.
  arXiv  Detail & Related papers  (2022-11-03T00:04:04Z)
• To the Fairness Frontier and Beyond: Identifying, Quantifying, and Optimizing the Fairness-Accuracy Pareto Frontier [1.5293427903448022]
  Algorithmic fairness has emerged as an important consideration when using machine learning to make high-stakes societal decisions. Yet, improved fairness often comes at the expense of model accuracy. We seek to identify, quantify, and optimize the empirical Pareto frontier of the fairness-accuracy tradeoff.
  arXiv  Detail & Related papers  (2022-05-31T19:35:53Z)
• Metrizing Fairness [5.323439381187456]
  We study supervised learning problems that have significant effects on individuals from two demographic groups. We seek predictors that are fair with respect to a group fairness criterion such as statistical parity (SP) In this paper, we identify conditions under which hard SP constraints are guaranteed to improve predictive accuracy.
  arXiv  Detail & Related papers  (2022-05-30T12:28:10Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Entropy-based adaptive design for contour finding and estimating reliability [0.24466725954625884]
  In reliability analysis, methods used to estimate failure probability are often limited by the costs associated with model evaluations. We introduce an entropy-based GP adaptive design that, when paired with MFIS, provides more accurate failure probability estimates. Illustrative examples are provided on benchmark data as well as an application to an impact damage simulator for National Aeronautics and Space Administration (NASA) spacesuits.
  arXiv  Detail & Related papers  (2021-05-24T15:41:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.