Exploring Equality: An Investigation into Custom Loss Functions for Fairness Definitions

• URL: http://arxiv.org/abs/2501.01889v1
• Date: Fri, 03 Jan 2025 16:49:17 GMT
• Title: Exploring Equality: An Investigation into Custom Loss Functions for Fairness Definitions
• Authors: Gordon Lee, Simeon Sayer,
• Abstract summary: This paper creates the first fairness-driven implementation of the novel Group Accuracy Parity framework, as theoretically proposed by Gupta et al.<n>This paper concludes that GAP achieves an enhanced equilibrium between fairness and accuracy compared to COMPAS's current nationwide implementation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper explores the complex tradeoffs between various fairness metrics such as equalized odds, disparate impact, and equal opportunity and predictive accuracy within COMPAS by building neural networks trained with custom loss functions optimized to specific fairness criteria. This paper creates the first fairness-driven implementation of the novel Group Accuracy Parity (GAP) framework, as theoretically proposed by Gupta et al. (2024), and applies it to COMPAS. To operationalize and accurately compare the fairness of COMPAS models optimized to differing fairness ideals, this paper develops and proposes a combinatory analytical procedure that incorporates Pareto front and multivariate analysis, leveraging data visualizations such as violin graphs. This paper concludes that GAP achieves an enhanced equilibrium between fairness and accuracy compared to COMPAS's current nationwide implementation and alternative implementations of COMPAS optimized to more traditional fairness definitions. While this paper's algorithmic improvements of COMPAS significantly augment its fairness, external biases undermine the fairness of its implementation. Practices such as predictive policing and issues such as the lack of transparency regarding COMPAS's internal workings have contributed to the algorithm's historical injustice. In conjunction with developments regarding COMPAS's predictive methodology, legal and institutional changes must happen for COMPAS's just deployment.

Related papers

• Selecting for Less Discriminatory Algorithms: A Relational Search Framework for Navigating Fairness-Accuracy Trade-offs in Practice [0.0]
  Machine learning models are increasingly embedded into society through high-stakes decision-making.<n>As machine learning models are increasingly embedded into society through high-stakes decision-making, selecting the right algorithm for a given task, audience, and sector presents a critical challenge.<n>Traditional assessments of model fairness have often framed fairness as an objective mathematical property, treating model selection as an optimization problem under idealized informational conditions.
  arXiv  Detail & Related papers  (2025-06-02T12:28:51Z)
• A Generic Framework for Conformal Fairness [7.073917553857755]
  We formalize textitConformal Fairness, a notion of fairness using conformal predictors.<n>We provide a theoretically well-founded algorithm and associated framework to control for the gaps in coverage between different sensitive groups.
  arXiv  Detail & Related papers  (2025-05-22T01:41:12Z)
• Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness [26.49261268883266]
  We study whether and how the choice optimization algorithm can impact group fairness in deep neural networks. We show that the choice of optimization indeed influences fairness outcomes, particularly under severe imbalance. Our results highlight the role of adaptive updates as a crucial mechanism for promoting fair outcomes.
  arXiv  Detail & Related papers  (2025-04-21T06:20:50Z)
• FAIR-SIGHT: Fairness Assurance in Image Recognition via Simultaneous Conformal Thresholding and Dynamic Output Repair [4.825037489691159]
  We introduce a post-hoc framework designed to ensure fairness in computer vision systems by combining conformal prediction with a dynamic output repair mechanism. Our approach calculates a fairness-aware non-conformity score that simultaneously assesses prediction errors and fairness violations. When the non-conformity score for a new image exceeds the threshold, FAIR-SIGHT implements targeted corrective adjustments, such as logit shifts for classification and confidence recalibration for detection.
  arXiv  Detail & Related papers  (2025-04-10T02:23:06Z)
• Editable Fairness: Fine-Grained Bias Mitigation in Language Models [52.66450426729818]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Top-K Pairwise Ranking: Bridging the Gap Among Ranking-Based Measures for Multi-Label Classification [120.37051160567277]
  This paper proposes a novel measure named Top-K Pairwise Ranking (TKPR) A series of analyses show that TKPR is compatible with existing ranking-based measures. On the other hand, we establish a sharp generalization bound for the proposed framework based on a novel technique named data-dependent contraction.
  arXiv  Detail & Related papers  (2024-07-09T09:36:37Z)
• Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer [52.09480867526656]
  We identify the source of misalignment as a form of distributional shift and uncertainty in learning human preferences.<n>To mitigate overoptimization, we first propose a theoretical algorithm that chooses the best policy for an adversarially chosen reward model.<n>Using the equivalence between reward models and the corresponding optimal policy, the algorithm features a simple objective that combines a preference optimization loss and a supervised learning loss.
  arXiv  Detail & Related papers  (2024-05-26T05:38:50Z)
• Flexible Fairness-Aware Learning via Inverse Conditional Permutation [0.0]
  We introduce an in-processing fairness-aware learning approach, FairICP, which integrates adversarial learning with a novel inverse conditional permutation scheme. We show that FairICP offers a theoretically justified, flexible, and efficient scheme to promote equalized odds under fairness conditions described by complex and multidimensional sensitive attributes.
  arXiv  Detail & Related papers  (2024-04-08T16:57:44Z)
• Learning Fair Classifiers via Min-Max F-divergence Regularization [13.81078324883519]
  We introduce a novel min-max F-divergence regularization framework for learning fair classification models. We show that F-divergence measures possess convexity and differentiability properties. We show that the proposed framework achieves state-of-the-art performance with respect to the trade-off between accuracy and fairness.
  arXiv  Detail & Related papers  (2023-06-28T20:42:04Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach [72.19525160912943]
  We first theoretically demonstrate the inherent connection between distribution shift, data perturbation, and model weight perturbation. We then analyze the sufficient conditions to guarantee fairness for the target dataset. Motivated by these sufficient conditions, we propose robust fairness regularization (RFR)
  arXiv  Detail & Related papers  (2023-03-06T17:19:23Z)
• Beyond Incompatibility: Trade-offs between Mutually Exclusive Fairness Criteria in Machine Learning and Law [2.959308758321417]
  We present a novel algorithm (FAir Interpolation Method: FAIM) for continuously interpolating between three fairness criteria.<n>We demonstrate the effectiveness of our algorithm when applied to synthetic data, the COMPAS data set, and a new, real-world data set from the e-commerce sector.
  arXiv  Detail & Related papers  (2022-12-01T12:47:54Z)
• Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing [26.704236797908177]
  algorithmic decision-making systems are becoming more pervasive. Due to the inherent trade-off between measures and accuracy, it is desirable to ensure the trade-off between overall loss and other criteria. We propose a definition-agnostic, meaning that any well-defined notion of can be reduced to the PEF notion.
  arXiv  Detail & Related papers  (2021-04-04T15:49:35Z)
• Towards Fair Knowledge Transfer for Imbalanced Domain Adaptation [61.317911756566126]
  We propose a Towards Fair Knowledge Transfer framework to handle the fairness challenge in imbalanced cross-domain learning. Specifically, a novel cross-domain mixup generation is exploited to augment the minority source set with target information to enhance fairness. Our model significantly improves over 20% on two benchmarks in terms of the overall accuracy.
  arXiv  Detail & Related papers  (2020-10-23T06:29:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.