When Fair Classification Meets Noisy Protected Attributes

• URL: http://arxiv.org/abs/2307.03306v2
• Date: Tue, 11 Jul 2023 14:20:50 GMT
• Title: When Fair Classification Meets Noisy Protected Attributes
• Authors: Avijit Ghosh, Pablo Kvitca, Christo Wilson
• Abstract summary: This study is the first head-to-head study of fair classification algorithms to compare attribute-reliant, noise-tolerant and attribute-blind algorithms. Our study reveals that attribute-blind and noise-tolerant fair classifiers can potentially achieve similar level of performance as attribute-reliant algorithms.
• Score: 8.362098382773265
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The operationalization of algorithmic fairness comes with several practical challenges, not the least of which is the availability or reliability of protected attributes in datasets. In real-world contexts, practical and legal impediments may prevent the collection and use of demographic data, making it difficult to ensure algorithmic fairness. While initial fairness algorithms did not consider these limitations, recent proposals aim to achieve algorithmic fairness in classification by incorporating noisiness in protected attributes or not using protected attributes at all. To the best of our knowledge, this is the first head-to-head study of fair classification algorithms to compare attribute-reliant, noise-tolerant and attribute-blind algorithms along the dual axes of predictivity and fairness. We evaluated these algorithms via case studies on four real-world datasets and synthetic perturbations. Our study reveals that attribute-blind and noise-tolerant fair classifiers can potentially achieve similar level of performance as attribute-reliant algorithms, even when protected attributes are noisy. However, implementing them in practice requires careful nuance. Our study provides insights into the practical implications of using fair classification algorithms in scenarios where protected attributes are noisy or partially available.

Related papers

• Provable Optimization for Adversarial Fair Self-supervised Contrastive Learning [49.417414031031264]
  This paper studies learning fair encoders in a self-supervised learning setting. All data are unlabeled and only a small portion of them are annotated with sensitive attributes.
  arXiv  Detail & Related papers  (2024-06-09T08:11:12Z)
• Differentially Private Fair Binary Classifications [1.8087157239832476]
  We first propose an algorithm for learning a classifier with only fairness guarantee. We then refine this algorithm to incorporate differential privacy. Empirical evaluations conducted on the Adult and Credit Card datasets illustrate that our algorithm outperforms the state-of-the-art in terms of fairness guarantees.
  arXiv  Detail & Related papers  (2024-02-23T20:52:59Z)
• Group Fairness with Uncertainty in Sensitive Attributes [34.608332397776245]
  A fair predictive model is crucial to mitigate biased decisions against minority groups in high-stakes applications. We propose a bootstrap-based algorithm that achieves the target level of fairness despite the uncertainty in sensitive attributes. Our algorithm is applicable to both discrete and continuous sensitive attributes and is effective in real-world classification and regression tasks.
  arXiv  Detail & Related papers  (2023-02-16T04:33:00Z)
• Practical Approaches for Fair Learning with Multitype and Multivariate Sensitive Attributes [70.6326967720747]
  It is important to guarantee that machine learning algorithms deployed in the real world do not result in unfairness or unintended social consequences. We introduce FairCOCCO, a fairness measure built on cross-covariance operators on reproducing kernel Hilbert Spaces. We empirically demonstrate consistent improvements against state-of-the-art techniques in balancing predictive power and fairness on real-world datasets.
  arXiv  Detail & Related papers  (2022-11-11T11:28:46Z)
• Fairness via Adversarial Attribute Neighbourhood Robust Learning [49.93775302674591]
  We propose a principled underlineRobust underlineAdversarial underlineAttribute underlineNeighbourhood (RAAN) loss to debias the classification head.
  arXiv  Detail & Related papers  (2022-10-12T23:39:28Z)
• Mitigating Algorithmic Bias with Limited Annotations [65.060639928772]
  When sensitive attributes are not disclosed or available, it is needed to manually annotate a small part of the training data to mitigate bias. We propose Active Penalization Of Discrimination (APOD), an interactive framework to guide the limited annotations towards maximally eliminating the effect of algorithmic bias. APOD shows comparable performance to fully annotated bias mitigation, which demonstrates that APOD could benefit real-world applications when sensitive information is limited.
  arXiv  Detail & Related papers  (2022-07-20T16:31:19Z)
• Dikaios: Privacy Auditing of Algorithmic Fairness via Attribute Inference Attacks [0.5801044612920815]
  We propose Dikaios, a privacy auditing tool for fairness algorithms for model builders. We show that our attribute inference attacks with adaptive prediction threshold significantly outperform prior attacks.
  arXiv  Detail & Related papers  (2022-02-04T17:19:59Z)
• Fair Classification with Adversarial Perturbations [35.030329189029246]
  We study fair classification in the presence of an omniscient adversary that, given an $eta$, is allowed to choose an arbitrary $eta$-fraction of the training samples and arbitrarily perturb their protected attributes. Our main contribution is an optimization framework to learn fair classifiers in this adversarial setting that comes with provable guarantees on accuracy and fairness. We prove near-tightness of our framework's guarantees for natural hypothesis classes: no algorithm can have significantly better accuracy and any algorithm with better fairness must have lower accuracy.
  arXiv  Detail & Related papers  (2021-06-10T17:56:59Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• Mitigating Bias in Set Selection with Noisy Protected Attributes [16.882719401742175]
  We show that in the presence of noisy protected attributes, in attempting to increase fairness without considering noise, one can, in fact, decrease the fairness of the result! We formulate a denoised'' selection problem which functions for a large class of fairness metrics. Our empirical results show that this approach can produce subsets which significantly improve the fairness metrics despite the presence of noisy protected attributes.
  arXiv  Detail & Related papers  (2020-11-09T06:45:15Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.