FARE: Provably Fair Representation Learning with Practical Certificates

• URL: http://arxiv.org/abs/2210.07213v2
• Date: Thu, 8 Jun 2023 13:20:01 GMT
• Title: FARE: Provably Fair Representation Learning with Practical Certificates
• Authors: Nikola Jovanovi\'c, Mislav Balunovi\'c, Dimitar I. Dimitrov, Martin Vechev
• Abstract summary: We introduce FARE, the first FRL method with practical fairness certificates. FARE is based on our key insight that restricting the representation space of the encoder enables the derivation of practical guarantees. We show that FARE produces practical certificates that are tight and often even comparable with purely empirical results.
• Score: 9.242965489146398
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fair representation learning (FRL) is a popular class of methods aiming to produce fair classifiers via data preprocessing. Recent regulatory directives stress the need for FRL methods that provide practical certificates, i.e., provable upper bounds on the unfairness of any downstream classifier trained on preprocessed data, which directly provides assurance in a practical scenario. Creating such FRL methods is an important challenge that remains unsolved. In this work, we address that challenge and introduce FARE (Fairness with Restricted Encoders), the first FRL method with practical fairness certificates. FARE is based on our key insight that restricting the representation space of the encoder enables the derivation of practical guarantees, while still permitting favorable accuracy-fairness tradeoffs for suitable instantiations, such as one we propose based on fair trees. To produce a practical certificate, we develop and apply a statistical procedure that computes a finite sample high-confidence upper bound on the unfairness of any downstream classifier trained on FARE embeddings. In our comprehensive experimental evaluation, we demonstrate that FARE produces practical certificates that are tight and often even comparable with purely empirical results obtained by prior methods, which establishes the practical value of our approach.

Related papers

• Back to the Drawing Board for Fair Representation Learning [2.7379431425414684]
  The evaluation of Fair Representation Learning (FRL) methods primarily focuses on the tradeoff between downstream fairness and accuracy with respect to a single task. In this work, we argue that this approach is fundamentally mismatched with the original motivation of FRL. We propose TransFair, a benchmark that satisfies four criteria that a suitable evaluation procedure should fulfill.
  arXiv  Detail & Related papers  (2024-05-28T13:23:04Z)
• A Channel-ensemble Approach: Unbiased and Low-variance Pseudo-labels is Critical for Semi-supervised Classification [61.473485511491795]
  Semi-supervised learning (SSL) is a practical challenge in computer vision. Pseudo-label (PL) methods, e.g., FixMatch and FreeMatch, obtain the State Of The Art (SOTA) performances in SSL. We propose a lightweight channel-based ensemble method to consolidate multiple inferior PLs into the theoretically guaranteed unbiased and low-variance one.
  arXiv  Detail & Related papers  (2024-03-27T09:49:37Z)
• Distribution-Free Fair Federated Learning with Small Samples [54.63321245634712]
  FedFaiREE is a post-processing algorithm developed specifically for distribution-free fair learning in decentralized settings with small samples. We provide rigorous theoretical guarantees for both fairness and accuracy, and our experimental results further provide robust empirical validation for our proposed method.
  arXiv  Detail & Related papers  (2024-02-25T17:37:53Z)
• Certifying the Fairness of KNN in the Presence of Dataset Bias [8.028344363418865]
  We propose a method for certifying the fairness of the classification result of a widely used supervised learning algorithm, the k-nearest neighbors (KNN) This is the first certification method for KNN based on three variants of the fairness definition: individual fairness, $epsilon$-fairness, and label-flipping fairness. We show effectiveness of this abstract interpretation based technique through experimental evaluation on six datasets widely used in the fairness research literature.
  arXiv  Detail & Related papers  (2023-07-17T07:09:55Z)
• Learning Fair Classifiers via Min-Max F-divergence Regularization [13.81078324883519]
  We introduce a novel min-max F-divergence regularization framework for learning fair classification models. We show that F-divergence measures possess convexity and differentiability properties. We show that the proposed framework achieves state-of-the-art performance with respect to the trade-off between accuracy and fairness.
  arXiv  Detail & Related papers  (2023-06-28T20:42:04Z)
• FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods [84.1077756698332]
  This paper introduces the Fair Fairness Benchmark (textsfFFB), a benchmarking framework for in-processing group fairness methods. We provide a comprehensive analysis of state-of-the-art methods to ensure different notions of group fairness.
  arXiv  Detail & Related papers  (2023-06-15T19:51:28Z)
• Certifying Some Distributional Fairness with Subpopulation Decomposition [20.009388617013986]
  We first formulate the certified fairness of an ML model trained on a given data distribution as an optimization problem. We then propose a general fairness certification framework and instantiate it for both sensitive shifting and general shifting scenarios. Our framework is flexible to integrate additional non-skewness constraints and we show that it provides even tighter certification under different real-world scenarios.
  arXiv  Detail & Related papers  (2022-05-31T01:17:50Z)
• Self-Certifying Classification by Linearized Deep Assignment [65.0100925582087]
  We propose a novel class of deep predictors for classifying metric data on graphs within PAC-Bayes risk certification paradigm. Building on the recent PAC-Bayes literature and data-dependent priors, this approach enables learning posterior distributions on the hypothesis space.
  arXiv  Detail & Related papers  (2022-01-26T19:59:14Z)
• RATT: Leveraging Unlabeled Data to Guarantee Generalization [96.08979093738024]
  We introduce a method that leverages unlabeled data to produce generalization bounds. We prove that our bound is valid for 0-1 empirical risk minimization. This work provides practitioners with an option for certifying the generalization of deep nets even when unseen labeled data is unavailable.
  arXiv  Detail & Related papers  (2021-05-01T17:05:29Z)
• Fair Densities via Boosting the Sufficient Statistics of Exponential Families [72.34223801798422]
  We introduce a boosting algorithm to pre-process data for fairness. Our approach shifts towards better data fitting while still ensuring a minimal fairness guarantee. Empirical results are present to display the quality of result on real-world data.
  arXiv  Detail & Related papers  (2020-12-01T00:49:17Z)
• Certified Distributional Robustness on Smoothed Classifiers [27.006844966157317]
  We propose the worst-case adversarial loss over input distributions as a robustness certificate. By exploiting duality and the smoothness property, we provide an easy-to-compute upper bound as a surrogate for the certificate.
  arXiv  Detail & Related papers  (2020-10-21T13:22:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.