FAIRM: Learning invariant representations for algorithmic fairness and domain generalization with minimax optimality

• URL: http://arxiv.org/abs/2404.01608v1
• Date: Tue, 2 Apr 2024 03:06:25 GMT
• Title: FAIRM: Learning invariant representations for algorithmic fairness and domain generalization with minimax optimality
• Authors: Sai Li, Linjun Zhang,
• Abstract summary: We propose a training environment-based oracle, FAIRM, which has desirable fairness and domain generalization properties under a diversity-type condition. We develop efficient algorithms to realize FAIRM in linear models and demonstrate the nonasymptotic performance with minimax optimality.
• Score: 15.71499916304475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning methods often assume that the test data have the same distribution as the training data. However, this assumption may not hold due to multiple levels of heterogeneity in applications, raising issues in algorithmic fairness and domain generalization. In this work, we address the problem of fair and generalizable machine learning by invariant principles. We propose a training environment-based oracle, FAIRM, which has desirable fairness and domain generalization properties under a diversity-type condition. We then provide an empirical FAIRM with finite-sample theoretical guarantees under weak distributional assumptions. We then develop efficient algorithms to realize FAIRM in linear models and demonstrate the nonasymptotic performance with minimax optimality. We evaluate our method in numerical experiments with synthetic data and MNIST data and show that it outperforms its counterparts.

Related papers

• Dr. FERMI: A Stochastic Distributionally Robust Fair Empirical Risk Minimization Framework [12.734559823650887]
  In the presence of distribution shifts, fair machine learning models may behave unfairly on test data. Existing algorithms require full access to data and cannot be used when small batches are used. This paper proposes the first distributionally robust fairness framework with convergence guarantees that do not require knowledge of the causal graph.
  arXiv  Detail & Related papers  (2023-09-20T23:25:28Z)
• Coping with Change: Learning Invariant and Minimum Sufficient Representations for Fine-Grained Visual Categorization [26.254072665916155]
  Fine-grained visual categorization (FGVC) is a challenging task due to similar visual appearances between various species. Previous studies assume that the training and test data have the same underlying distributions, and that features extracted by modern backbone architectures remain discriminative and generalize well to unseen test data. We combine the merits of invariant risk minimization (IRM) and information bottleneck (IB) principle to learn invariant and minimum sufficient (IMS) representations for FGVC.
  arXiv  Detail & Related papers  (2023-06-08T02:45:15Z)
• Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach [72.19525160912943]
  We first theoretically demonstrate the inherent connection between distribution shift, data perturbation, and model weight perturbation. We then analyze the sufficient conditions to guarantee fairness for the target dataset. Motivated by these sufficient conditions, we propose robust fairness regularization (RFR)
  arXiv  Detail & Related papers  (2023-03-06T17:19:23Z)
• Making Linear MDPs Practical via Contrastive Representation Learning [101.75885788118131]
  It is common to address the curse of dimensionality in Markov decision processes (MDPs) by exploiting low-rank representations. We consider an alternative definition of linear MDPs that automatically ensures normalization while allowing efficient representation learning. We demonstrate superior performance over existing state-of-the-art model-based and model-free algorithms on several benchmarks.
  arXiv  Detail & Related papers  (2022-07-14T18:18:02Z)
• Domain Generalization via Domain-based Covariance Minimization [4.414778226415752]
  We propose a novel variance measurement for multiple domains so as to minimize the difference between conditional distributions across domains. We show that for small-scale datasets, we are able to achieve better quantitative results indicating better generalization performance over unseen test datasets.
  arXiv  Detail & Related papers  (2021-10-12T19:30:15Z)
• Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification [13.456851070400024]
  Invariant Risk Minimization (IRM) is a domain generalization algorithm that employs a causal discovery inspired method to find robust predictors. We show that IRM achieves better out-of-distribution accuracy and fairness than Empirical Risk Minimization (ERM) methods.
  arXiv  Detail & Related papers  (2020-11-12T16:42:14Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• On the Benefits of Invariance in Neural Networks [56.362579457990094]
  We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation. We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
  arXiv  Detail & Related papers  (2020-05-01T02:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.