Efficient fair PCA for fair representation learning

• URL: http://arxiv.org/abs/2302.13319v1
• Date: Sun, 26 Feb 2023 13:34:43 GMT
• Title: Efficient fair PCA for fair representation learning
• Authors: Matth\"aus Kleindessner, Michele Donini, Chris Russell, Muhammad Bilal Zafar
• Abstract summary: We propose a conceptually simple approach that allows for an analytic solution similar to standard PCA and can be kernelized. Our methods have the same complexity as standard PCA, or kernel PCA, and run much faster than existing methods for fair PCA based on semidefinite programming or manifold optimization.
• Score: 21.990310743597174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We revisit the problem of fair principal component analysis (PCA), where the goal is to learn the best low-rank linear approximation of the data that obfuscates demographic information. We propose a conceptually simple approach that allows for an analytic solution similar to standard PCA and can be kernelized. Our methods have the same complexity as standard PCA, or kernel PCA, and run much faster than existing methods for fair PCA based on semidefinite programming or manifold optimization, while achieving similar results.

Related papers

• Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation [62.2436697657307]
  Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data. We propose a method called Stratified Prediction-Powered Inference (StratPPI) We show that the basic PPI estimates can be considerably improved by employing simple data stratification strategies.
  arXiv  Detail & Related papers  (2024-06-06T17:37:39Z)
• Fine-Tuning Language Models with Reward Learning on Policy [68.70065254564642]
  Reinforcement learning from human feedback (RLHF) has emerged as an effective approach to aligning large language models (LLMs) to human preferences. Despite its popularity, (fixed) reward models may suffer from inaccurate off-distribution. We propose reward learning on policy (RLP), an unsupervised framework that refines a reward model using policy samples to keep it on-distribution.
  arXiv  Detail & Related papers  (2024-03-28T10:02:10Z)
• Fair Streaming Principal Component Analysis: Statistical and Algorithmic Viewpoint [38.86637435197192]
  We present a new theoretical and practical approach to fair Principal Component Analysis (PCA) using a new notion called emphprobably approximately fair and optimal (PAFO) learnability. We then provide its it statistical guarantee in terms of PAFO-learnability, which is the first of its kind in fair PCA literature. Lastly, we verify the efficacy and memory efficiency of our algorithm on real-world datasets.
  arXiv  Detail & Related papers  (2023-10-28T05:09:30Z)
• Fair principal component analysis (PCA): minorization-maximization algorithms for Fair PCA, Fair Robust PCA and Fair Sparse PCA [6.974999794070285]
  We propose a new iterative algorithm to solve the fair PCA (FPCA) problem. The proposed algorithm relies on the relaxation of a semi-orthogonality constraint which is proved to be tight at every iteration of the algorithm. We numerically compare the performance of the proposed methods with two of the state-of-the-art approaches on synthetic data sets and a real-life data set.
  arXiv  Detail & Related papers  (2023-05-10T08:14:32Z)
• An online algorithm for contrastive Principal Component Analysis [9.090031210111919]
  We derive an online algorithm for cPCA* and show that it maps onto a neural network with local learning rules, so it can potentially be implemented in energy efficient neuromorphic hardware. We evaluate the performance of our online algorithm on real datasets and highlight the differences and similarities with the original formulation.
  arXiv  Detail & Related papers  (2022-11-14T19:48:48Z)
• Asymptotically Unbiased Instance-wise Regularized Partial AUC Optimization: Theory and Algorithm [101.44676036551537]
  One-way Partial AUC (OPAUC) and Two-way Partial AUC (TPAUC) measures the average performance of a binary classifier. Most of the existing methods could only optimize PAUC approximately, leading to inevitable biases that are not controllable. We present a simpler reformulation of the PAUC problem via distributional robust optimization AUC.
  arXiv  Detail & Related papers  (2022-10-08T08:26:22Z)
• A novel approach for Fair Principal Component Analysis based on eigendecomposition [10.203602318836444]
  We propose a novel PCA algorithm which tackles fairness issues by means of a simple strategy comprising a one-dimensional search. Our findings are consistent in several real situations as well as in scenarios with both unbalanced and balanced datasets.
  arXiv  Detail & Related papers  (2022-08-24T08:20:16Z)
• You May Not Need Ratio Clipping in PPO [117.03368180633463]
  Proximal Policy Optimization (PPO) methods learn a policy by iteratively performing multiple mini-batch optimization epochs of a surrogate objective with one set of sampled data. Ratio clipping PPO is a popular variant that clips the probability ratios between the target policy and the policy used to collect samples. We show in this paper that such ratio clipping may not be a good option as it can fail to effectively bound the ratios. We show that ESPO can be easily scaled up to distributed training with many workers, delivering strong performance as well.
  arXiv  Detail & Related papers  (2022-01-31T20:26:56Z)
• AgFlow: Fast Model Selection of Penalized PCA via Implicit Regularization Effects of Gradient Flow [64.81110234990888]
  Principal component analysis (PCA) has been widely used as an effective technique for feature extraction and dimension reduction. In the High Dimension Low Sample Size (HDLSS) setting, one may prefer modified principal components, with penalized loadings. We propose Approximated Gradient Flow (AgFlow) as a fast model selection method for penalized PCA.
  arXiv  Detail & Related papers  (2021-10-07T08:57:46Z)
• FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysis [12.91948651812873]
  Principal Component Analysis (PCA) is a fundamental data preprocessing tool in the world of machine learning. This paper proposes a distributed PCA algorithm called FAST-PCA (Fast and exAct diSTributed PCA)
  arXiv  Detail & Related papers  (2021-08-27T16:10:59Z)
• Approximation Algorithms for Sparse Principal Component Analysis [57.5357874512594]
  Principal component analysis (PCA) is a widely used dimension reduction technique in machine learning and statistics. Various approaches to obtain sparse principal direction loadings have been proposed, which are termed Sparse Principal Component Analysis. We present thresholding as a provably accurate, time, approximation algorithm for the SPCA problem.
  arXiv  Detail & Related papers  (2020-06-23T04:25:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.