Fairness Reprogramming
- URL: http://arxiv.org/abs/2209.10222v4
- Date: Mon, 12 Dec 2022 09:52:59 GMT
- Title: Fairness Reprogramming
- Authors: Guanhua Zhang, Yihua Zhang, Yang Zhang, Wenqi Fan, Qing Li, Sijia Liu,
Shiyu Chang
- Abstract summary: We propose a new generic fairness learning paradigm, called FairReprogram, which incorporates the model reprogramming technique.
Specifically, FairReprogram considers the case where models can not be changed and appends to the input a set of perturbations, called the fairness trigger.
We show both theoretically and empirically that the fairness trigger can effectively obscure demographic biases in the output prediction of fixed ML models.
- Score: 42.65700878967251
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Despite a surge of recent advances in promoting machine Learning (ML)
fairness, the existing mainstream approaches mostly require retraining or
finetuning the entire weights of the neural network to meet the fairness
criteria. However, this is often infeasible in practice for those large-scale
trained models due to large computational and storage costs, low data
efficiency, and model privacy issues. In this paper, we propose a new generic
fairness learning paradigm, called FairReprogram, which incorporates the model
reprogramming technique. Specifically, FairReprogram considers the case where
models can not be changed and appends to the input a set of perturbations,
called the fairness trigger, which is tuned towards the fairness criteria under
a min-max formulation. We further introduce an information-theoretic framework
that explains why and under what conditions fairness goals can be achieved
using the fairness trigger. We show both theoretically and empirically that the
fairness trigger can effectively obscure demographic biases in the output
prediction of fixed ML models by providing false demographic information that
hinders the model from utilizing the correct demographic information to make
the prediction. Extensive experiments on both NLP and CV datasets demonstrate
that our method can achieve better fairness improvements than retraining-based
methods with far less data dependency under two widely-used fairness criteria.
Codes are available at
https://github.com/UCSB-NLP-Chang/Fairness-Reprogramming.git.
Related papers
- Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium [0.3350491650545292]
Current methods for mitigating bias often result in information loss and an inadequate balance between accuracy and fairness.
We propose a novel methodology grounded in bilevel optimization principles.
Our deep learning-based approach concurrently optimize for both accuracy and fairness objectives.
arXiv Detail & Related papers (2024-10-21T18:53:39Z) - Editable Fairness: Fine-Grained Bias Mitigation in Language Models [52.66450426729818]
We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.
FAST surpasses state-of-the-art baselines with superior debiasing performance.
This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
arXiv Detail & Related papers (2024-08-07T17:14:58Z) - Improving Fairness and Mitigating MADness in Generative Models [21.024727486615646]
We show that training generative models with intentionally designed hypernetworks leads to models that are more fair when generating datapoints belonging to minority classes.
We introduce a regularization term that penalizes discrepancies between a generative model's estimated weights when trained on real data versus its own synthetic data.
arXiv Detail & Related papers (2024-05-22T20:24:41Z) - Fast Model Debias with Machine Unlearning [54.32026474971696]
Deep neural networks might behave in a biased manner in many real-world scenarios.
Existing debiasing methods suffer from high costs in bias labeling or model re-training.
We propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases.
arXiv Detail & Related papers (2023-10-19T08:10:57Z) - Learning for Counterfactual Fairness from Observational Data [62.43249746968616]
Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age.
A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data.
In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE.
arXiv Detail & Related papers (2023-07-17T04:08:29Z) - Non-Invasive Fairness in Learning through the Lens of Data Drift [88.37640805363317]
We show how to improve the fairness of Machine Learning models without altering the data or the learning algorithm.
We use a simple but key insight: the divergence of trends between different populations, and, consecutively, between a learned model and minority populations, is analogous to data drift.
We explore two strategies (model-splitting and reweighing) to resolve this drift, aiming to improve the overall conformance of models to the underlying data.
arXiv Detail & Related papers (2023-03-30T17:30:42Z) - FairAdaBN: Mitigating unfairness with adaptive batch normalization and
its application to dermatological disease classification [14.589159162086926]
We propose FairAdaBN, which makes batch normalization adaptive to sensitive attribute.
We propose a new metric, named Fairness-Accuracy Trade-off Efficiency (FATE), to compute normalized fairness improvement over accuracy drop.
Experiments on two dermatological datasets show that our proposed method outperforms other methods on fairness criteria and FATE.
arXiv Detail & Related papers (2023-03-15T02:22:07Z) - Fairness and Accuracy under Domain Generalization [10.661409428935494]
Concerns have arisen that machine learning algorithms may be biased against certain social groups.
Many approaches have been proposed to make ML models fair, but they typically rely on the assumption that data distributions in training and deployment are identical.
We study the transfer of both fairness and accuracy under domain generalization where the data at test time may be sampled from never-before-seen domains.
arXiv Detail & Related papers (2023-01-30T23:10:17Z) - FairIF: Boosting Fairness in Deep Learning via Influence Functions with
Validation Set Sensitive Attributes [51.02407217197623]
We propose a two-stage training algorithm named FAIRIF.
It minimizes the loss over the reweighted data set where the sample weights are computed.
We show that FAIRIF yields models with better fairness-utility trade-offs against various types of bias.
arXiv Detail & Related papers (2022-01-15T05:14:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.