Large Language Model Bias Mitigation from the Perspective of Knowledge Editing

• URL: http://arxiv.org/abs/2405.09341v2
• Date: Sat, 29 Jun 2024 05:50:28 GMT
• Title: Large Language Model Bias Mitigation from the Perspective of Knowledge Editing
• Authors: Ruizhe Chen, Yichen Li, Zikai Xiao, Zuozhu Liu,
• Abstract summary: We propose a novel debiasing method, Fairness Stamp (FAST), which enables editable fairness through fine-grained calibration on individual biased knowledge. FAST surpasses state-of-the-art baselines with remarkable debiasing performance while not hampering overall model capability for knowledge preservation.
• Score: 12.855975783223236
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing debiasing methods inevitably make unreasonable or undesired predictions as they are designated and evaluated to achieve parity across different social groups but leave aside individual facts, resulting in modified existing knowledge. In this paper, we first establish a new bias mitigation benchmark BiasKE leveraging existing and additional constructed datasets, which systematically assesses debiasing performance by complementary metrics on fairness, specificity, and generalization. Meanwhile, we propose a novel debiasing method, Fairness Stamp (FAST), which enables editable fairness through fine-grained calibration on individual biased knowledge. Comprehensive experiments demonstrate that FAST surpasses state-of-the-art baselines with remarkable debiasing performance while not hampering overall model capability for knowledge preservation, highlighting the prospect of fine-grained debiasing strategies for editable fairness in LLMs.

Related papers

• Editable Fairness: Fine-Grained Bias Mitigation in Language Models [52.66450426729818]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulness [10.081447621656523]
  The impact on language modeling ability can be alleviated given a high-quality and long-contextualized debiasing corpus. The effectiveness of task-agnostic debiasing hinges on the quantitative bias level of both the task-specific data used for downstream applications and the debiased model. We propose a novel framework which can Propagate Socially-fair Debiasing to Downstream Fine-tuning, ProSocialTuning.
  arXiv  Detail & Related papers  (2024-06-06T15:11:11Z)
• Causality and Independence Enhancement for Biased Node Classification [56.38828085943763]
  We propose a novel Causality and Independence Enhancement (CIE) framework, applicable to various graph neural networks (GNNs) Our approach estimates causal and spurious features at the node representation level and mitigates the influence of spurious correlations. Our approach CIE not only significantly enhances the performance of GNNs but outperforms state-of-the-art debiased node classification methods.
  arXiv  Detail & Related papers  (2023-10-14T13:56:24Z)
• Balancing Unobserved Confounding with a Few Unbiased Ratings in Debiased Recommendations [4.960902915238239]
  We propose a theoretically guaranteed model-agnostic balancing approach that can be applied to any existing debiasing method. The proposed approach makes full use of unbiased data by alternatively correcting model parameters learned with biased data, and adaptively learning balance coefficients of biased samples for further debiasing.
  arXiv  Detail & Related papers  (2023-04-17T08:56:55Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Fairness and Explainability: Bridging the Gap Towards Fair Model Explanations [12.248793742165278]
  We bridge the gap between fairness and explainability by presenting a novel perspective of procedure-oriented fairness based on explanations. We propose a Comprehensive Fairness Algorithm (CFA), which simultaneously fulfills multiple objectives - improving traditional fairness, satisfying explanation fairness, and maintaining the utility performance.
  arXiv  Detail & Related papers  (2022-12-07T18:35:54Z)
• Information-Theoretic Bias Reduction via Causal View of Spurious Correlation [71.9123886505321]
  We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation. We present a novel debiasing framework against the algorithmic bias, which incorporates a bias regularization loss. The proposed bias measurement and debiasing approaches are validated in diverse realistic scenarios.
  arXiv  Detail & Related papers  (2022-01-10T01:19:31Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)
• Uncertainty Calibration for Ensemble-Based Debiasing Methods [27.800387167841972]
  In this paper, we focus on the bias-only model in ensemble-based debiasing methods. We show that the debiasing performance can be damaged by inaccurate uncertainty estimations of the bias-only model. Motivated by these findings, we propose to conduct calibration on the bias-only model, thus achieving a three-stage ensemble-based debiasing framework.
  arXiv  Detail & Related papers  (2021-11-07T15:13:32Z)
• Towards Debiasing NLU Models from Unknown Biases [70.31427277842239]
  NLU models often exploit biases to achieve high dataset-specific performance without properly learning the intended task. We present a self-debiasing framework that prevents models from mainly utilizing biases without knowing them in advance.
  arXiv  Detail & Related papers  (2020-09-25T15:49:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.