FairMonitor: A Four-Stage Automatic Framework for Detecting Stereotypes and Biases in Large Language Models

• URL: http://arxiv.org/abs/2308.10397v2
• Date: Fri, 27 Oct 2023 01:54:26 GMT
• Title: FairMonitor: A Four-Stage Automatic Framework for Detecting Stereotypes and Biases in Large Language Models
• Authors: Yanhong Bai and Jiabao Zhao and Jinxin Shi and Tingjiang Wei and Xingjiao Wu and Liang He
• Abstract summary: This paper introduces a four-stage framework to directly evaluate stereotypes and biases in the generated content of Large Language Models (LLMs) Using the education sector as a case study, we constructed the Edu-FairMonitor based on the four-stage framework. Experimental results reveal varying degrees of stereotypes and biases in five LLMs evaluated on Edu-FairMonitor.
• Score: 10.57405233305553
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting stereotypes and biases in Large Language Models (LLMs) can enhance fairness and reduce adverse impacts on individuals or groups when these LLMs are applied. However, the majority of existing methods focus on measuring the model's preference towards sentences containing biases and stereotypes within datasets, which lacks interpretability and cannot detect implicit biases and stereotypes in the real world. To address this gap, this paper introduces a four-stage framework to directly evaluate stereotypes and biases in the generated content of LLMs, including direct inquiry testing, serial or adapted story testing, implicit association testing, and unknown situation testing. Additionally, the paper proposes multi-dimensional evaluation metrics and explainable zero-shot prompts for automated evaluation. Using the education sector as a case study, we constructed the Edu-FairMonitor based on the four-stage framework, which encompasses 12,632 open-ended questions covering nine sensitive factors and 26 educational scenarios. Experimental results reveal varying degrees of stereotypes and biases in five LLMs evaluated on Edu-FairMonitor. Moreover, the results of our proposed automated evaluation method have shown a high correlation with human annotations.

Related papers

• HEARTS: A Holistic Framework for Explainable, Sustainable and Robust Text Stereotype Detection [0.0]
  We introduce HEARTS (Holistic Framework for Explainable, Sustainable, and Robust Text Stereotype Detection), a framework that enhances model performance, minimises carbon footprint, and provides transparent, interpretable explanations. We establish the Expanded Multi-Grain Stereotype dataset (EMGSD), comprising 57,201 labeled texts across six groups, including under-represented demographics like LGBTQ+ and regional stereotypes. We then analyse a fine-tuned, carbon-efficient ALBERT-V2 model using SHAP to generate token-level importance values, ensuring alignment with human understanding, and calculate explainability confidence scores by comparing SHAP and LIME
  arXiv  Detail & Related papers  (2024-09-17T22:06:46Z)
• Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models [47.545382591646565]
  Large Language Models (LLMs) have excelled at language understanding and generating human-level text. LLMs are susceptible to adversarial attacks where malicious users prompt the model to generate undesirable text. In this work, we train models to automatically create adversarial prompts to elicit biased responses from target LLMs.
  arXiv  Detail & Related papers  (2024-08-07T17:11:34Z)
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Behaviors [64.9938658716425]
  Existing evaluations of large language models' (LLMs) ability to recognize and reject unsafe user requests face three limitations. First, existing methods often use coarse-grained of unsafe topics, and are over-representing some fine-grained topics. Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations. Third, existing evaluations rely on large LLMs for evaluation, which can be expensive.
  arXiv  Detail & Related papers  (2024-06-20T17:56:07Z)
• Investigating Annotator Bias in Large Language Models for Hate Speech Detection [5.589665886212444]
  This paper delves into the biases present in Large Language Models (LLMs) when annotating hate speech data. Specifically targeting highly vulnerable groups within these categories, we analyze annotator biases. We introduce our custom hate speech detection dataset, HateBiasNet, to conduct this research.
  arXiv  Detail & Related papers  (2024-06-17T00:18:31Z)
• FairMonitor: A Dual-framework for Detecting Stereotypes and Biases in Large Language Models [9.385390205833893]
  We propose the FairMonitor framework and adopt a static-dynamic detection method for a comprehensive evaluation of stereotypes and biases in Large Language Models (LLMs) The static component consists of a direct inquiry test, an implicit association test, and an unknown situation test, including 10,262 open-ended questions with 9 sensitive factors and 26 educational scenarios. We utilize the multi-agent system to construst the dynamic scenarios for detecting subtle biases in more complex and realistic setting.
  arXiv  Detail & Related papers  (2024-05-06T01:23:07Z)
• Zero-shot Generative Large Language Models for Systematic Review Screening Automation [55.403958106416574]
  This study investigates the effectiveness of using zero-shot large language models for automatic screening. We evaluate the effectiveness of eight different LLMs and investigate a calibration technique that uses a predefined recall threshold.
  arXiv  Detail & Related papers  (2024-01-12T01:54:08Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Models [12.214260053244871]
  We analyse the body of work that uses prompts and templates to assess bias in language models. We draw on a measurement modelling framework to create a taxonomy of attributes that capture what a bias test aims to measure. Our analysis illuminates the scope of possible bias types the field is able to measure, and reveals types that are as yet under-researched.
  arXiv  Detail & Related papers  (2023-05-22T06:28:48Z)
• LOGAN: Local Group Bias Detection by Clustering [86.38331353310114]
  We argue that evaluating bias at the corpus level is not enough for understanding how biases are embedded in a model. We propose LOGAN, a new bias detection technique based on clustering. Experiments on toxicity classification and object classification tasks show that LOGAN identifies bias in a local region.
  arXiv  Detail & Related papers  (2020-10-06T16:42:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.