CORGI-PM: A Chinese Corpus For Gender Bias Probing and Mitigation

• URL: http://arxiv.org/abs/2301.00395v1
• Date: Sun, 1 Jan 2023 12:48:12 GMT
• Title: CORGI-PM: A Chinese Corpus For Gender Bias Probing and Mitigation
• Authors: Ge Zhang, Yizhi Li, Yaoyao Wu, Linyuan Zhang, Chenghua Lin, Jiayi Geng, Shi Wang, Jie Fu
• Abstract summary: We propose a Chinese cOrpus foR Gender bIas Probing and Mitigation CORGI-PM, which contains 32.9k sentences with high-quality labels. We address three challenges for automatic textual gender bias mitigation, which requires the models to detect, classify, and mitigate textual gender bias. CORGI-PM is the first sentence-level Chinese corpus for gender bias probing and mitigation.
• Score: 28.38578407487603
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As natural language processing (NLP) for gender bias becomes a significant interdisciplinary topic, the prevalent data-driven techniques such as large-scale language models suffer from data inadequacy and biased corpus, especially for languages with insufficient resources such as Chinese. To this end, we propose a Chinese cOrpus foR Gender bIas Probing and Mitigation CORGI-PM, which contains 32.9k sentences with high-quality labels derived by following an annotation scheme specifically developed for gender bias in the Chinese context. Moreover, we address three challenges for automatic textual gender bias mitigation, which requires the models to detect, classify, and mitigate textual gender bias. We also conduct experiments with state-of-the-art language models to provide baselines. To our best knowledge, CORGI-PM is the first sentence-level Chinese corpus for gender bias probing and mitigation.

Related papers

• Beyond Binary Gender: Evaluating Gender-Inclusive Machine Translation with Ambiguous Attitude Words [85.48043537327258]
  Existing machine translation gender bias evaluations are primarily focused on male and female genders. This study presents a benchmark AmbGIMT (Gender-Inclusive Machine Translation with Ambiguous attitude words) We propose a novel process to evaluate gender bias based on the Emotional Attitude Score (EAS), which is used to quantify ambiguous attitude words.
  arXiv  Detail & Related papers  (2024-07-23T08:13:51Z)
• Leveraging Large Language Models to Measure Gender Bias in Gendered Languages [9.959039325564744]
  This paper introduces a novel methodology that leverages the contextual understanding capabilities of large language models (LLMs) to quantitatively analyze gender representation in Spanish corpora. We empirically validate our method on four widely-used benchmark datasets, uncovering significant gender disparities with a male-to-female ratio ranging from 4:01.
  arXiv  Detail & Related papers  (2024-06-19T16:30:58Z)
• Gender Bias in Large Language Models across Multiple Languages [10.068466432117113]
  We examine gender bias in large language models (LLMs) generated for different languages. We use three measurements: 1) gender bias in selecting descriptive words given the gender-related context. 2) gender bias in selecting gender-related pronouns (she/he) given the descriptive words.
  arXiv  Detail & Related papers  (2024-03-01T04:47:16Z)
• Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You [64.74707085021858]
  We show that multilingual models suffer from significant gender biases just as monolingual models do. We propose a novel benchmark, MAGBIG, intended to foster research on gender bias in multilingual models. Our results show that not only do models exhibit strong gender biases but they also behave differently across languages.
  arXiv  Detail & Related papers  (2024-01-29T12:02:28Z)
• Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation [64.79319733514266]
  Large Language Models (LLMs) can generate biased and toxic responses. We propose a conditional text generation mechanism without the need for predefined gender phrases and stereotypes.
  arXiv  Detail & Related papers  (2023-11-01T05:31:46Z)
• On Evaluating and Mitigating Gender Biases in Multilingual Settings [5.248564173595024]
  We investigate some of the challenges with evaluating and mitigating biases in multilingual settings. We first create a benchmark for evaluating gender biases in pre-trained masked language models. We extend various debiasing methods to work beyond English and evaluate their effectiveness for SOTA massively multilingual models.
  arXiv  Detail & Related papers  (2023-07-04T06:23:04Z)
• Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translation [28.471506840241602]
  Gender bias is a significant issue in machine translation, leading to ongoing research efforts in developing bias mitigation techniques. We propose a bias mitigation method based on a novel approach. Gender-Aware Contrastive Learning, GACL, encodes contextual gender information into the representations of non-explicit gender words.
  arXiv  Detail & Related papers  (2023-05-23T12:53:39Z)
• COLD: A Benchmark for Chinese Offensive Language Detection [54.60909500459201]
  We use COLDataset, a Chinese offensive language dataset with 37k annotated sentences. We also propose textscCOLDetector to study output offensiveness of popular Chinese language models. Our resources and analyses are intended to help detoxify the Chinese online communities and evaluate the safety performance of generative language models.
  arXiv  Detail & Related papers  (2022-01-16T11:47:23Z)
• Evaluating Gender Bias in Natural Language Inference [5.034017602990175]
  We propose an evaluation methodology to measure gender bias in natural language understanding through inference. We use our challenge task to investigate state-of-the-art NLI models on the presence of gender stereotypes using occupations. Our findings suggest that three models trained on MNLI and SNLI datasets are significantly prone to gender-induced prediction errors.
  arXiv  Detail & Related papers  (2021-05-12T09:41:51Z)
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer [101.58431011820755]
  We study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications. We create a multilingual dataset for bias analysis and propose several ways for quantifying bias in multilingual representations.
  arXiv  Detail & Related papers  (2020-05-02T04:34:37Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.