Debiasing Word Embeddings with Nonlinear Geometry

• URL: http://arxiv.org/abs/2208.13899v1
• Date: Mon, 29 Aug 2022 21:40:27 GMT
• Title: Debiasing Word Embeddings with Nonlinear Geometry
• Authors: Lu Cheng, Nayoung Kim, Huan Liu
• Abstract summary: This work studies biases associated with multiple social categories. Individual biases intersect non-trivially over a one-dimensional subspace. We then construct an intersectional subspace to debias for multiple social categories using the nonlinear geometry of individual biases.
• Score: 37.88933175338274
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Debiasing word embeddings has been largely limited to individual and independent social categories. However, real-world corpora typically present multiple social categories that possibly correlate or intersect with each other. For instance, "hair weaves" is stereotypically associated with African American females, but neither African American nor females alone. Therefore, this work studies biases associated with multiple social categories: joint biases induced by the union of different categories and intersectional biases that do not overlap with the biases of the constituent categories. We first empirically observe that individual biases intersect non-trivially (i.e., over a one-dimensional subspace). Drawing from the intersectional theory in social science and the linguistic theory, we then construct an intersectional subspace to debias for multiple social categories using the nonlinear geometry of individual biases. Empirical evaluations corroborate the efficacy of our approach. Data and implementation code can be downloaded at https://github.com/GitHubLuCheng/Implementation-of-JoSEC-COLING-22.

Related papers

• Social-Group-Agnostic Word Embedding Debiasing via the Stereotype Content Model [3.0869883531083233]
  Existing word embedding debiasing methods require social-group-specific word pairs for each social attribute. We propose that the Stereotype Content Model (SCM) can help debiasing efforts to become social-group-agnostic.
  arXiv  Detail & Related papers  (2022-10-11T23:26:23Z)
• The SAME score: Improved cosine based bias score for word embeddings [49.75878234192369]
  We introduce SAME, a novel bias score for semantic bias in embeddings. We show that SAME is capable of measuring semantic bias and identify potential causes for social bias in downstream tasks.
  arXiv  Detail & Related papers  (2022-03-28T09:28:13Z)
• WordBias: An Interactive Visual Tool for Discovering Intersectional Biases Encoded in Word Embeddings [39.87681037622605]
  We present WordBias, an interactive visual tool designed to explore biases against intersectional groups encoded in word embeddings. Given a pretrained static word embedding, WordBias computes the association of each word along different groups based on race, age, etc.
  arXiv  Detail & Related papers  (2021-03-05T11:04:35Z)
• How True is GPT-2? An Empirical Analysis of Intersectional Occupational Biases [50.591267188664666]
  Downstream applications are at risk of inheriting biases contained in natural language models. We analyze the occupational biases of a popular generative language model, GPT-2. For a given job, GPT-2 reflects the societal skew of gender and ethnicity in the US, and in some cases, pulls the distribution towards gender parity.
  arXiv  Detail & Related papers  (2021-02-08T11:10:27Z)
• One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision [75.82110684355979]
  We study the racial system encoded by computer vision datasets supplying categorical race labels for face images. We find that each dataset encodes a substantially unique racial system, despite nominally equivalent racial categories. We find evidence that racial categories encode stereotypes, and exclude ethnic groups from categories on the basis of nonconformity to stereotypes.
  arXiv  Detail & Related papers  (2021-02-03T22:50:04Z)
• Unequal Representations: Analyzing Intersectional Biases in Word Embeddings Using Representational Similarity Analysis [3.8580784887142774]
  We probe contextualized and non-contextualized embeddings for evidence of intersectional biases against Black women. We show that these embeddings represent Black women as simultaneously less feminine than White women, and less Black than Black men.
  arXiv  Detail & Related papers  (2020-11-24T13:45:14Z)
• Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation [57.292988892028134]
  Bolukbasi et al. present one of the first gender bias mitigation techniques for word representations. We generalize their method to a kernelized, nonlinear version. We analyze empirically whether the bias subspace is actually linear.
  arXiv  Detail & Related papers  (2020-09-20T14:13:45Z)
• Discovering and Categorising Language Biases in Reddit [5.670038395203354]
  This paper proposes a data-driven approach to automatically discover language biases encoded in the vocabulary of online discourse communities on Reddit. We use word embeddings to transform text into high-dimensional dense vectors and capture semantic relations between words. We successfully discover gender bias, religion bias, and ethnic bias in different Reddit communities.
  arXiv  Detail & Related papers  (2020-08-06T16:42:10Z)
• Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases [10.713568409205077]
  State-of-the-art neural language models generate dynamic word embeddings dependent on the context in which the word appears. We introduce the Contextualized Embedding Association Test (CEAT), that can summarize the magnitude of overall bias in neural language models. We develop two methods, Intersectional Bias Detection (IBD) and Emergent Intersectional Bias Detection (EIBD), to automatically identify the intersectional biases and emergent intersectional biases from static word embeddings.
  arXiv  Detail & Related papers  (2020-06-06T19:49:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.