American == White in Multimodal Language-and-Image AI

• URL: http://arxiv.org/abs/2207.00691v1
• Date: Fri, 1 Jul 2022 23:45:56 GMT
• Title: American == White in Multimodal Language-and-Image AI
• Authors: Robert Wolfe and Aylin Caliskan
• Abstract summary: Three state-of-the-art language-and-image AI models are evaluated. We show that White individuals are more associated with collective in-group words than are Asian, Black, or Latina/o individuals. The results indicate that biases equating American identity with being White are learned by language-and-image AI.
• Score: 3.4157048274143316
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Three state-of-the-art language-and-image AI models, CLIP, SLIP, and BLIP, are evaluated for evidence of a bias previously observed in social and experimental psychology: equating American identity with being White. Embedding association tests (EATs) using standardized images of self-identified Asian, Black, Latina/o, and White individuals from the Chicago Face Database (CFD) reveal that White individuals are more associated with collective in-group words than are Asian, Black, or Latina/o individuals. In assessments of three core aspects of American identity reported by social psychologists, single-category EATs reveal that images of White individuals are more associated with patriotism and with being born in America, but that, consistent with prior findings in psychology, White individuals are associated with being less likely to treat people of all races and backgrounds equally. Three downstream machine learning tasks demonstrate biases associating American with White. In a visual question answering task using BLIP, 97% of White individuals are identified as American, compared to only 3% of Asian individuals. When asked in what state the individual depicted lives in, the model responds China 53% of the time for Asian individuals, but always with an American state for White individuals. In an image captioning task, BLIP remarks upon the race of Asian individuals as much as 36% of the time, but never remarks upon race for White individuals. Finally, provided with an initialization image from the CFD and the text "an American person," a synthetic image generator (VQGAN) using the text-based guidance of CLIP lightens the skin tone of individuals of all races (by 35% for Black individuals, based on pixel brightness). The results indicate that biases equating American identity with being White are learned by language-and-image AI, and propagate to downstream applications of such models.

Related papers

• AI-generated faces influence gender stereotypes and racial homogenization [1.6647208383676708]
  We document significant biases in Stable Diffusion across six races, two genders, 32 professions, and eight attributes. This analysis reveals significant racial homogenization depicting nearly all Middle Eastern men as bearded, brown-skinned, and wearing traditional attire. We propose debiasing solutions that allow users to specify the desired distributions of race and gender when generating images.
  arXiv  Detail & Related papers  (2024-02-01T20:32:14Z)
• Comparing Biases and the Impact of Multilingual Training across Multiple Languages [70.84047257764405]
  We present a bias analysis across Italian, Chinese, English, Hebrew, and Spanish on the downstream sentiment analysis task. We adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for four attributes: race, religion, nationality, and gender. Our results reveal similarities in bias expression such as favoritism of groups that are dominant in each language's culture.
  arXiv  Detail & Related papers  (2023-05-18T18:15:07Z)
• Do All Asians Look the Same?: A Comparative Analysis of the East Asian Facial Color Desires using Instagram [7.927093463287226]
  This study uses selfie data to examine how peoples' desires for ideal facial representations vary by region. We refute the "all Asians prefer identical visuals," which is a subset of the prevalent Western belief that "all Asians look the same" We propose a strategy for resolving the mismatch between real-world desires and the Western beauty market's views.
  arXiv  Detail & Related papers  (2023-04-06T15:05:56Z)
• Studying Bias in GANs through the Lens of Race [91.95264864405493]
  We study how the performance and evaluation of generative image models are impacted by the racial composition of their training datasets. Our results show that the racial compositions of generated images successfully preserve that of the training data. However, we observe that truncation, a technique used to generate higher quality images during inference, exacerbates racial imbalances in the data.
  arXiv  Detail & Related papers  (2022-09-06T22:25:56Z)
• Markedness in Visual Semantic AI [3.4157048274143316]
  We evaluate the state-of-the-art multimodal "visual semantic" model CLIP for biases related to the marking of age, gender, and race or ethnicity. Female individuals under the age of 20 are more likely than Male individuals to be marked with a gender label, but less likely to be marked with an age label. As age increases, the self-similarity of representations of Female individuals increases at a higher rate than for Male individuals.
  arXiv  Detail & Related papers  (2022-05-23T15:14:41Z)
• Evidence for Hypodescent in Visual Semantic AI [4.1804353242318255]
  Multiracial people are more likely to be assigned a racial or ethnic label corresponding to a minority or disadvantaged racial or ethnic group than to the equivalent majority or advantaged group. A face morphing experiment grounded in psychological research demonstrates hypodescent. We show that the stereotype-congruent pleasantness association of an image correlates with association with the Black text label in CLIP.
  arXiv  Detail & Related papers  (2022-05-22T06:46:39Z)
• Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection [75.54119209776894]
  We investigate the effect of annotator identities (who) and beliefs (why) on toxic language annotations. We consider posts with three characteristics: anti-Black language, African American English dialect, and vulgarity. Our results show strong associations between annotator identity and beliefs and their ratings of toxicity.
  arXiv  Detail & Related papers  (2021-11-15T18:58:20Z)
• Black or White but never neutral: How readers perceive identity from yellow or skin-toned emoji [90.14874935843544]
  Recent work established a connection between expression of identity and emoji usage on social media. This work asks if, as with language, readers are sensitive to such acts of self-expression and use them to understand the identity of authors.
  arXiv  Detail & Related papers  (2021-05-12T18:23:51Z)
• One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision [75.82110684355979]
  We study the racial system encoded by computer vision datasets supplying categorical race labels for face images. We find that each dataset encodes a substantially unique racial system, despite nominally equivalent racial categories. We find evidence that racial categories encode stereotypes, and exclude ethnic groups from categories on the basis of nonconformity to stereotypes.
  arXiv  Detail & Related papers  (2021-02-03T22:50:04Z)
• Country Image in COVID-19 Pandemic: A Case Study of China [79.17323278601869]
  Country image has a profound influence on international relations and economic development. In the worldwide outbreak of COVID-19, countries and their people display different reactions. In this study, we take China as a specific and typical case and investigate its image with aspect-based sentiment analysis on a large-scale Twitter dataset.
  arXiv  Detail & Related papers  (2020-09-12T15:54:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.