Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approach
- URL: http://arxiv.org/abs/2407.14779v2
- Date: Wed, 24 Jul 2024 17:25:26 GMT
- Title: Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approach
- Authors: Sourojit Ghosh, Pranav Narayanan Venkit, Sanjana Gautam, Shomir Wilson, Aylin Caliskan,
- Abstract summary: This research investigates the impact of Generative Artificial Intelligence (GAI) models, specifically text-to-image generators (T2Is), on the representation of non-Western cultures.
- Score: 8.805524738976073
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Our research investigates the impact of Generative Artificial Intelligence (GAI) models, specifically text-to-image generators (T2Is), on the representation of non-Western cultures, with a focus on Indian contexts. Despite the transformative potential of T2Is in content creation, concerns have arisen regarding biases that may lead to misrepresentations and marginalizations. Through a community-centered approach and grounded theory analysis of 5 focus groups from diverse Indian subcultures, we explore how T2I outputs to English prompts depict Indian culture and its subcultures, uncovering novel representational harms such as exoticism and cultural misappropriation. These findings highlight the urgent need for inclusive and culturally sensitive T2I systems. We propose design guidelines informed by a sociotechnical perspective, aiming to address these issues and contribute to the development of more equitable and representative GAI technologies globally. Our work also underscores the necessity of adopting a community-centered approach to comprehend the sociotechnical dynamics of these models, complementing existing work in this space while identifying and addressing the potential negative repercussions and harms that may arise when these models are deployed on a global scale.
Related papers
- Beyond Aesthetics: Cultural Competence in Text-to-Image Models [34.98692829036475]
CUBE is a first-of-its-kind benchmark to evaluate cultural competence of Text-to-Image models.
CUBE covers cultural artifacts associated with 8 countries across different geo-cultural regions.
CUBE-CSpace is a larger dataset of cultural artifacts that serves as grounding to evaluate cultural diversity.
arXiv Detail & Related papers (2024-07-09T13:50:43Z) - Extrinsic Evaluation of Cultural Competence in Large Language Models [53.626808086522985]
We focus on extrinsic evaluation of cultural competence in two text generation tasks.
We evaluate model outputs when an explicit cue of culture, specifically nationality, is perturbed in the prompts.
We find weak correlations between text similarity of outputs for different countries and the cultural values of these countries.
arXiv Detail & Related papers (2024-06-17T14:03:27Z) - T-HITL Effectively Addresses Problematic Associations in Image
Generation and Maintains Overall Visual Quality [52.5529784801908]
We focus on addressing the generation of problematic associations between demographic groups and semantic concepts.
We propose a new methodology with twice-human-in-the-loop (T-HITL) that promises improvements in both reducing problematic associations and also maintaining visual quality.
arXiv Detail & Related papers (2024-02-27T00:29:33Z) - Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking [48.21982147529661]
This paper introduces a novel approach for massively multicultural knowledge acquisition.
Our method strategically navigates from densely informative Wikipedia documents on cultural topics to an extensive network of linked pages.
Our work marks an important step towards deeper understanding and bridging the gaps of cultural disparities in AI.
arXiv Detail & Related papers (2024-02-14T18:16:54Z) - Language Models: A Guide for the Perplexed [51.88841610098437]
This tutorial aims to help narrow the gap between those who study language models and those who are intrigued and want to learn more.
We offer a scientific viewpoint that focuses on questions amenable to study through experimentation.
We situate language models as they are today in the context of the research that led to their development.
arXiv Detail & Related papers (2023-11-29T01:19:02Z) - Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in
Large Language Models [89.94270049334479]
This paper identifies a cultural dominance issue within large language models (LLMs)
LLMs often provide inappropriate English-culture-related answers that are not relevant to the expected culture when users ask in non-English languages.
arXiv Detail & Related papers (2023-10-19T05:38:23Z) - Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of
Text-To-Image Models [36.04866429768613]
We explore the cultural perception embedded in Text-To-Image models by characterizing culture across three tiers: cultural dimensions, cultural domains, and cultural concepts.
We propose a comprehensive suite of evaluation techniques, including intrinsic evaluations using the CLIP space, extrinsic evaluations with a Visual-Question-Answer (VQA) model and human assessments.
Our experiments provide insights regarding Do, What, Which and How research questions about the nature of cultural encoding in TTI models, paving the way for cross-cultural applications.
arXiv Detail & Related papers (2023-10-03T10:13:36Z) - On the Cultural Gap in Text-to-Image Generation [75.69755281031951]
One challenge in text-to-image (T2I) generation is the inadvertent reflection of culture gaps present in the training data.
There is no benchmark to systematically evaluate a T2I model's ability to generate cross-cultural images.
We propose a Challenging Cross-Cultural (C3) benchmark with comprehensive evaluation criteria, which can assess how well-suited a model is to a target culture.
arXiv Detail & Related papers (2023-07-06T13:17:55Z) - AI's Regimes of Representation: A Community-centered Study of
Text-to-Image Models in South Asia [18.308417975842058]
We show how generative AI can reproduce an outsiders gaze for viewing South Asian cultures, shaped by global and regional power inequities.
We distill lessons for responsible development of T2I models, recommending concrete pathways forward.
arXiv Detail & Related papers (2023-05-19T17:35:20Z) - An Analytics of Culture: Modeling Subjectivity, Scalability,
Contextuality, and Temporality [13.638494941763637]
There is a bidirectional relationship between culture and AI; AI models are increasingly used to analyse culture, thereby shaping our understanding of culture.
On the other hand, the models are trained on collections of cultural artifacts thereby implicitly, and not always correctly, encoding expressions of culture.
This creates a tension that both limits the use of AI for analysing culture and leads to problems in AI with respect to cultural complex issues such as bias.
arXiv Detail & Related papers (2022-11-14T15:42:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.