Insights on Disagreement Patterns in Multimodal Safety Perception across Diverse Rater Groups

• URL: http://arxiv.org/abs/2410.17032v1
• Date: Tue, 22 Oct 2024 13:59:21 GMT
• Title: Insights on Disagreement Patterns in Multimodal Safety Perception across Diverse Rater Groups
• Authors: Charvi Rastogi, Tian Huey Teh, Pushkar Mishra, Roma Patel, Zoe Ashwood, Aida Mostafazadeh Davani, Mark Diaz, Michela Paganini, Alicia Parrish, Ding Wang, Vinodkumar Prabhakaran, Lora Aroyo, Verena Rieser,
• Abstract summary: AI systems crucially rely on human ratings, but these ratings are often aggregated. This is particularly concerning when evaluating the safety of generative AI, where perceptions and associated harms can vary significantly across socio-cultural contexts. We conduct a large-scale study employing highly-parallel safety ratings of about 1000 text-to-image (T2I) generations from a demographically diverse rater pool of 630 raters.
• Score: 29.720095331989064
• License:
• Abstract: AI systems crucially rely on human ratings, but these ratings are often aggregated, obscuring the inherent diversity of perspectives in real-world phenomenon. This is particularly concerning when evaluating the safety of generative AI, where perceptions and associated harms can vary significantly across socio-cultural contexts. While recent research has studied the impact of demographic differences on annotating text, there is limited understanding of how these subjective variations affect multimodal safety in generative AI. To address this, we conduct a large-scale study employing highly-parallel safety ratings of about 1000 text-to-image (T2I) generations from a demographically diverse rater pool of 630 raters balanced across 30 intersectional groups across age, gender, and ethnicity. Our study shows that (1) there are significant differences across demographic groups (including intersectional groups) on how severe they assess the harm to be, and that these differences vary across different types of safety violations, (2) the diverse rater pool captures annotation patterns that are substantially different from expert raters trained on specific set of safety policies, and (3) the differences we observe in T2I safety are distinct from previously documented group level differences in text-based safety tasks. To further understand these varying perspectives, we conduct a qualitative analysis of the open-ended explanations provided by raters. This analysis reveals core differences into the reasons why different groups perceive harms in T2I generations. Our findings underscore the critical need for incorporating diverse perspectives into safety evaluation of generative AI ensuring these systems are truly inclusive and reflect the values of all users.

Related papers

• The Factuality Tax of Diversity-Intervened Text-to-Image Generation: Benchmark and Fact-Augmented Intervention [61.80236015147771]
  We quantify the trade-off between using diversity interventions and preserving demographic factuality in T2I models. Experiments on DoFaiR reveal that diversity-oriented instructions increase the number of different gender and racial groups. We propose Fact-Augmented Intervention (FAI) to reflect on verbalized or retrieved factual information about gender and racial compositions of generation subjects in history.
  arXiv  Detail & Related papers  (2024-06-29T09:09:42Z)
• Diverse, but Divisive: LLMs Can Exaggerate Gender Differences in Opinion Related to Harms of Misinformation [8.066880413153187]
  This paper examines whether a large language model (LLM) can reflect the views of various groups when assessing the harms of misinformation. We present the TopicMisinfo dataset, containing 160 fact-checked claims from diverse topics. We find that GPT 3.5-Turbo reflects empirically observed gender differences in opinion but amplifies the extent of these differences.
  arXiv  Detail & Related papers  (2024-01-29T20:50:28Z)
• Exploring Public's Perception of Safety and Video Surveillance Technology: A Survey Approach [2.473948454680334]
  This study presents a comprehensive analysis of the community's general public safety concerns, their view of existing surveillance technologies, and their perception of AI-driven solutions for enhancing safety in urban environments, focusing on Charlotte, NC. This research investigates demographic factors such as age, gender, ethnicity, and educational level to gain insights into public perception and concerns toward public safety and possible solutions.
  arXiv  Detail & Related papers  (2023-12-10T15:53:37Z)
• GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives [18.574420136899978]
  We propose GRASP, a comprehensive disagreement analysis framework to measure group association in perspectives among different rater sub-groups. Our framework reveals specific rater groups that have significantly different perspectives than others on certain tasks, and helps identify demographic axes that are crucial to consider in specific task contexts.
  arXiv  Detail & Related papers  (2023-11-09T00:12:21Z)
• Stable Bias: Analyzing Societal Representations in Diffusion Models [72.27121528451528]
  We propose a new method for exploring the social biases in Text-to-Image (TTI) systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts. We leverage this method to analyze images generated by 3 popular TTI systems and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents.
  arXiv  Detail & Related papers  (2023-03-20T19:32:49Z)
• The Double-Edged Sword of Diversity: How Diversity, Conflict, and Psychological Safety Impact Software Teams [6.190511747986327]
  Team diversity can be seen as a double-edged sword, bringing cognitive resources to teams at the risk of increased conflict. This study views diversity through the lens of the categorization-elaboration model (CEM) We investigated how diversity in gender, age, role, and cultural background impacts team effectiveness and conflict.
  arXiv  Detail & Related papers  (2023-01-30T14:54:44Z)
• MultiFair: Multi-Group Fairness in Machine Learning [52.24956510371455]
  We study multi-group fairness in machine learning (MultiFair) We propose a generic end-to-end algorithmic framework to solve it. Our proposed framework is generalizable to many different settings.
  arXiv  Detail & Related papers  (2021-05-24T02:30:22Z)
• On the confidence of stereo matching in a deep-learning era: a quantitative evaluation [124.09613797008099]
  We review more than ten years of developments in the field of confidence estimation for stereo matching. We study the different behaviors of each measure when applied to a pool of different stereo algorithms and, for the first time in literature, when paired with a state-of-the-art deep stereo network.
  arXiv  Detail & Related papers  (2021-01-02T11:40:17Z)
• Fully Unsupervised Person Re-identification viaSelective Contrastive Learning [58.5284246878277]
  Person re-identification (ReID) aims at searching the same identity person among images captured by various cameras. We propose a novel selective contrastive learning framework for unsupervised feature learning. Experimental results demonstrate the superiority of our method in unsupervised person ReID compared with the state-of-the-arts.
  arXiv  Detail & Related papers  (2020-10-15T09:09:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.