Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images

• URL: http://arxiv.org/abs/2401.14322v1
• Date: Thu, 25 Jan 2024 17:19:22 GMT
• Title: Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images
• Authors: Hansa Srinivasan, Candice Schumann, Aradhana Sinha, David Madras, Gbolahan Oluwafemi Olanubi, Alex Beutel, Susanna Ricco, Jilin Chen
• Abstract summary: We introduce a diverse people image ranking method which more flexibly aligns with human notions of people diversity. The Perception-Aligned Text-derived Human representation Space (PATHS) aims to capture all or many relevant features of people-related diversity.
• Score: 11.038712922077458
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Capturing the diversity of people in images is challenging: recent literature tends to focus on diversifying one or two attributes, requiring expensive attribute labels or building classifiers. We introduce a diverse people image ranking method which more flexibly aligns with human notions of people diversity in a less prescriptive, label-free manner. The Perception-Aligned Text-derived Human representation Space (PATHS) aims to capture all or many relevant features of people-related diversity, and, when used as the representation space in the standard Maximal Marginal Relevance (MMR) ranking algorithm, is better able to surface a range of types of people-related diversity (e.g. disability, cultural attire). PATHS is created in two stages. First, a text-guided approach is used to extract a person-diversity representation from a pre-trained image-text model. Then this representation is fine-tuned on perception judgments from human annotators so that it captures the aspects of people-related similarity that humans find most salient. Empirical results show that the PATHS method achieves diversity better than baseline methods, according to side-by-side ratings from human annotators.

Related papers

• GRADE: Quantifying Sample Diversity in Text-to-Image Models [66.12068246962762]
  We propose GRADE: Granular Attribute Diversity Evaluation, an automatic method for quantifying sample diversity. We measure the overall diversity of 12 T2I models using 400 concept-attribute pairs, revealing that all models display limited variation. Our work proposes a modern, semantically-driven approach to measure sample diversity and highlights the stunning homogeneity in outputs by T2I models.
  arXiv  Detail & Related papers  (2024-10-29T23:10:28Z)
• Whose Preferences? Differences in Fairness Preferences and Their Impact on the Fairness of AI Utilizing Human Feedback [8.04095222893591]
  We find significant gaps in fairness preferences depending on the race, age, political stance, educational level, and LGBTQ+ identity of annotators. We also demonstrate that demographics mentioned in text have a strong influence on how users perceive individual fairness in moderation.
  arXiv  Detail & Related papers  (2024-06-09T19:42:25Z)
• Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment [84.32768080422349]
  Alignment with human preference prevents large language models from generating misleading or toxic content. We propose a new formulation of prompt diversity, implying a linear correlation with the final performance of LLMs after fine-tuning.
  arXiv  Detail & Related papers  (2024-03-17T07:08:55Z)
• Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL [51.48239006107272]
  In this paper, we discuss how to measure and improve the diversity of the demonstrations for text-to-diversity research. We propose fusing iteratively for demonstrations (Fused) to build a high-diversity demonstration pool. Our method achieves an average improvement of 3.2% and 5.0% with and without human labeling on several mainstream datasets.
  arXiv  Detail & Related papers  (2024-02-16T13:13:18Z)
• UniHuman: A Unified Model for Editing Human Images in the Wild [49.896715833075106]
  We propose UniHuman, a unified model that addresses multiple facets of human image editing in real-world settings. To enhance the model's generation quality and generalization capacity, we leverage guidance from human visual encoders. In user studies, UniHuman is preferred by the users in an average of 77% of cases.
  arXiv  Detail & Related papers  (2023-12-22T05:00:30Z)
• Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation [0.0]
  We introduce Diverse Diffusion, a method for boosting image diversity beyond gender and ethnicity. Our approach contributes to the creation of more inclusive and representative AI-generated art.
  arXiv  Detail & Related papers  (2023-10-19T08:48:23Z)
• Diversity from Human Feedback [39.05609489642456]
  We propose the problem of learning a behavior space from human feedback and present a general method called Diversity from Human Feedback (DivHF) to solve it. DivHF learns a behavior consistent with human preference by querying human feedback. We demonstrate the effectiveness of DivHF by integrating it with the Quality-Diversity optimization algorithm MAP-Elites and conducting experiments on the QDax suite.
  arXiv  Detail & Related papers  (2023-10-10T14:13:59Z)
• Stable Bias: Analyzing Societal Representations in Diffusion Models [72.27121528451528]
  We propose a new method for exploring the social biases in Text-to-Image (TTI) systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts. We leverage this method to analyze images generated by 3 popular TTI systems and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents.
  arXiv  Detail & Related papers  (2023-03-20T19:32:49Z)
• Auditing Gender Presentation Differences in Text-to-Image Models [54.16959473093973]
  We study how gender is presented differently in text-to-image models. By probing gender indicators in the input text, we quantify the frequency differences of presentation-centric attributes. We propose an automatic method to estimate such differences.
  arXiv  Detail & Related papers  (2023-02-07T18:52:22Z)
• Evaluating the Evaluation of Diversity in Natural Language Generation [43.05127848086264]
  We propose a framework for evaluating diversity metrics in natural language generation systems. Our framework can advance the understanding of different diversity metrics, an essential step on the road towards better NLG systems.
  arXiv  Detail & Related papers  (2020-04-06T20:44:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.