Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities

• URL: http://arxiv.org/abs/2406.01757v1
• Date: Mon, 3 Jun 2024 19:52:41 GMT
• Title: Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities
• Authors: Golnoosh Farnadi, Mohammad Havaei, Negar Rostamzadeh,
• Abstract summary: We discuss that disparities towards marginalized communities are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon. We emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences. We conclude with a set of calls to action to mitigate disparity at its source.
• Score: 14.497314894554828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rise of foundation models holds immense promise for advancing AI, but this progress may amplify existing risks and inequalities, leaving marginalized communities behind. In this position paper, we discuss that disparities towards marginalized communities - performance, representation, privacy, robustness, interpretability and safety - are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon. We contrast foundation models with traditional models and highlight the potential for exacerbated disparity against marginalized communities. Moreover, we emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences, specifically to the people on the margin. We define marginalized communities within the machine learning context and explore the multifaceted nature of disparities. We analyze the sources of these disparities, tracing them from data creation, training and deployment procedures to highlight the complex technical and socio-technical landscape. To mitigate the pressing crisis, we conclude with a set of calls to action to mitigate disparity at its source.

Related papers

• Benchmarking the Spatial Robustness of DNNs via Natural and Adversarial Localized Corruptions [49.546479320670464]
  This paper introduces specialized metrics for benchmarking the spatial robustness of segmentation models. We propose region-aware multi-attack adversarial analysis, a method that enables a deeper understanding of model robustness. The results reveal that models respond to these two types of threats differently.
  arXiv  Detail & Related papers  (2025-04-02T11:37:39Z)
• A Theoretical Model for Grit in Pursuing Ambitious Ends [48.43624563381919]
  We provide a model of decision-making between stable and risky choices in the improving multi-armed bandits framework. We study the impact of various interventions, such as increasing grit or providing a financial safety net.
  arXiv  Detail & Related papers  (2025-03-04T19:17:42Z)
• Protocol Learning, Decentralized Frontier Risk and the No-Off Problem [56.74434512241989]
  We identify a third paradigm - Protocol Learning - where models are trained across decentralized networks of incentivized participants. This approach has the potential to aggregate orders of magnitude more computational resources than any single centralized entity. It also introduces novel challenges: heterogeneous and unreliable nodes, malicious participants, the need for unextractable models to preserve incentives, and complex governance dynamics.
  arXiv  Detail & Related papers  (2024-12-10T19:53:50Z)
• On the Fairness, Diversity and Reliability of Text-to-Image Generative Models [68.62012304574012]
  multimodal generative models have sparked critical discussions on their reliability, fairness and potential for misuse.<n>We propose an evaluation framework to assess model reliability by analyzing responses to global and local perturbations in the embedding space.<n>Our method lays the groundwork for detecting unreliable, bias-injected models and tracing the provenance of embedded biases.
  arXiv  Detail & Related papers  (2024-11-21T09:46:55Z)
• Reconciling Heterogeneous Effects in Causal Inference [44.99833362998488]
  We apply the Reconcile algorithm for model multiplicity in machine learning to reconcile heterogeneous effects in causal inference. Our results have tangible implications for ensuring fair outcomes in high-stakes such as healthcare, insurance, and housing.
  arXiv  Detail & Related papers  (2024-06-05T18:43:46Z)
• Participation in the age of foundation models [5.0317146589850195]
  Participatory approaches hold promise to lend agency and decision-making power to marginalized stakeholders. We examine existing attempts at incorporating participation into foundation models. We develop a blueprint for participatory foundation models that identifies more local, application-oriented opportunities for meaningful participation.
  arXiv  Detail & Related papers  (2024-05-29T19:53:23Z)
• On the Societal Impact of Open Foundation Models [93.67389739906561]
  We focus on open foundation models, defined here as those with broadly available model weights. We identify five distinctive properties of open foundation models that lead to both their benefits and risks.
  arXiv  Detail & Related papers  (2024-02-27T16:49:53Z)
• On Catastrophic Inheritance of Large Foundation Models [51.41727422011327]
  Large foundation models (LFMs) are claiming incredible performances. Yet great concerns have been raised about their mythic and uninterpreted potentials. We propose to identify a neglected issue deeply rooted in LFMs: Catastrophic Inheritance. We discuss the challenges behind this issue and propose UIM, a framework to understand the catastrophic inheritance of LFMs from both pre-training and downstream adaptation.
  arXiv  Detail & Related papers  (2024-02-02T21:21:55Z)
• A Survey on Transferability of Adversarial Examples across Deep Neural Networks [53.04734042366312]
  adversarial examples can manipulate machine learning models into making erroneous predictions. The transferability of adversarial examples enables black-box attacks which circumvent the need for detailed knowledge of the target model. This survey explores the landscape of the adversarial transferability of adversarial examples.
  arXiv  Detail & Related papers  (2023-10-26T17:45:26Z)
• Fair Models in Credit: Intersectional Discrimination and the Amplification of Inequity [5.333582981327497]
  The authors demonstrate the impact of such algorithmic bias in the microfinance context. We find that in addition to legally protected characteristics, sensitive attributes such as single parent status and number of children can result in imbalanced harm.
  arXiv  Detail & Related papers  (2023-08-01T10:34:26Z)
• Tensions Between the Proxies of Human Values in AI [20.303537771118048]
  We argue that the AI community needs to consider all the consequences of choosing certain formulations of these pillars. We point towards sociotechnical research for frameworks for the latter, but push for broader efforts into implementing these in practice.
  arXiv  Detail & Related papers  (2022-12-14T21:13:48Z)
• Fairness Increases Adversarial Vulnerability [50.90773979394264]
  This paper shows the existence of a dichotomy between fairness and robustness, and analyzes when achieving fairness decreases the model robustness to adversarial samples. Experiments on non-linear models and different architectures validate the theoretical findings in multiple vision domains. The paper proposes a simple, yet effective, solution to construct models achieving good tradeoffs between fairness and robustness.
  arXiv  Detail & Related papers  (2022-11-21T19:55:35Z)
• Social Diversity Reduces the Complexity and Cost of Fostering Fairness [63.70639083665108]
  We investigate the effects of interference mechanisms which assume incomplete information and flexible standards of fairness. We quantify the role of diversity and show how it reduces the need for information gathering. Our results indicate that diversity changes and opens up novel mechanisms available to institutions wishing to promote fairness.
  arXiv  Detail & Related papers  (2022-11-18T21:58:35Z)
• On the Opportunities and Risks of Foundation Models [256.61956234436553]
  We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration.
  arXiv  Detail & Related papers  (2021-08-16T17:50:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.