Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities

• URL: http://arxiv.org/abs/2406.01757v1
• Date: Mon, 3 Jun 2024 19:52:41 GMT
• Title: Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities
• Abstract summary: We discuss that disparities towards marginalized communities are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon. We emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences. We conclude with a set of calls to action to mitigate disparity at its source.
• Score: 14.497314894554828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rise of foundation models holds immense promise for advancing AI, but this progress may amplify existing risks and inequalities, leaving marginalized communities behind. In this position paper, we discuss that disparities towards marginalized communities - performance, representation, privacy, robustness, interpretability and safety - are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon. We contrast foundation models with traditional models and highlight the potential for exacerbated disparity against marginalized communities. Moreover, we emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences, specifically to the people on the margin. We define marginalized communities within the machine learning context and explore the multifaceted nature of disparities. We analyze the sources of these disparities, tracing them from data creation, training and deployment procedures to highlight the complex technical and socio-technical landscape. To mitigate the pressing crisis, we conclude with a set of calls to action to mitigate disparity at its source.

Related papers

• MoCA: Implicit Social Context Analysis [36.39584457494872]
  Implicit social contexts are pervasive in real-world interactions.<n>We introduce Implicit Social Context Analysis (MoCA), a novel task that systematically models implicit social scenarios.<n>We propose Conflict-Driven Abductive Reasoning (CoDAR), a novel framework that models the discrepancy between observed expressions and expected truthful behavior as cognitive conflict.
  arXiv  Detail & Related papers  (2026-08-06T09:54:24Z)
• Partial Fairness Awareness: Belief-Guided Strategic Mechanism for Strategic Agents [23.40393298376199]
  We propose the problem of partial fairness awareness (PFA)<n>We introduce a belief-guided strategic mechanism, wherein agents iteratively interact with the decision system and maintain a belief distribution over the candidate set of fairness constraints.<n>This belief-guided process enables agents, through iterative interaction and feedback, to update their belief distribution over the candidate set, thereby gradually aligning their belief with the grounding fairness constraint employed by the system.
  arXiv  Detail & Related papers  (2026-05-30T17:53:43Z)
• Emergent Social Intelligence Risks in Generative Multi-Agent Systems [106.60961969939676]
  Multi-agent systems are rapidly moving from lab prototypes to real-world deployments, where they jointly plan, negotiate, and allocate resources to solve complex tasks.<n>Here, we present a pioneer study of such emergent multi-agent risk in pathologies that involve competition over shared resources, sequential handoff collaboration, collective decision aggregation, and others.<n>We observe that such group behaviors arise frequently across repeated trials and a wide range of interaction conditions, rather than as rare or pathological cases.<n>These findings expose the dark side of intelligent multi-agent systems: a social intelligence risk where agent collectives spontaneously reproduce familiar failure patterns from human societies.
  arXiv  Detail & Related papers  (2026-03-29T17:10:28Z)
• Position: General Alignment Has Hit a Ceiling; Edge Alignment Must Be Taken Seriously [51.03213216886717]
  We take the position that the dominant paradigm of General Alignment reaches a structural ceiling in settings with conflicting values.<n>We introduce Edge Alignment as a distinct approach in which systems preserve multi dimensional value structure.
  arXiv  Detail & Related papers  (2026-02-23T16:51:43Z)
• A Feminist Account of Intersectional Algorithmic Fairness [0.0]
  We propose Substantive Intersectional Algorithmic Fairness, extending Green's notion of substantive algorithmic fairness with insights from intersectional feminist theory.<n>We introduce ten desiderata within the ROOF methodology to guide the design, assessment, and deployment of algorithmic systems.<n>By bridging computational and social science perspectives, we provide actionable guidance for more equitable, inclusive, and context-sensitive intersectional algorithmic practices.
  arXiv  Detail & Related papers  (2025-08-25T12:09:04Z)
• Benchmarking the Spatial Robustness of DNNs via Natural and Adversarial Localized Corruptions [49.546479320670464]
  This paper introduces specialized metrics for benchmarking the spatial robustness of segmentation models. We propose region-aware multi-attack adversarial analysis, a method that enables a deeper understanding of model robustness. The results reveal that models respond to these two types of threats differently.
  arXiv  Detail & Related papers  (2025-04-02T11:37:39Z)
• A Theoretical Model for Grit in Pursuing Ambitious Ends [48.43624563381919]
  We provide a model of decision-making between stable and risky choices in the improving multi-armed bandits framework. We study the impact of various interventions, such as increasing grit or providing a financial safety net.
  arXiv  Detail & Related papers  (2025-03-04T19:17:42Z)
• Protocol Learning, Decentralized Frontier Risk and the No-Off Problem [56.74434512241989]
  We identify a third paradigm - Protocol Learning - where models are trained across decentralized networks of incentivized participants. This approach has the potential to aggregate orders of magnitude more computational resources than any single centralized entity. It also introduces novel challenges: heterogeneous and unreliable nodes, malicious participants, the need for unextractable models to preserve incentives, and complex governance dynamics.
  arXiv  Detail & Related papers  (2024-12-10T19:53:50Z)
• On the Fairness, Diversity and Reliability of Text-to-Image Generative Models [68.62012304574012]
  multimodal generative models have sparked critical discussions on their reliability, fairness and potential for misuse.<n>We propose an evaluation framework to assess model reliability by analyzing responses to global and local perturbations in the embedding space.<n>Our method lays the groundwork for detecting unreliable, bias-injected models and tracing the provenance of embedded biases.
  arXiv  Detail & Related papers  (2024-11-21T09:46:55Z)
• Reconciling Heterogeneous Effects in Causal Inference [44.99833362998488]
  We apply the Reconcile algorithm for model multiplicity in machine learning to reconcile heterogeneous effects in causal inference. Our results have tangible implications for ensuring fair outcomes in high-stakes such as healthcare, insurance, and housing.
  arXiv  Detail & Related papers  (2024-06-05T18:43:46Z)
• Participation in the age of foundation models [5.0317146589850195]
  Participatory approaches hold promise to lend agency and decision-making power to marginalized stakeholders. We examine existing attempts at incorporating participation into foundation models. We develop a blueprint for participatory foundation models that identifies more local, application-oriented opportunities for meaningful participation.
  arXiv  Detail & Related papers  (2024-05-29T19:53:23Z)
• On the Societal Impact of Open Foundation Models [93.67389739906561]
  We focus on open foundation models, defined here as those with broadly available model weights. We identify five distinctive properties of open foundation models that lead to both their benefits and risks.
  arXiv  Detail & Related papers  (2024-02-27T16:49:53Z)
• On Catastrophic Inheritance of Large Foundation Models [51.41727422011327]
  Large foundation models (LFMs) are claiming incredible performances. Yet great concerns have been raised about their mythic and uninterpreted potentials. We propose to identify a neglected issue deeply rooted in LFMs: Catastrophic Inheritance. We discuss the challenges behind this issue and propose UIM, a framework to understand the catastrophic inheritance of LFMs from both pre-training and downstream adaptation.
  arXiv  Detail & Related papers  (2024-02-02T21:21:55Z)
• A Survey on Transferability of Adversarial Examples across Deep Neural Networks [53.04734042366312]
  adversarial examples can manipulate machine learning models into making erroneous predictions. The transferability of adversarial examples enables black-box attacks which circumvent the need for detailed knowledge of the target model. This survey explores the landscape of the adversarial transferability of adversarial examples.
  arXiv  Detail & Related papers  (2023-10-26T17:45:26Z)
• Fair Models in Credit: Intersectional Discrimination and the Amplification of Inequity [5.333582981327497]
  The authors demonstrate the impact of such algorithmic bias in the microfinance context. We find that in addition to legally protected characteristics, sensitive attributes such as single parent status and number of children can result in imbalanced harm.
  arXiv  Detail & Related papers  (2023-08-01T10:34:26Z)
• Tensions Between the Proxies of Human Values in AI [20.303537771118048]
  We argue that the AI community needs to consider all the consequences of choosing certain formulations of these pillars. We point towards sociotechnical research for frameworks for the latter, but push for broader efforts into implementing these in practice.
  arXiv  Detail & Related papers  (2022-12-14T21:13:48Z)
• Fairness Increases Adversarial Vulnerability [50.90773979394264]
  This paper shows the existence of a dichotomy between fairness and robustness, and analyzes when achieving fairness decreases the model robustness to adversarial samples. Experiments on non-linear models and different architectures validate the theoretical findings in multiple vision domains. The paper proposes a simple, yet effective, solution to construct models achieving good tradeoffs between fairness and robustness.
  arXiv  Detail & Related papers  (2022-11-21T19:55:35Z)
• Social Diversity Reduces the Complexity and Cost of Fostering Fairness [63.70639083665108]
  We investigate the effects of interference mechanisms which assume incomplete information and flexible standards of fairness. We quantify the role of diversity and show how it reduces the need for information gathering. Our results indicate that diversity changes and opens up novel mechanisms available to institutions wishing to promote fairness.
  arXiv  Detail & Related papers  (2022-11-18T21:58:35Z)
• On the Opportunities and Risks of Foundation Models [256.61956234436553]
  We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration.
  arXiv  Detail & Related papers  (2021-08-16T17:50:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.