The Value of Disagreement in AI Design, Evaluation, and Alignment

• URL: http://arxiv.org/abs/2505.07772v1
• Date: Mon, 12 May 2025 17:22:30 GMT
• Title: The Value of Disagreement in AI Design, Evaluation, and Alignment
• Authors: Sina Fazelpour, Will Fleisher,
• Abstract summary: Disagreements are widespread across the design, evaluation, and alignment pipelines of AI systems.<n>Standard practices in AI development often obscure or eliminate disagreement, resulting in an engineered homogenization.<n>We develop a normative framework to guide practical reasoning about disagreement in the AI lifecycle.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Disagreements are widespread across the design, evaluation, and alignment pipelines of artificial intelligence (AI) systems. Yet, standard practices in AI development often obscure or eliminate disagreement, resulting in an engineered homogenization that can be epistemically and ethically harmful, particularly for marginalized groups. In this paper, we characterize this risk, and develop a normative framework to guide practical reasoning about disagreement in the AI lifecycle. Our contributions are two-fold. First, we introduce the notion of perspectival homogenization, characterizing it as a coupled ethical-epistemic risk that arises when an aspect of an AI system's development unjustifiably suppresses disagreement and diversity of perspectives. We argue that perspectival homogenization is best understood as a procedural risk, which calls for targeted interventions throughout the AI development pipeline. Second, we propose a normative framework to guide such interventions, grounded in lines of research that explain why disagreement can be epistemically beneficial, and how its benefits can be realized in practice. We apply this framework to key design questions across three stages of AI development tasks: when disagreement is epistemically valuable; whose perspectives should be included and preserved; how to structure tasks and navigate trade-offs; and how disagreement should be documented and communicated. In doing so, we challenge common assumptions in AI practice, offer a principled foundation for emerging participatory and pluralistic approaches, and identify actionable pathways for future work in AI design and governance.

Related papers

• Bridging the Gap: Integrating Ethics and Environmental Sustainability in AI Research and Practice [57.94036023167952]
  We argue that the efforts aiming to study AI's ethical ramifications should be made in tandem with those evaluating its impacts on the environment.<n>We propose best practices to better integrate AI ethics and sustainability in AI research and practice.
  arXiv  Detail & Related papers  (2025-04-01T13:53:11Z)
• Towards Developing Ethical Reasoners: Integrating Probabilistic Reasoning and Decision-Making for Complex AI Systems [4.854297874710511]
  A computational ethics framework is essential for AI and autonomous systems operating in complex, real-world environments.<n>Existing approaches often lack the adaptability needed to integrate ethical principles into dynamic and ambiguous contexts.<n>We outline the necessary ingredients for building a holistic, meta-level framework that combines intermediate representations, probabilistic reasoning, and knowledge representation.
  arXiv  Detail & Related papers  (2025-02-28T17:25:11Z)
• AI Ethics by Design: Implementing Customizable Guardrails for Responsible AI Development [0.0]
  We propose a structure that integrates rules, policies, and AI assistants to ensure responsible AI behavior.<n>Our approach accommodates ethical pluralism, offering a flexible and adaptable solution for the evolving landscape of AI governance.
  arXiv  Detail & Related papers  (2024-11-05T18:38:30Z)
• Towards Bidirectional Human-AI Alignment: A Systematic Review for Clarifications, Framework, and Future Directions [101.67121669727354]
  Recent advancements in AI have highlighted the importance of guiding AI systems towards the intended goals, ethical principles, and values of individuals and groups, a concept broadly recognized as alignment. The lack of clarified definitions and scopes of human-AI alignment poses a significant obstacle, hampering collaborative efforts across research domains to achieve this alignment. We introduce a systematic review of over 400 papers published between 2019 and January 2024, spanning multiple domains such as Human-Computer Interaction (HCI), Natural Language Processing (NLP), Machine Learning (ML)
  arXiv  Detail & Related papers  (2024-06-13T16:03:25Z)
• Crossing the principle-practice gap in AI ethics with ethical problem-solving [0.0]
  How to bridge the principle-practice gap separating ethical discourse from the technical side of AI development remains an open problem. EPS is a methodology promoting responsible, human-centric, and value-oriented AI development. We utilize EPS as a blueprint to propose the implementation of Ethics as a Service Platform.
  arXiv  Detail & Related papers  (2024-04-16T14:35:13Z)
• Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits [54.648819983899614]
  General purpose AI seems to have lowered the barriers for the public to use AI and harness its power. We introduce PARTICIP-AI, a framework for laypeople to speculate and assess AI use cases and their impacts.
  arXiv  Detail & Related papers  (2024-03-21T19:12:37Z)
• Towards Responsible AI in Banking: Addressing Bias for Fair Decision-Making [69.44075077934914]
  "Responsible AI" emphasizes the critical nature of addressing biases within the development of a corporate culture. This thesis is structured around three fundamental pillars: understanding bias, mitigating bias, and accounting for bias. In line with open-source principles, we have released Bias On Demand and FairView as accessible Python packages.
  arXiv  Detail & Related papers  (2024-01-13T14:07:09Z)
• AI Alignment: A Comprehensive Survey [69.61425542486275]
  AI alignment aims to make AI systems behave in line with human intentions and values.<n>We identify four principles as the key objectives of AI alignment: Robustness, Interpretability, Controllability, and Ethicality.<n>We decompose current alignment research into two key components: forward alignment and backward alignment.
  arXiv  Detail & Related papers  (2023-10-30T15:52:15Z)
• Ethics in conversation: Building an ethics assurance case for autonomous AI-enabled voice agents in healthcare [1.8964739087256175]
  The principles-based ethics assurance argument pattern is one proposal in the AI ethics landscape. This paper presents the interim findings of a case study applying this ethics assurance framework to the use of Dora, an AI-based telemedicine system.
  arXiv  Detail & Related papers  (2023-05-23T16:04:59Z)
• Fairness in Agreement With European Values: An Interdisciplinary Perspective on AI Regulation [61.77881142275982]
  This interdisciplinary position paper considers various concerns surrounding fairness and discrimination in AI, and discusses how AI regulations address them. We first look at AI and fairness through the lenses of law, (AI) industry, sociotechnology, and (moral) philosophy, and present various perspectives. We identify and propose the roles AI Regulation should take to make the endeavor of the AI Act a success in terms of AI fairness concerns.
  arXiv  Detail & Related papers  (2022-06-08T12:32:08Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)
• Transdisciplinary AI Observatory -- Retrospective Analyses and Future-Oriented Contradistinctions [22.968817032490996]
  This paper motivates the need for an inherently transdisciplinary AI observatory approach. Building on these AI observatory tools, we present near-term transdisciplinary guidelines for AI safety.
  arXiv  Detail & Related papers  (2020-11-26T16:01:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.