Normative Moral Pluralism for AI: A Framework for Deliberation in Complex Moral Contexts

• URL: http://arxiv.org/abs/2508.08333v1
• Date: Sun, 10 Aug 2025 14:52:23 GMT
• Title: Normative Moral Pluralism for AI: A Framework for Deliberation in Complex Moral Contexts
• Authors: David-Doron Yaacov,
• Abstract summary: The conceptual framework proposed in this paper centers on the development of a deliberative moral reasoning system.<n>It is designed to process complex moral situations by generating, filtering, and weighing normative arguments drawn from diverse ethical perspectives.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The conceptual framework proposed in this paper centers on the development of a deliberative moral reasoning system - one designed to process complex moral situations by generating, filtering, and weighing normative arguments drawn from diverse ethical perspectives. While the framework is rooted in Machine Ethics, it also makes a substantive contribution to Value Alignment by outlining a system architecture that links structured moral reasoning to action under time constraints. Grounded in normative moral pluralism, this system is not constructed to imitate behavior but is built on reason-sensitive deliberation over structured moral content in a transparent and principled manner. Beyond its role as a deliberative system, it also serves as the conceptual foundation for a novel two-level architecture: functioning as a moral reasoning teacher envisioned to train faster models that support real-time responsiveness without reproducing the full structure of deliberative reasoning. Together, the deliberative and intuitive components are designed to enable both deep reflection and responsive action. A key design feature is the dual-hybrid structure: a universal layer that defines a moral threshold through top-down and bottom-up learning, and a local layer that learns to weigh competing considerations in context while integrating culturally specific normative content, so long as it remains within the universal threshold. By extending the notion of moral complexity to include not only conflicting beliefs but also multifactorial dilemmas, multiple stakeholders, and the integration of non-moral considerations, the framework aims to support morally grounded decision-making in realistic, high-stakes contexts.

Related papers

• MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents [10.221486703870996]
  We introduce Morality Chains, a novel formalism for representing moral norms as ordered deontic constraints, and MoralityGym, a benchmark of 98 ethical-dilemma problems presented as trolley-dilemma-style Gymnasium environments.<n>This work provides a foundation for developing AI systems that behave more reliably, transparently, and ethically in complex real-world contexts.
  arXiv  Detail & Related papers  (2026-02-13T15:40:32Z)
• Mirror: A Multi-Agent System for AI-Assisted Ethics Review [104.3684024153469]
  Mirror is an agentic framework for AI-assisted ethical review.<n>It integrates ethical reasoning, structured rule interpretation, and multi-agent deliberation within a unified architecture.
  arXiv  Detail & Related papers  (2026-02-09T03:38:55Z)
• AI Safety, Alignment, and Ethics (AI SAE) [0.0]
  Moral norms are adaptive mechanisms that render cooperation fitness-viable under selection pressure.<n>Current alignment approaches add ethics post hoc, treating it as an external constraint rather than embedding it as an evolutionary strategy for cooperation.<n>I propose a governance--embedding--representation pipeline linking moral representation learning to system-level design and institutional governance.
  arXiv  Detail & Related papers  (2025-09-28T20:52:36Z)
• Are Language Models Consequentialist or Deontological Moral Reasoners? [69.85385952436044]
  We focus on a large-scale analysis of the moral reasoning traces provided by large language models (LLMs)<n>We introduce and test a taxonomy of moral rationales to systematically classify reasoning traces according to two main normative ethical theories: consequentialism and deontology.
  arXiv  Detail & Related papers  (2025-05-27T17:51:18Z)
• Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt [44.522343543870804]
  We are worried that such systems, which overlook enduring moral diversity, will spark resistance, erode trust, and destabilize our institutions.<n>This paper traces the underlying problem to an often-unstated Axiom of Rational Convergence: the idea that under ideal conditions, rational agents will converge in the limit of conversation on a single ethics.<n>Treating that premise as both optional and doubtful, we propose what we call the appropriateness framework: an alternative approach grounded in conflict theory, cultural evolution, multi-agent systems, and institutional economics.
  arXiv  Detail & Related papers  (2025-05-08T12:55:07Z)
• The Convergent Ethics of AI? Analyzing Moral Foundation Priorities in Large Language Models with a Multi-Framework Approach [6.0972634521845475]
  This paper introduces the Priorities in Reasoning and Intrinsic Moral Evaluation (PRIME) framework.<n>PRIME is a comprehensive methodology for analyzing moral priorities across foundational ethical dimensions.<n>We apply this framework to six leading large language models (LLMs) through a dual-protocol approach.
  arXiv  Detail & Related papers  (2025-04-27T14:26:48Z)
• Towards Developing Ethical Reasoners: Integrating Probabilistic Reasoning and Decision-Making for Complex AI Systems [4.854297874710511]
  A computational ethics framework is essential for AI and autonomous systems operating in complex, real-world environments.<n>Existing approaches often lack the adaptability needed to integrate ethical principles into dynamic and ambiguous contexts.<n>We outline the necessary ingredients for building a holistic, meta-level framework that combines intermediate representations, probabilistic reasoning, and knowledge representation.
  arXiv  Detail & Related papers  (2025-02-28T17:25:11Z)
• Technology as uncharted territory: Contextual integrity and the notion of AI as new ethical ground [55.2480439325792]
  I argue that efforts to promote responsible and ethical AI can inadvertently contribute to and seemingly legitimize this disregard for established contextual norms.<n>I question the current narrow prioritization in AI ethics of moral innovation over moral preservation.
  arXiv  Detail & Related papers  (2024-12-06T15:36:13Z)
• Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories? [78.3738172874685]
  Making moral judgments is an essential step toward developing ethical AI systems. Prevalent approaches are mostly implemented in a bottom-up manner, which uses a large set of annotated data to train models based on crowd-sourced opinions about morality. This work proposes a flexible top-down framework to steer (Large) Language Models (LMs) to perform moral reasoning with well-established moral theories from interdisciplinary research.
  arXiv  Detail & Related papers  (2023-08-29T15:57:32Z)
• MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions [71.25236662907056]
  A moral dialogue system aligned with users' values could enhance conversation engagement and user connections. We propose a framework, MoralDial, to train and evaluate moral dialogue systems.
  arXiv  Detail & Related papers  (2022-12-21T02:21:37Z)
• Reinforcement Learning Under Moral Uncertainty [13.761051314923634]
  An ambitious goal for machine learning is to create agents that behave ethically. While ethical agents could be trained by rewarding correct behavior under a specific moral theory, there remains widespread disagreement about the nature of morality. This paper proposes two training methods that realize different points among competing desiderata, and trains agents in simple environments to act under moral uncertainty.
  arXiv  Detail & Related papers  (2020-06-08T16:40:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.