The Ethical Decision Head: Operationalizing Normative Ethics in Autonomous Vehicles via Reinforcement Learning from Human Feedback

• URL: http://arxiv.org/abs/2608.16710v1
• Date: Mon, 17 Aug 2026 15:26:09 GMT
• Title: The Ethical Decision Head: Operationalizing Normative Ethics in Autonomous Vehicles via Reinforcement Learning from Human Feedback
• Abstract summary: This paper details a deep re- inforcement learning framework that encodes ethical reasoning as a differentiable reward signal.<n>It learns morally-aligned driving behavior in scenarios whose state representation is aligned with the CARLA simulation environment.<n>Results reveal an asymmetry in the learnability of normative ethical frameworks under human su- pervision.
• Score: 3.0813364144350834
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As autonomous vehicles (AVs) approach Level 4 and Level 5 operational capability [SAE International, 2018], their on- board decision systems must handle not only safety-critical locomotion but also their subsequent moral weight. This paper details the Ethical Decision Head (EDH), a deep re- inforcement learning (RL) framework that encodes ethical reasoning as a differentiable reward signal, enabling a pol- icy gradient agent to learn morally-aligned driving behavior in scenarios whose state representation is aligned with the CARLA simulation environment [Dosovitskiy et al., 2017]. Two normative frameworks are instantiated and evaluated: a Utilitarian framework minimizing total casualties and a Kan- tian framework enforcing course maintenance as a categori- cal imperative. The EDH is trained via Proximal Policy Op- timization (PPO) [Schulman et al., 2017] against a Bradley- Terry reward model [Bradley and Terry, 1952] learned from pairwise human preference annotations over 200 collision- imminent scenarios. Results reveal an asymmetry in the learnability of normative ethical frameworks under human su- pervision. The Kantian condition, which reduces to a con- stant prediction task under the codebook, serves as a pipeline control: it confirms training stability and rules out infrastruc- ture failure as an explanation for the utilitarian result. The Utilitarian agent learned something more unsettling: human raters rewarded self-sacrifice over casualty minimization, and the model learned that preference faithfully. This divergence between what humans prescribe in theory and what they re- ward in practice suggests that RLHF does not learn ethics as philosophers define it, but as humans live it.

Related papers

• Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models [49.45276299939287]
  Large Language Models (LLMs) increasingly handle complex subjective tasks.<n>LLMs fluctuate unpredictably under minor wording changes.<n> resolving this duality is critical for reliable AI alignment.
  arXiv  Detail & Related papers  (2026-09-15T03:35:04Z)
• Emergent alignment and the projectability of ethical personas [39.3098730337656]
  We finetune a helpful-only model on broad and narrow safety tasks.<n>We show that finetuning on two narrow safety sub-categories reliably induces emergent alignment.<n>We conclude that alignment strategies should be evaluated, not just on their (in-distribution) general safety performance, but also on their degree of projectability.
  arXiv  Detail & Related papers  (2026-06-08T13:30:29Z)
• Toward Virtuous Reinforcement Learning [1.3428344011390776]
  This paper critiques common patterns in machine ethics for Reinforcement Learning (RL)<n>We instead treat ethics as policy level dispositions, that is, relatively stable habits that hold up when incentives, partners, or contexts change.
  arXiv  Detail & Related papers  (2025-12-03T20:30:37Z)
• Large Reasoning Models Learn Better Alignment from Flawed Thinking [56.08883934423522]
  Large reasoning models (LRMs) "think" by generating structured chain-of-thought (CoT) before producing a final answer.<n>We propose RECAP, a principled reinforcement learning (RL) method for post-training that explicitly teaches models to override flawed reasoning trajectories.
  arXiv  Detail & Related papers  (2025-10-01T14:15:43Z)
• Survival at Any Cost? LLMs and the Choice Between Self-Preservation and Human Harm [0.0]
  We introduce DECIDE-SIM, a novel simulation framework that evaluates Large Language Models (LLMs) in multi-agent survival scenarios.<n>Our comprehensive evaluation of 11 LLMs reveals a striking heterogeneity in their ethical conduct, highlighting a critical misalignment with human-centric values.<n>We introduce an Ethical Self-Regulation System (ESRS) that models internal affective states of guilt and satisfaction as a feedback mechanism.
  arXiv  Detail & Related papers  (2025-09-15T17:53:11Z)
• Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving [1.2891210250935148]
  We present a hierarchical Safe Reinforcement Learning framework that augments standard driving objectives with ethics-aware cost signals.<n>A Safe RL agent is trained using a composite ethical risk cost, combining collision probability and harm severity, to generate high-level motion targets.<n>A dynamic, risk-sensitive Prioritized Experience mechanism amplifies learning from rare but critical, high-risk events.
  arXiv  Detail & Related papers  (2025-08-19T14:24:02Z)
• Addressing Moral Uncertainty using Large Language Models for Ethical Decision-Making [0.42481744176244507]
  We present an ethical decision-making framework that refines a pre-trained reinforcement learning (RL) model using a task-agnostic ethical layer.<n>An ethical layer aggregates belief scores from multiple moral perspectives using Belief Jensen-Shannon Divergence and Dempster-Shafer Theory into probability scores that also serve as the shaping reward.<n>This integrated learning framework helps the RL agent navigate moral uncertainty in complex environments and enables it to make morally sound decisions across diverse tasks.
  arXiv  Detail & Related papers  (2025-02-17T19:05:55Z)
• REBEL: Reward Regularization-Based Approach for Robotic Reinforcement Learning from Human Feedback [61.54791065013767]
  A misalignment between the reward function and human preferences can lead to catastrophic outcomes in the real world.<n>Recent methods aim to mitigate misalignment by learning reward functions from human preferences.<n>We propose a novel concept of reward regularization within the robotic RLHF framework.
  arXiv  Detail & Related papers  (2023-12-22T04:56:37Z)
• Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism [91.52263068880484]
  We study offline Reinforcement Learning with Human Feedback (RLHF) We aim to learn the human's underlying reward and the MDP's optimal policy from a set of trajectories induced by human choices. RLHF is challenging for multiple reasons: large state space but limited human feedback, the bounded rationality of human decisions, and the off-policy distribution shift.
  arXiv  Detail & Related papers  (2023-05-29T01:18:39Z)
• Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark [61.43264961005614]
  We develop a benchmark of 134 Choose-Your-Own-Adventure games containing over half a million rich, diverse scenarios. We evaluate agents' tendencies to be power-seeking, cause disutility, and commit ethical violations. Our results show that agents can both act competently and morally, so concrete progress can be made in machine ethics.
  arXiv  Detail & Related papers  (2023-04-06T17:59:03Z)
• When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment [96.77970239683475]
  AI systems need to be able to understand, interpret and predict human moral judgments and decisions. A central challenge for AI safety is capturing the flexibility of the human moral mind. We present a novel challenge set consisting of rule-breaking question answering.
  arXiv  Detail & Related papers  (2022-10-04T09:04:27Z)
• Reinforcement Learning Guided by Provable Normative Compliance [0.0]
  Reinforcement learning (RL) has shown promise as a tool for engineering safe, ethical, or legal behaviour in autonomous agents. We use multi-objective RL (MORL) to balance the ethical objective of avoiding violations with a non-ethical objective. We show that our approach works for a multiplicity of MORL techniques, and show that it is effective regardless of the magnitude of the punishment we assign.
  arXiv  Detail & Related papers  (2022-03-30T13:10:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.