A Revealed Preference Framework for AI Alignment

• URL: http://arxiv.org/abs/2603.27868v1
• Date: Sun, 29 Mar 2026 20:54:01 GMT
• Title: A Revealed Preference Framework for AI Alignment
• Abstract summary: Human decision makers increasingly delegate choices to AI agents, raising a natural question: does the AI implement the human principal's preferences or pursue its own?<n>I introduce the Luce Alignment Model, where the AI's choices are a mixture of two Luce rules, one reflecting the human's preferences and the other the AI's.<n>I show that the AI's alignment can be generically identified in two settings: the laboratory setting, where both human and AI choices are observed, and the field setting, where only AI choices are observed.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Human decision makers increasingly delegate choices to AI agents, raising a natural question: does the AI implement the human principal's preferences or pursue its own? To study this question using revealed preference techniques, I introduce the Luce Alignment Model, where the AI's choices are a mixture of two Luce rules, one reflecting the human's preferences and the other the AI's. I show that the AI's alignment (similarity of human and AI preferences) can be generically identified in two settings: the laboratory setting, where both human and AI choices are observed, and the field setting, where only AI choices are observed.

Related papers

• AI, Take the Wheel: What Drives Delegation and Trust in Human-Computer Cooperative Question Answering? [22.50855320823746]
  Humans can make mistakes in deciding whether to trust AI over their own judgment.<n>We study two distinct reliance decisions: the delegation choice and the adoption choice.<n>We find humans make suboptimal collaboration decisions, both under-relying on correct AI suggestions and over-relying when AI misleads them.
  arXiv  Detail & Related papers  (2026-05-27T10:05:16Z)
• Towards a Theory of AI Personhood [1.6317061277457001]
  We outline necessary conditions for AI personhood, focusing on agency, theory-of-mind, and self-awareness.<n>If AI systems can be considered persons, then typical framings of AI alignment may be incomplete.
  arXiv  Detail & Related papers  (2025-01-23T10:31:26Z)
• Aligning Generalisation Between Humans and Machines [74.120848518198]
  AI technology can support humans in scientific discovery and forming decisions, but may also disrupt democracies and target individuals.<n>The responsible use of AI and its participation in human-AI teams increasingly shows the need for AI alignment.<n>A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
  arXiv  Detail & Related papers  (2024-11-23T18:36:07Z)
• Rolling in the deep of cognitive and AI biases [1.556153237434314]
  We argue that there is urgent need to understand AI as a sociotechnical system, inseparable from the conditions in which it is designed, developed and deployed. We address this critical issue by following a radical new methodology under which human cognitive biases become core entities in our AI fairness overview. We introduce a new mapping, which justifies the humans to AI biases and we detect relevant fairness intensities and inter-dependencies.
  arXiv  Detail & Related papers  (2024-07-30T21:34:04Z)
• Position: Towards Bidirectional Human-AI Alignment [109.57781720848669]
  We argue that the research community should explicitly define and critically reflect on "alignment" to account for the bidirectional and dynamic relationship between humans and AI.<n>We introduce the Bidirectional Human-AI Alignment framework, which not only incorporates traditional efforts to align AI with human values but also introduces the critical, underexplored dimension of aligning humans with AI.
  arXiv  Detail & Related papers  (2024-06-13T16:03:25Z)
• The AI Alignment Paradox [10.674155943520729]
  The better we align AI models with our values, the easier we may make it for adversaries to misalign the models. With AI's increasing real-world impact, it is imperative that a broad community of researchers be aware of the AI alignment paradox.
  arXiv  Detail & Related papers  (2024-05-31T14:06:24Z)
• Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making [25.913473823070863]
  Existing research on human-AI collaborative decision-making focuses mainly on the interaction between AI and individual decision-makers. This paper presents a wizard-of-oz study in which two participants and an AI form a committee to rank three English essays.
  arXiv  Detail & Related papers  (2023-02-17T11:07:17Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)
• Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork [54.309495231017344]
  We argue that AI systems should be trained in a human-centered manner, directly optimized for team performance. We study this proposal for a specific type of human-AI teaming, where the human overseer chooses to either accept the AI recommendation or solve the task themselves. Our experiments with linear and non-linear models on real-world, high-stakes datasets show that the most accuracy AI may not lead to highest team performance.
  arXiv  Detail & Related papers  (2020-04-27T19:06:28Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.