Roadmap on Incentive Compatibility for AI Alignment and Governance in Sociotechnical Systems

• URL: http://arxiv.org/abs/2402.12907v3
• Date: Mon, 16 Jun 2025 17:46:07 GMT
• Title: Roadmap on Incentive Compatibility for AI Alignment and Governance in Sociotechnical Systems
• Authors: Zhaowei Zhang, Fengshuo Bai, Mingzhi Wang, Haoyang Ye, Chengdong Ma, Yaodong Yang,
• Abstract summary: Existing methodologies primarily focus on technical facets, often neglecting the intricate sociotechnical nature of AI systems.<n>We posit a new problem worth exploring: Incentive Compatibility Sociotechnical Alignment Problem (ICSAP)<n>We discuss three classical game problems for achieving IC: mechanism design, contract theory, and Bayesian persuasion, in addressing the perspectives, potentials, and challenges of solving ICSAP.
• Score: 10.463105660497357
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The burgeoning integration of artificial intelligence (AI) into human society brings forth significant implications for societal governance and safety. While considerable strides have been made in addressing AI alignment challenges, existing methodologies primarily focus on technical facets, often neglecting the intricate sociotechnical nature of AI systems, which can lead to a misalignment between the development and deployment contexts. To this end, we posit a new problem worth exploring: Incentive Compatibility Sociotechnical Alignment Problem (ICSAP). We hope this can call for more researchers to explore how to leverage the principles of Incentive Compatibility (IC) from game theory to bridge the gap between technical and societal components to maintain AI consensus with human societies in different contexts. We further discuss three classical game problems for achieving IC: mechanism design, contract theory, and Bayesian persuasion, in addressing the perspectives, potentials, and challenges of solving ICSAP, and provide preliminary implementation conceptions.

Related papers

• Amplifying Human Creativity and Problem Solving with AI Through Generative Collective Intelligence [3.160501875040035]
  We propose a general framework for human-AI collaboration that amplifies the capabilities of both types of intelligence.<n>We refer to this as Generative Collective Intelligence (GCI)<n>GCI employs AI in dual roles: as interactive agents and as technology that accumulates, organizes, and leverages knowledge.
  arXiv  Detail & Related papers  (2025-05-25T14:33:49Z)
• Aligning Generalisation Between Humans and Machines [74.120848518198]
  Recent advances in AI have resulted in technology that can support humans in scientific discovery and decision support but may also disrupt democracies and target individuals. The responsible use of AI increasingly shows the need for human-AI teaming. A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
  arXiv  Detail & Related papers  (2024-11-23T18:36:07Z)
• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We argue that shortcomings stem from one overarching failure: AI systems lack wisdom. While AI research has focused on task-level strategies, metacognition is underdeveloped in AI systems. We propose that integrating metacognitive capabilities into AI systems is crucial for enhancing their robustness, explainability, cooperation, and safety.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Navigating the sociotechnical labyrinth: Dynamic certification for responsible embodied AI [19.959138971887395]
  We argue that sociotechnical requirements shape the governance of artificially intelligent (AI) systems. Our proposed transdisciplinary approach is designed to ensure the safe, ethical, and practical deployment of AI systems.
  arXiv  Detail & Related papers  (2024-08-16T08:35:26Z)
• Problem Solving Through Human-AI Preference-Based Cooperation [74.39233146428492]
  We propose HAICo2, a novel human-AI co-construction framework. We take first steps towards a formalization of HAICo2 and discuss the difficult open research problems that it faces.
  arXiv  Detail & Related papers  (2024-08-14T11:06:57Z)
• Towards Bidirectional Human-AI Alignment: A Systematic Review for Clarifications, Framework, and Future Directions [101.67121669727354]
  Recent advancements in AI have highlighted the importance of guiding AI systems towards the intended goals, ethical principles, and values of individuals and groups, a concept broadly recognized as alignment. The lack of clarified definitions and scopes of human-AI alignment poses a significant obstacle, hampering collaborative efforts across research domains to achieve this alignment. We introduce a systematic review of over 400 papers published between 2019 and January 2024, spanning multiple domains such as Human-Computer Interaction (HCI), Natural Language Processing (NLP), Machine Learning (ML)
  arXiv  Detail & Related papers  (2024-06-13T16:03:25Z)
• Quantifying AI Vulnerabilities: A Synthesis of Complexity, Dynamical Systems, and Game Theory [0.0]
  We propose a novel approach that introduces three metrics: System Complexity Index (SCI), Lyapunov Exponent for AI Stability (LEAIS), and Nash Equilibrium Robustness (NER) SCI quantifies the inherent complexity of an AI system, LEAIS captures its stability and sensitivity to perturbations, and NER evaluates its strategic robustness against adversarial manipulation.
  arXiv  Detail & Related papers  (2024-04-07T07:05:59Z)
• A call for embodied AI [1.7544885995294304]
  We propose Embodied AI as the next fundamental step in the pursuit of Artificial General Intelligence. By broadening the scope of Embodied AI, we introduce a theoretical framework based on cognitive architectures. This framework is aligned with Friston's active inference principle, offering a comprehensive approach to EAI development.
  arXiv  Detail & Related papers  (2024-02-06T09:11:20Z)
• Fairness in Agreement With European Values: An Interdisciplinary Perspective on AI Regulation [61.77881142275982]
  This interdisciplinary position paper considers various concerns surrounding fairness and discrimination in AI, and discusses how AI regulations address them. We first look at AI and fairness through the lenses of law, (AI) industry, sociotechnology, and (moral) philosophy, and present various perspectives. We identify and propose the roles AI Regulation should take to make the endeavor of the AI Act a success in terms of AI fairness concerns.
  arXiv  Detail & Related papers  (2022-06-08T12:32:08Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)
• AI Development for the Public Interest: From Abstraction Traps to Sociotechnical Risks [2.765897573789737]
  We track the emergence of sociotechnical inquiry in three distinct subfields of AI research: AI Safety, Fair Machine Learning (Fair ML) and Human-in-the-Loop (HIL) Autonomy. We show that for each subfield, perceptions of Public Interest Technology (PIT) stem from the particular dangers faced by past integration of technical systems within a normative social order. We present a roadmap for a unified approach to sociotechnical graduate pedagogy in AI.
  arXiv  Detail & Related papers  (2021-02-04T18:54:20Z)
• Expanding Explainability: Towards Social Transparency in AI systems [20.41177660318785]
  Social Transparency (ST) is a socio-technically informed perspective that incorporates the socio-organizational context into explaining AI-mediated decision-making. Our work contributes to the discourse of Human-Centered XAI by expanding the design space of XAI.
  arXiv  Detail & Related papers  (2021-01-12T19:44:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.