AI and Human Oversight: A Risk-Based Framework for Alignment

• URL: http://arxiv.org/abs/2510.09090v1
• Date: Fri, 10 Oct 2025 07:36:44 GMT
• Title: AI and Human Oversight: A Risk-Based Framework for Alignment
• Authors: Laxmiraju Kandikatla, Branislav Radeljic,
• Abstract summary: This paper examines strategies for designing AI systems that uphold fundamental rights, strengthen human agency, and embed effective human oversight mechanisms.<n>By linking the level of AI model risk to the appropriate form of human oversight, the paper underscores the critical role of human involvement in the responsible deployment of AI.
• Score: 0.2039123720459736
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As Artificial Intelligence (AI) technologies continue to advance, protecting human autonomy and promoting ethical decision-making are essential to fostering trust and accountability. Human agency (the capacity of individuals to make informed decisions) should be actively preserved and reinforced by AI systems. This paper examines strategies for designing AI systems that uphold fundamental rights, strengthen human agency, and embed effective human oversight mechanisms. It discusses key oversight models, including Human-in-Command (HIC), Human-in-the-Loop (HITL), and Human-on-the-Loop (HOTL), and proposes a risk-based framework to guide the implementation of these mechanisms. By linking the level of AI model risk to the appropriate form of human oversight, the paper underscores the critical role of human involvement in the responsible deployment of AI, balancing technological innovation with the protection of individual values and rights. In doing so, it aims to ensure that AI technologies are used responsibly, safeguarding individual autonomy while maximizing societal benefits.

Related papers

• Fully Autonomous AI Agents Should Not be Developed [58.88624302082713]
  This paper argues that fully autonomous AI agents should not be developed.<n>In support of this position, we build from prior scientific literature and current product marketing to delineate different AI agent levels.<n>Our analysis reveals that risks to people increase with the autonomy of a system.
  arXiv  Detail & Related papers  (2025-02-04T19:00:06Z)
• Building Symbiotic AI: Reviewing the AI Act for a Human-Centred, Principle-Based Framework [3.723174617224632]
  The European Union has released a new legal framework, the AI Act, to regulate AI.<n>At the same time, researchers offer a new perspective on AI systems, commonly known as Human-Centred AI (HCAI)<n>This article aims to identify principles that characterise the design and development of Symbiotic AI systems.
  arXiv  Detail & Related papers  (2025-01-14T11:53:10Z)
• Aligning Generalisation Between Humans and Machines [74.120848518198]
  AI technology can support humans in scientific discovery and forming decisions, but may also disrupt democracies and target individuals.<n>The responsible use of AI and its participation in human-AI teams increasingly shows the need for AI alignment.<n>A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
  arXiv  Detail & Related papers  (2024-11-23T18:36:07Z)
• Engineering Trustworthy AI: A Developer Guide for Empirical Risk Minimization [53.80919781981027]
  Key requirements for trustworthy AI can be translated into design choices for the components of empirical risk minimization. We hope to provide actionable guidance for building AI systems that meet emerging standards for trustworthiness of AI.
  arXiv  Detail & Related papers  (2024-10-25T07:53:32Z)
• Combining Theory of Mind and Kindness for Self-Supervised Human-AI Alignment [0.0]
  Current AI models prioritize task optimization over safety, leading to risks of unintended harm. We propose a novel human-inspired approach which aims to address these various concerns and help align competing objectives.
  arXiv  Detail & Related papers  (2024-10-21T22:04:44Z)
• The European Commitment to Human-Centered Technology: The Integral Role of HCI in the EU AI Act's Success [4.202570851109354]
  The EU has enacted the AI Act, regulating market access for AI-based systems. The Act focuses regulation on transparency, explainability, and the human ability to understand and control AI systems. The EU issues a democratic call for human-centered AI systems and, in turn, an interdisciplinary research agenda for human-centered innovation in AI development.
  arXiv  Detail & Related papers  (2024-02-22T17:35:29Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• AI Potentiality and Awareness: A Position Paper from the Perspective of Human-AI Teaming in Cybersecurity [18.324118502535775]
  We argue that human-AI teaming is worthwhile in cybersecurity. We emphasize the importance of a balanced approach that incorporates AI's computational power with human expertise.
  arXiv  Detail & Related papers  (2023-09-28T01:20:44Z)
• Reflective Hybrid Intelligence for Meaningful Human Control in Decision-Support Systems [4.1454448964078585]
  We introduce the notion of self-reflective AI systems for meaningful human control over AI systems. We propose a framework that integrates knowledge from psychology and philosophy with formal reasoning methods and machine learning approaches. We argue that self-reflective AI systems can lead to self-reflective hybrid systems (human + AI)
  arXiv  Detail & Related papers  (2023-07-12T13:32:24Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)
• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims [59.64274607533249]
  AI developers need to make verifiable claims to which they can be held accountable. This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems. We analyze ten mechanisms for this purpose--spanning institutions, software, and hardware--and make recommendations aimed at implementing, exploring, or improving those mechanisms.
  arXiv  Detail & Related papers  (2020-04-15T17:15:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.