Prioritization of Risks from Artificial Intelligence: A Delphi Study of 272 International Experts

• URL: http://arxiv.org/abs/2606.04490v1
• Date: Wed, 03 Jun 2026 06:14:41 GMT
• Title: Prioritization of Risks from Artificial Intelligence: A Delphi Study of 272 International Experts
• Abstract summary: We report results from a three-round Delphi study conducted late 2025 with 272 international AI experts.<n>Experts rated 24 AI risks on harm probability and severity, sector and actor vulnerability, actor responsibility, and overall concern.
• Score: 115.34779081791287
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence poses many risks, ranging from familiar present-day harms to unprecedented and potentially catastrophic ones. Effective risk management requires prioritization: we must understand which risks are most severe, who is most vulnerable, and who is most responsible for addressing them. We report results from a three-round Delphi study conducted late 2025 with 272 international AI experts. Experts rated 24 AI risks on harm probability and severity, sector and actor vulnerability, actor responsibility, and overall concern. Experts estimated the five most severe harms in the next 5 years were likely to come from dangerous capabilities, competitive dynamics, weapons & cyberattacks (including CBRNE), power centralization, and false information. In a business-as-usual scenario, experts judged 18 of 24 risks as having a more than 10% probability of catastrophic outcomes (e.g., more than 1 million deaths or more than USD 100B in financial loss) in the next 5 years (2025-2030). In a scenario where pragmatic mitigations are implemented, experts still judged five risks as having a more than 10% probability of catastrophic outcomes: dangerous capabilities, weapons & cyberattacks, environmental harm, inequality & unemployment, and power centralization. All 24 risks were judged as being more than 5% likely to cause catastrophic outcomes. AI users and the general public were judged the most vulnerable to these risks, but experts assigned the highest responsibility for addressing them to general-purpose AI developers and governance actors (including governments, regulators, and standards bodies). Across most risks, experts identified information, finance, and national security as the most vulnerable sectors. These findings can guide AI risk prioritization and clarify expert expectations about who should bear responsibility for mitigation.

Related papers

• Mapping U.S. Federal AI Governance Against Sector Vulnerability [33.757821744179346]
  We assess 684 federal AI governance documents for their coverage of 14 sectors and 24 AI risks.<n>We compare sector coverage patterns for each of the 24 risks with vulnerability assessments from a Delphi study of 272 experts.
  arXiv  Detail & Related papers  (2026-09-14T19:23:46Z)
• "We are not Future-ready": Understanding AI Privacy Risks and Existing Mitigation Strategies from the Perspective of AI Developers in Europe [56.1653658714305]
  We interviewed 25 AI developers based in Europe to understand which privacy threats they believe pose the greatest risk to users, developers, and businesses.<n>We find that there is little consensus among AI developers on the relative ranking of privacy risks.<n>While AI developers are aware of proposed mitigation strategies for addressing these risks, they reported minimal real-world adoption.
  arXiv  Detail & Related papers  (2025-10-01T13:51:33Z)
• Are Companies Taking AI Risks Seriously? A Systematic Analysis of Companies' AI Risk Disclosures in SEC 10-K forms [9.295655764985318]
  Regulators are pushing for greater transparency in how companies identify, report and mitigate AI-related risks.<n>In the US, the Securities and Exchange Commission repeatedly warned companies to provide their investors with more accurate disclosures of AI-related risks.<n>In the EU, new laws - like the AI Act and Digital Services Act - introduced additional rules on AI risk reporting and mitigation.<n>This study presents the first large-scale systematic analysis of AI risk disclosures in SEC 10-K filings.
  arXiv  Detail & Related papers  (2025-08-26T10:19:29Z)
• Intolerable Risk Threshold Recommendations for Artificial Intelligence [0.2383122657918106]
  Frontier AI models may pose severe risks to public safety, human rights, economic stability, and societal value.<n>Risks could arise from deliberate adversarial misuse, system failures, unintended cascading effects, or simultaneous failures across multiple models.<n>16 global AI industry organizations signed the Frontier AI Safety Commitments, and 27 nations and the EU issued a declaration on their intent to define these thresholds.
  arXiv  Detail & Related papers  (2025-03-04T12:30:37Z)
• Fully Autonomous AI Agents Should Not be Developed [50.61667544399082]
  This paper argues that fully autonomous AI agents should not be developed.<n>In support of this position, we build from prior scientific literature and current product marketing to delineate different AI agent levels.<n>Our analysis reveals that risks to people increase with the autonomy of a system.
  arXiv  Detail & Related papers  (2025-02-04T19:00:06Z)
• Perception Gaps in Risk, Benefit, and Value Between Experts and Public Challenge Socially Accepted AI [0.0]
  This study examines how the general public and academic AI experts perceive AI's capabilities and impact.<n>The scenarios span domains such as sustainability, healthcare, job performance, societal inequality, art, and warfare.<n>Experts consistently anticipate higher probabilities, perceive lower risks, report greater benefits, and express more positive sentiment toward AI compared to the non-experts.
  arXiv  Detail & Related papers  (2024-12-02T12:51:45Z)
• Engineering Trustworthy AI: A Developer Guide for Empirical Risk Minimization [53.80919781981027]
  Key requirements for trustworthy AI can be translated into design choices for the components of empirical risk minimization. We hope to provide actionable guidance for building AI systems that meet emerging standards for trustworthiness of AI.
  arXiv  Detail & Related papers  (2024-10-25T07:53:32Z)
• Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy [65.77763092833348]
  This perspective examines vulnerabilities in AI scientists, shedding light on potential risks associated with their misuse.<n>We take into account user intent, the specific scientific domain, and their potential impact on the external environment.<n>We propose a triadic framework involving human regulation, agent alignment, and an understanding of environmental feedback.
  arXiv  Detail & Related papers  (2024-02-06T18:54:07Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• An Overview of Catastrophic AI Risks [38.84933208563934]
  This paper provides an overview of the main sources of catastrophic AI risks, which we organize into four categories. Malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs. organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents. rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans.
  arXiv  Detail & Related papers  (2023-06-21T03:35:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.