AI Research Considerations for Human Existential Safety (ARCHES)

• URL: http://arxiv.org/abs/2006.04948v1
• Date: Sat, 30 May 2020 02:05:16 GMT
• Title: AI Research Considerations for Human Existential Safety (ARCHES)
• Authors: Andrew Critch, David Krueger
• Abstract summary: In negative terms, we ask what existential risks humanity might face from AI development in the next century. Key property of hypothetical AI technologies is introduced, called emphprepotence A set of auxrefdirtot contemporary research directions are then examined for their potential benefit to existential safety.
• Score: 6.40842967242078
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Framed in positive terms, this report examines how technical AI research might be steered in a manner that is more attentive to humanity's long-term prospects for survival as a species. In negative terms, we ask what existential risks humanity might face from AI development in the next century, and by what principles contemporary technical research might be directed to address those risks. A key property of hypothetical AI technologies is introduced, called \emph{prepotence}, which is useful for delineating a variety of potential existential risks from artificial intelligence, even as AI paradigms might shift. A set of \auxref{dirtot} contemporary research \directions are then examined for their potential benefit to existential safety. Each research direction is explained with a scenario-driven motivation, and examples of existing work from which to build. The research directions present their own risks and benefits to society that could occur at various scales of impact, and in particular are not guaranteed to benefit existential safety if major developments in them are deployed without adequate forethought and oversight. As such, each direction is accompanied by a consideration of potentially negative side effects.

Related papers

• A Trilogy of AI Safety Frameworks: Paths from Facts and Knowledge Gaps to Reliable Predictions and New Knowledge [0.0]
  AI Safety has become a vital front-line concern of many scientists within and outside the AI community. There are many immediate and long term anticipated risks that range from existential risk to human existence to deep fakes and bias in machine learning systems.
  arXiv  Detail & Related papers  (2024-10-09T14:43:06Z)
• Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? [59.96471873997733]
  We propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context. We aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
  arXiv  Detail & Related papers  (2024-07-31T17:59:24Z)
• Now, Later, and Lasting: Ten Priorities for AI Research, Policy, and Practice [63.20307830884542]
  Next several decades may well be a turning point for humanity, comparable to the industrial revolution. Launched a decade ago, the project is committed to a perpetual series of studies by multidisciplinary experts. We offer ten recommendations for action that collectively address both the short- and long-term potential impacts of AI technologies.
  arXiv  Detail & Related papers  (2024-04-06T22:18:31Z)
• Artificial Intelligence: Arguments for Catastrophic Risk [0.0]
  We review two influential arguments purporting to show how AI could pose catastrophic risks. The first argument -- the Problem of Power-Seeking -- claims that advanced AI systems are likely to engage in dangerous power-seeking behavior. The second argument claims that the development of human-level AI will unlock rapid further progress.
  arXiv  Detail & Related papers  (2024-01-27T19:34:13Z)
• Control Risk for Potential Misuse of Artificial Intelligence in Science [85.91232985405554]
  We aim to raise awareness of the dangers of AI misuse in science. We highlight real-world examples of misuse in chemical science. We propose a system called SciGuard to control misuse risks for AI models in science.
  arXiv  Detail & Related papers  (2023-12-11T18:50:57Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Predictable Artificial Intelligence [77.1127726638209]
  This paper introduces the ideas and challenges of Predictable AI. It explores the ways in which we can anticipate key validity indicators of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
  arXiv  Detail & Related papers  (2023-10-09T21:36:21Z)
• An Overview of Catastrophic AI Risks [38.84933208563934]
  This paper provides an overview of the main sources of catastrophic AI risks, which we organize into four categories. Malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs. organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents. rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans.
  arXiv  Detail & Related papers  (2023-06-21T03:35:06Z)
• Current and Near-Term AI as a Potential Existential Risk Factor [5.1806669555925975]
  We problematise the notion that current and near-term artificial intelligence technologies have the potential to contribute to existential risk. We propose the hypothesis that certain already-documented effects of AI can act as existential risk factors. Our main contribution is an exposition of potential AI risk factors and the causal relationships between them.
  arXiv  Detail & Related papers  (2022-09-21T18:56:14Z)
• Metaethical Perspectives on 'Benchmarking' AI Ethics [81.65697003067841]
  Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. We argue that it makes more sense to talk about 'values' rather than 'ethics' when considering the possible actions of present and future AI systems.
  arXiv  Detail & Related papers  (2022-04-11T14:36:39Z)
• Transdisciplinary AI Observatory -- Retrospective Analyses and Future-Oriented Contradistinctions [22.968817032490996]
  This paper motivates the need for an inherently transdisciplinary AI observatory approach. Building on these AI observatory tools, we present near-term transdisciplinary guidelines for AI safety.
  arXiv  Detail & Related papers  (2020-11-26T16:01:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.