Generative AI and the problem of existential risk

• URL: http://arxiv.org/abs/2407.13365v1
• Date: Thu, 18 Jul 2024 10:16:24 GMT
• Title: Generative AI and the problem of existential risk
• Authors: Lynette Webb, Daniel Schönberger,
• Abstract summary: Generative AI has been a focal point for concerns about AI's perceived existential risk. This chapter aims to demystify the debate by highlighting the key worries that underpin existential risk fears in relation to generative AI.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Ever since the launch of ChatGPT, Generative AI has been a focal point for concerns about AI's perceived existential risk. Once a niche topic in AI research and philosophy, AI safety and existential risk has now entered mainstream debate among policy makers and leading foundation models developers, much to the chagrin of those who see it as a distraction from addressing more pressing nearer-term harms. This chapter aims to demystify the debate by highlighting the key worries that underpin existential risk fears in relation to generative AI, and spotlighting the key actions that governments and industry are taking thus far to helping address them.

Related papers

• AI Safety for Everyone [3.440579243843689]
  Recent discussions and research in AI safety have increasingly emphasized the deep connection between AI safety and existential risk from advanced AI systems. This framing may exclude researchers and practitioners who are committed to AI safety but approach the field from different angles. We find a vast array of concrete safety work that addresses immediate and practical concerns with current AI systems.
  arXiv  Detail & Related papers  (2025-02-13T13:04:59Z)
• Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI [52.138044013005]
  generative AI, particularly large language models (LLMs), become increasingly integrated into production applications. New attack surfaces and vulnerabilities emerge and put a focus on adversarial threats in natural language and multi-modal systems. Red-teaming has gained importance in proactively identifying weaknesses in these systems, while blue-teaming works to protect against such adversarial attacks. This work aims to bridge the gap between academic insights and practical security measures for the protection of generative AI systems.
  arXiv  Detail & Related papers  (2024-09-23T10:18:10Z)
• Mapping the individual, social, and biospheric impacts of Foundation Models [0.39843531413098965]
  This paper offers a critical framework to account for the social, political, and environmental dimensions of foundation models and generative AI. We identify 14 categories of risks and harms and map them according to their individual, social, and biospheric impacts.
  arXiv  Detail & Related papers  (2024-07-24T10:05:40Z)
• Implications for Governance in Public Perceptions of Societal-scale AI Risks [0.29022435221103454]
  Voters perceive AI risks as both more likely and more impactful than experts, and also advocate for slower AI development. Policy interventions may best assuage collective concerns if they attempt to more carefully balance mitigation efforts across all classes of societal-scale risks.
  arXiv  Detail & Related papers  (2024-06-10T11:52:25Z)
• AI Risk Management Should Incorporate Both Safety and Security [185.68738503122114]
  We argue that stakeholders in AI risk management should be aware of the nuances, synergies, and interplay between safety and security. We introduce a unified reference framework to clarify the differences and interplay between AI safety and AI security.
  arXiv  Detail & Related papers  (2024-05-29T21:00:47Z)
• Risks and Opportunities of Open-Source Generative AI [64.86989162783648]
  Applications of Generative AI (Gen AI) are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about the potential risks of the technology, and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source generative AI.
  arXiv  Detail & Related papers  (2024-05-14T13:37:36Z)
• Near to Mid-term Risks and Opportunities of Open-Source Generative AI [94.06233419171016]
  Applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about potential risks and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source Generative AI.
  arXiv  Detail & Related papers  (2024-04-25T21:14:24Z)
• Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits [54.648819983899614]
  General purpose AI seems to have lowered the barriers for the public to use AI and harness its power. We introduce PARTICIP-AI, a framework for laypeople to speculate and assess AI use cases and their impacts.
  arXiv  Detail & Related papers  (2024-03-21T19:12:37Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• An Overview of Catastrophic AI Risks [38.84933208563934]
  This paper provides an overview of the main sources of catastrophic AI risks, which we organize into four categories. Malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs. organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents. rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans.
  arXiv  Detail & Related papers  (2023-06-21T03:35:06Z)
• Current and Near-Term AI as a Potential Existential Risk Factor [5.1806669555925975]
  We problematise the notion that current and near-term artificial intelligence technologies have the potential to contribute to existential risk. We propose the hypothesis that certain already-documented effects of AI can act as existential risk factors. Our main contribution is an exposition of potential AI risk factors and the causal relationships between them.
  arXiv  Detail & Related papers  (2022-09-21T18:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.