Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity

• URL: http://arxiv.org/abs/2006.07495v1
• Date: Fri, 12 Jun 2020 22:28:09 GMT
• Title: Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity
• Authors: Adrien Ecoffet and Jeff Clune and Joel Lehman
• Abstract summary: open-ended evolution and artificial life have much to contribute towards the understanding of open-ended AI. This paper argues that open-ended AI has its own safety challenges, whether the creativity of open-ended systems can be productively and predictably controlled.
• Score: 15.60659580411643
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial life originated and has long studied the topic of open-ended evolution, which seeks the principles underlying artificial systems that innovate continually, inspired by biological evolution. Recently, interest has grown within the broader field of AI in a generalization of open-ended evolution, here called open-ended search, wherein such questions of open-endedness are explored for advancing AI, whatever the nature of the underlying search algorithm (e.g. evolutionary or gradient-based). For example, open-ended search might design new architectures for neural networks, new reinforcement learning algorithms, or most ambitiously, aim at designing artificial general intelligence. This paper proposes that open-ended evolution and artificial life have much to contribute towards the understanding of open-ended AI, focusing here in particular on the safety of open-ended search. The idea is that AI systems are increasingly applied in the real world, often producing unintended harms in the process, which motivates the growing field of AI safety. This paper argues that open-ended AI has its own safety challenges, in particular, whether the creativity of open-ended systems can be productively and predictably controlled. This paper explains how unique safety problems manifest in open-ended search, and suggests concrete contributions and research questions to explore them. The hope is to inspire progress towards creative, useful, and safe open-ended search algorithms.

Related papers

• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We argue that shortcomings stem from one overarching failure: AI systems lack wisdom. While AI research has focused on task-level strategies, metacognition is underdeveloped in AI systems. We propose that integrating metacognitive capabilities into AI systems is crucial for enhancing their robustness, explainability, cooperation, and safety.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Open-Endedness is Essential for Artificial Superhuman Intelligence [19.381655909809776]
  We argue that the ingredients are now in place to achieve openendedness in AI systems with respect to a human observer. We conclude by examining the safety implications of generally-capable openended AI.
  arXiv  Detail & Related papers  (2024-06-06T17:15:02Z)
• Near to Mid-term Risks and Opportunities of Open-Source Generative AI [94.06233419171016]
  Applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about potential risks and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source Generative AI.
  arXiv  Detail & Related papers  (2024-04-25T21:14:24Z)
• Open-world Machine Learning: A Review and New Outlooks [83.6401132743407]
  This paper aims to provide a comprehensive introduction to the emerging open-world machine learning paradigm. It aims to help researchers build more powerful AI systems in their respective fields, and to promote the development of artificial general intelligence.
  arXiv  Detail & Related papers  (2024-03-04T06:25:26Z)
• A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimization [55.11642177631929]
  Large neural generative models are capable of synthesizing semantically rich passages of text or producing complex images. We discuss the COGnitive Neural GENerative system, such an architecture that casts the Common Model of Cognition.
  arXiv  Detail & Related papers  (2023-10-14T23:28:48Z)
• The Future of Fundamental Science Led by Generative Closed-Loop Artificial Intelligence [67.70415658080121]
  Recent advances in machine learning and AI are disrupting technological innovation, product development, and society as a whole. AI has contributed less to fundamental science in part because large data sets of high-quality data for scientific practice and model discovery are more difficult to access. Here we explore and investigate aspects of an AI-driven, automated, closed-loop approach to scientific discovery.
  arXiv  Detail & Related papers  (2023-07-09T21:16:56Z)
• A Survey on Neural Open Information Extraction: Current Status and Future Directions [87.30702606041407]
  Open Information Extraction (OpenIE) facilitates domain-independent discovery of relational facts from large corpora. We provide an overview of the-state-of-the-art neural OpenIE models, their key design decisions, strengths and weakness.
  arXiv  Detail & Related papers  (2022-05-24T02:24:55Z)
• An Initial Look at Self-Reprogramming Artificial Intelligence [0.0]
  We develop and experimentally validate the first fully self-reprogramming AI system. Applying AI-based computer code generation to AI itself, we implement an algorithm with the ability to continuously modify and rewrite its own neural network source code.
  arXiv  Detail & Related papers  (2022-04-30T05:44:34Z)
• Creative AI Through Evolutionary Computation: Principles and Examples [16.8615211682877]
  Population-based search techniques make it possible to find creative solutions to practical problems in the real world. evolutionary computation is the likely "next deep learning"
  arXiv  Detail & Related papers  (2020-08-04T19:53:39Z)
• Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions [20.671903144896742]
  Paired Open-Ended Trailblazer (POET) is an algorithm that generates and solves its own challenges. POET was unable to demonstrate its full creative potential because of limitations of the algorithm itself. We introduce and empirically validate two new innovations to the original algorithm, as well as two external innovations designed to help elucidate its full potential.
  arXiv  Detail & Related papers  (2020-03-19T01:35:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.