Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives

• URL: http://arxiv.org/abs/2311.09227v1
• Date: Fri, 29 Sep 2023 17:03:45 GMT
• Title: Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives
• Authors: Elizabeth Seger, Noemi Dreksler, Richard Moulange, Emily Dardaman, Jonas Schuett, K. Wei, Christoph Winter, Mackenzie Arnold, Se\'an \'O h\'Eigeartaigh, Anton Korinek, Markus Anderljung, Ben Bucknall, Alan Chan, Eoghan Stafford, Leonie Koessler, Aviv Ovadya, Ben Garfinkel, Emma Bluemke, Michael Aird, Patrick Levermore, Julian Hazell, Abhishek Gupta
• Abstract summary: Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate. This paper offers an examination of the risks and benefits of open-sourcing highly capable foundation models.
• Score: 6.575445633821399
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI models should be shared. Open-sourcing in AI typically refers to making model architecture and weights freely and publicly accessible for anyone to modify, study, build on, and use. This offers advantages such as enabling external oversight, accelerating progress, and decentralizing control over AI development and use. However, it also presents a growing potential for misuse and unintended consequences. This paper offers an examination of the risks and benefits of open-sourcing highly capable foundation models. While open-sourcing has historically provided substantial net benefits for most software and AI development processes, we argue that for some highly capable foundation models likely to be developed in the near future, open-sourcing may pose sufficiently extreme risks to outweigh the benefits. In such a case, highly capable foundation models should not be open-sourced, at least not initially. Alternative strategies, including non-open-source model sharing options, are explored. The paper concludes with recommendations for developers, standard-setting bodies, and governments for establishing safe and responsible model sharing practices and preserving open-source benefits where safe.

Related papers

• OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models [61.14336781917986]
  We introduce OpenR, an open-source framework for enhancing the reasoning capabilities of large language models (LLMs) OpenR unifies data acquisition, reinforcement learning training, and non-autoregressive decoding into a cohesive software platform. Our work is the first to provide an open-source framework that explores the core techniques of OpenAI's o1 model with reinforcement learning.
  arXiv  Detail & Related papers  (2024-10-12T23:42:16Z)
• PRISM: A Design Framework for Open-Source Foundation Model Safety [0.0]
  This paper addresses the question of how open foundation model developers should approach model safety. We introduce PRISM, a design framework for open-source foundation model safety that emphasizes Private, Robust, Independent Safety measures. PRISM aims to create a safer open-source ecosystem that maximizes the potential of these powerful technologies while minimizing the risks to individuals and society as a whole.
  arXiv  Detail & Related papers  (2024-06-14T21:26:15Z)
• Towards a Framework for Openness in Foundation Models: Proceedings from the Columbia Convening on Openness in Artificial Intelligence [18.130525337375985]
  This paper presents a framework for grappling with openness across the AI stack. It summarizes previous work on this topic, analyzes the various potential reasons to pursue openness. It outlines how openness varies in different parts of the AI stack, both at the model and at the system level.
  arXiv  Detail & Related papers  (2024-05-17T20:35:39Z)
• Risks and Opportunities of Open-Source Generative AI [64.86989162783648]
  Applications of Generative AI (Gen AI) are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about the potential risks of the technology, and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source generative AI.
  arXiv  Detail & Related papers  (2024-05-14T13:37:36Z)
• Near to Mid-term Risks and Opportunities of Open-Source Generative AI [94.06233419171016]
  Applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about potential risks and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source Generative AI.
  arXiv  Detail & Related papers  (2024-04-25T21:14:24Z)
• Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software Learning [23.395624804517034]
  Large Language Models (LLMs) have become instrumental in advancing software engineering (SE) tasks. The collaboration of these AI-based SE models hinges on maximising the sources of high-quality data. Data especially of high quality, often holds commercial or sensitive value, making it less accessible for open-source AI-based SE projects.
  arXiv  Detail & Related papers  (2024-04-09T10:47:02Z)
• The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligence [0.0]
  We introduce the Model Openness Framework (MOF), a three-tiered ranked classification system that rates machine learning models based on their completeness and openness. For each MOF class, we specify code, data, and documentation components of the model development lifecycle that must be released and under which open licenses. In addition, the Model Openness Tool (MOT) provides a user-friendly reference implementation to evaluate the openness and completeness of models against the MOF classification system.
  arXiv  Detail & Related papers  (2024-03-20T17:47:08Z)
• On the Societal Impact of Open Foundation Models [93.67389739906561]
  We focus on open foundation models, defined here as those with broadly available model weights. We identify five distinctive properties of open foundation models that lead to both their benefits and risks.
  arXiv  Detail & Related papers  (2024-02-27T16:49:53Z)
• Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models [103.71308117592963]
  We present an algorithm for training self-destructing models leveraging techniques from meta-learning and adversarial learning. In a small-scale experiment, we show MLAC can largely prevent a BERT-style model from being re-purposed to perform gender identification.
  arXiv  Detail & Related papers  (2022-11-27T21:43:45Z)
• Towards Inheritable Models for Open-Set Domain Adaptation [56.930641754944915]
  We introduce a practical Domain Adaptation paradigm where a source-trained model is used to facilitate adaptation in the absence of the source dataset in future. We present an objective way to quantify inheritability to enable the selection of the most suitable source model for a given target domain, even in the absence of the source data.
  arXiv  Detail & Related papers  (2020-04-09T07:16:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.