Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives

• URL: http://arxiv.org/abs/2311.09227v1
• Date: Fri, 29 Sep 2023 17:03:45 GMT
• Title: Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives
• Authors: Elizabeth Seger, Noemi Dreksler, Richard Moulange, Emily Dardaman, Jonas Schuett, K. Wei, Christoph Winter, Mackenzie Arnold, Se\'an \'O h\'Eigeartaigh, Anton Korinek, Markus Anderljung, Ben Bucknall, Alan Chan, Eoghan Stafford, Leonie Koessler, Aviv Ovadya, Ben Garfinkel, Emma Bluemke, Michael Aird, Patrick Levermore, Julian Hazell, Abhishek Gupta
• Abstract summary: Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate. This paper offers an examination of the risks and benefits of open-sourcing highly capable foundation models.
• Score: 6.575445633821399
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI models should be shared. Open-sourcing in AI typically refers to making model architecture and weights freely and publicly accessible for anyone to modify, study, build on, and use. This offers advantages such as enabling external oversight, accelerating progress, and decentralizing control over AI development and use. However, it also presents a growing potential for misuse and unintended consequences. This paper offers an examination of the risks and benefits of open-sourcing highly capable foundation models. While open-sourcing has historically provided substantial net benefits for most software and AI development processes, we argue that for some highly capable foundation models likely to be developed in the near future, open-sourcing may pose sufficiently extreme risks to outweigh the benefits. In such a case, highly capable foundation models should not be open-sourced, at least not initially. Alternative strategies, including non-open-source model sharing options, are explored. The paper concludes with recommendations for developers, standard-setting bodies, and governments for establishing safe and responsible model sharing practices and preserving open-source benefits where safe.

Related papers

• PRISM: A Design Framework for Open-Source Foundation Model Safety [0.0]
  This paper addresses the question of how open foundation model developers should approach model safety. We introduce PRISM, a design framework for open-source foundation model safety that emphasizes Private, Robust, Independent Safety measures. PRISM aims to create a safer open-source ecosystem that maximizes the potential of these powerful technologies while minimizing the risks to individuals and society as a whole.
  arXiv  Detail & Related papers  (2024-06-14T21:26:15Z)
• Towards a Framework for Openness in Foundation Models: Proceedings from the Columbia Convening on Openness in Artificial Intelligence [18.130525337375985]
  This paper presents a framework for grappling with openness across the AI stack. It summarizes previous work on this topic, analyzes the various potential reasons to pursue openness. It outlines how openness varies in different parts of the AI stack, both at the model and at the system level.
  arXiv  Detail & Related papers  (2024-05-17T20:35:39Z)
• Risks and Opportunities of Open-Source Generative AI [64.86989162783648]
  Applications of Generative AI (Gen AI) are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about the potential risks of the technology, and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source generative AI.
  arXiv  Detail & Related papers  (2024-05-14T13:37:36Z)
• Near to Mid-term Risks and Opportunities of Open-Source Generative AI [94.06233419171016]
  Applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about potential risks and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source Generative AI.
  arXiv  Detail & Related papers  (2024-04-25T21:14:24Z)
• Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software Learning [23.395624804517034]
  Large Language Models (LLMs) have become instrumental in advancing software engineering (SE) tasks. The collaboration of these AI-based SE models hinges on maximising the sources of high-quality data. Data especially of high quality, often holds commercial or sensitive value, making it less accessible for open-source AI-based SE projects.
  arXiv  Detail & Related papers  (2024-04-09T10:47:02Z)
• The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligence [0.0]
  We propose the Model Openness Framework (MOF), a ranked classification system that rates machine learning models based on their completeness and openness. This framework aims to prevent misrepresentation of models claiming to be open, guide researchers and developers in providing all model components under permissive licenses, and help individuals and organizations identify models that can be safely adopted without restrictions.
  arXiv  Detail & Related papers  (2024-03-20T17:47:08Z)
• On the Societal Impact of Open Foundation Models [93.67389739906561]
  We focus on open foundation models, defined here as those with broadly available model weights. We identify five distinctive properties of open foundation models that lead to both their benefits and risks.
  arXiv  Detail & Related papers  (2024-02-27T16:49:53Z)
• Computing Power and the Governance of Artificial Intelligence [51.967584623262674]
  Governments and companies have started to leverage compute as a means to govern AI. compute-based policies and technologies have the potential to assist in these areas, but there is significant variation in their readiness for implementation. naive or poorly scoped approaches to compute governance carry significant risks in areas like privacy, economic impacts, and centralization of power.
  arXiv  Detail & Related papers  (2024-02-13T21:10:21Z)
• Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models [103.71308117592963]
  We present an algorithm for training self-destructing models leveraging techniques from meta-learning and adversarial learning. In a small-scale experiment, we show MLAC can largely prevent a BERT-style model from being re-purposed to perform gender identification.
  arXiv  Detail & Related papers  (2022-11-27T21:43:45Z)
• Towards Inheritable Models for Open-Set Domain Adaptation [56.930641754944915]
  We introduce a practical Domain Adaptation paradigm where a source-trained model is used to facilitate adaptation in the absence of the source dataset in future. We present an objective way to quantify inheritability to enable the selection of the most suitable source model for a given target domain, even in the absence of the source data.
  arXiv  Detail & Related papers  (2020-04-09T07:16:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.