A Methodology for Creating AI FactSheets
- URL: http://arxiv.org/abs/2006.13796v2
- Date: Sun, 28 Jun 2020 01:47:46 GMT
- Title: A Methodology for Creating AI FactSheets
- Authors: John Richards, David Piorkowski, Michael Hind, Stephanie Houde,
Aleksandra Mojsilovi\'c
- Abstract summary: This paper describes a methodology for creating the form of AI documentation we call FactSheets.
Within each step of the methodology, we describe the issues to consider and the questions to explore.
This methodology will accelerate the broader adoption of transparent AI documentation.
- Score: 67.65802440158753
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As AI models and services are used in a growing number of highstakes areas, a
consensus is forming around the need for a clearer record of how these models
and services are developed to increase trust. Several proposals for higher
quality and more consistent AI documentation have emerged to address ethical
and legal concerns and general social impacts of such systems. However, there
is little published work on how to create this documentation. This is the first
work to describe a methodology for creating the form of AI documentation we
call FactSheets. We have used this methodology to create useful FactSheets for
nearly two dozen models. This paper describes this methodology and shares the
insights we have gathered. Within each step of the methodology, we describe the
issues to consider and the questions to explore with the relevant people in an
organization who will be creating and consuming the AI facts in a FactSheet.
This methodology will accelerate the broader adoption of transparent AI
documentation.
Related papers
- Documenting Ethical Considerations in Open Source AI Models [8.517777178514242]
This study investigates how developers document ethical aspects of open source AI models in practice.
After filtering an initial set of 2,347 documents, we identified 265 relevant ones.
Six themes emerge, with the three largest ones being model behavioural risks, model use cases, and model risk mitigation.
arXiv Detail & Related papers (2024-06-26T05:02:44Z) - Participatory Approaches in AI Development and Governance: Case Studies [9.824305892501686]
This paper forms the second of a two-part series on the value of a participatory approach to AI development and deployment.
The first paper had crafted a principled, as well as pragmatic, justification for deploying participatory methods in these two exercises.
This paper will test these preliminary conclusions in two sectors, the use of facial recognition technology in the upkeep of law and order and the use of large language models in the healthcare sector.
arXiv Detail & Related papers (2024-06-03T10:10:23Z) - Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning [50.47568731994238]
Key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL)
This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies.
arXiv Detail & Related papers (2023-12-22T17:57:57Z) - Use case cards: a use case reporting framework inspired by the European
AI Act [0.0]
We propose a new framework for the documentation of use cases, that we call "use case cards"
Unlike other documentation methodologies, we focus on the purpose and operational use of an AI system.
The proposed framework is the result of a co-design process involving a relevant team of EU policy experts and scientists.
arXiv Detail & Related papers (2023-06-23T15:47:19Z) - Dynamic Documentation for AI Systems [0.0]
We show the limits of present documentation protocols for AI systems.
We argue for dynamic documentation as a new paradigm for understanding and evaluating AI systems.
arXiv Detail & Related papers (2023-03-20T04:23:07Z) - Evaluating a Methodology for Increasing AI Transparency: A Case Study [8.265282762929509]
Given growing concerns about the potential harms of artificial intelligence, societies have begun to demand more transparency about how AI models and systems are created and used.
To address these concerns, several efforts have proposed documentation templates containing questions to be answered by model developers.
No single template can cover the needs of diverse documentation consumers.
arXiv Detail & Related papers (2022-01-24T20:01:01Z) - AI Explainability 360: Impact and Design [120.95633114160688]
In 2019, we created AI Explainability 360 (Arya et al. 2020), an open source software toolkit featuring ten diverse and state-of-the-art explainability methods.
This paper examines the impact of the toolkit with several case studies, statistics, and community feedback.
The paper also describes the flexible design of the toolkit, examples of its use, and the significant educational material and documentation available to its users.
arXiv Detail & Related papers (2021-09-24T19:17:09Z) - Knowledge-Aware Procedural Text Understanding with Multi-Stage Training [110.93934567725826]
We focus on the task of procedural text understanding, which aims to comprehend such documents and track entities' states and locations during a process.
Two challenges, the difficulty of commonsense reasoning and data insufficiency, still remain unsolved.
We propose a novel KnOwledge-Aware proceduraL text understAnding (KOALA) model, which effectively leverages multiple forms of external knowledge.
arXiv Detail & Related papers (2020-09-28T10:28:40Z) - A general framework for scientifically inspired explanations in AI [76.48625630211943]
We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented.
This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
arXiv Detail & Related papers (2020-03-02T10:32:21Z) - Explaining Relationships Between Scientific Documents [55.23390424044378]
We address the task of explaining relationships between two scientific documents using natural language text.
In this paper we establish a dataset of 622K examples from 154K documents.
arXiv Detail & Related papers (2020-02-02T03:54:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.