Model Cards Revisited: Bridging the Gap Between Theory and Practice for Ethical AI Requirements

• URL: http://arxiv.org/abs/2507.06014v1
• Date: Tue, 08 Jul 2025 14:19:50 GMT
• Title: Model Cards Revisited: Bridging the Gap Between Theory and Practice for Ethical AI Requirements
• Authors: Tim Puhlfürß, Julia Butzke, Walid Maalej,
• Abstract summary: Model cards are the primary documentation framework for developers of artificial intelligence (AI) models.<n>Recent studies indicate inadequate model documentation practices, suggesting a gap between AI requirements and current practices in model documentation.<n>Our taxonomy serves as a foundation for a revised model card framework that holistically addresses ethical AI requirements.
• Score: 6.99674326582747
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model cards are the primary documentation framework for developers of artificial intelligence (AI) models to communicate critical information to their users. Those users are often developers themselves looking for relevant documentation to ensure that their AI systems comply with the ethical requirements of existing laws, guidelines, and standards. Recent studies indicate inadequate model documentation practices, suggesting a gap between AI requirements and current practices in model documentation. To understand this gap and provide actionable guidance to bridge it, we conducted a thematic analysis of 26 guidelines on ethics and AI, three AI documentation frameworks, three quantitative studies of model cards, and ten actual model cards. We identified a total of 43 ethical requirements relevant to model documentation and organized them into a taxonomy featuring four themes and twelve sub-themes representing ethical principles. Our findings indicate that model developers predominantly emphasize model capabilities and reliability in the documentation while overlooking other ethical aspects, such as explainability, user autonomy, and fairness. This underscores the need for enhanced support in documenting ethical AI considerations. Our taxonomy serves as a foundation for a revised model card framework that holistically addresses ethical AI requirements.

Related papers

• Establishing Knowledge Preference in Language Models [80.70632813935644]
  Language models are known to encode a great amount of factual knowledge through pretraining. Such knowledge might be insufficient to cater to user requests. When answering questions about ongoing events, the model should use recent news articles to update its response. When some facts are edited in the model, the updated facts should override all prior knowledge learned by the model.
  arXiv  Detail & Related papers  (2024-07-17T23:16:11Z)
• Documenting Ethical Considerations in Open Source AI Models [8.517777178514242]
  This study investigates how developers document ethical aspects of open source AI models in practice. After filtering an initial set of 2,347 documents, we identified 265 relevant ones. Six themes emerge, with the three largest ones being model behavioural risks, model use cases, and model risk mitigation.
  arXiv  Detail & Related papers  (2024-06-26T05:02:44Z)
• "Model Cards for Model Reporting" in 2024: Reclassifying Category of Ethical Considerations in Terms of Trustworthiness and Risk Management [0.0]
  In 2019, the paper entitled "Model Cards for Model Reporting" introduced a new tool for documenting model performance. One of the categories detailed in that paper is ethical considerations, which includes the subcategories of data, human life, mitigations, risks and harms, and use cases. We propose to reclassify this category in the original model card due to the recent maturing of the field known as trustworthy AI.
  arXiv  Detail & Related papers  (2024-02-15T14:56:00Z)
• What's documented in AI? Systematic Analysis of 32K AI Model Cards [40.170354637778345]
  We conduct a comprehensive analysis of 32,111 AI model documentations on Hugging Face. Most of the AI models with substantial downloads provide model cards, though the cards have uneven informativeness. We find that sections addressing environmental impact, limitations, and evaluation exhibit the lowest filled-out rates, while the training section is the most consistently filled-out.
  arXiv  Detail & Related papers  (2024-02-07T18:04:32Z)
• Unpacking the Ethical Value Alignment in Big Models [46.560886177083084]
  This paper provides an overview of the risks and challenges associated with big models, surveys existing AI ethics guidelines, and examines the ethical implications arising from the limitations of these models. We introduce a novel conceptual paradigm for aligning the ethical values of big models and discuss promising research directions for alignment criteria, evaluation, and method.
  arXiv  Detail & Related papers  (2023-10-26T16:45:40Z)
• Knowledge-Aware Bayesian Deep Topic Model [50.58975785318575]
  We propose a Bayesian generative model for incorporating prior domain knowledge into hierarchical topic modeling. Our proposed model efficiently integrates the prior knowledge and improves both hierarchical topic discovery and document representation.
  arXiv  Detail & Related papers  (2022-09-20T09:16:05Z)
• Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understanding [88.88844606781987]
  Multi-modal document pre-trained models have proven to be very effective in a variety of visually-rich document understanding (VrDU) tasks.<n>The way they model and exploit the interactions between vision and language on documents has hindered them from better generalization ability and higher accuracy.<n>In this work, we investigate the problem of vision-language joint representation learning for VrDU mainly from the perspective of supervisory signals.
  arXiv  Detail & Related papers  (2022-06-27T09:58:34Z)
• Aspirations and Practice of Model Documentation: Moving the Needle with Nudging and Traceability [8.875661788022637]
  We propose a set of design guidelines that aim to support the documentation practice for machine learning models. A prototype tool named DocML follows those guidelines to support model development in computational notebooks.
  arXiv  Detail & Related papers  (2022-04-13T14:39:18Z)
• Evaluating a Methodology for Increasing AI Transparency: A Case Study [8.265282762929509]
  Given growing concerns about the potential harms of artificial intelligence, societies have begun to demand more transparency about how AI models and systems are created and used. To address these concerns, several efforts have proposed documentation templates containing questions to be answered by model developers. No single template can cover the needs of diverse documentation consumers.
  arXiv  Detail & Related papers  (2022-01-24T20:01:01Z)
• A Simple Approach to Case-Based Reasoning in Knowledge Bases [56.661396189466664]
  We present a surprisingly simple yet accurate approach to reasoning in knowledge graphs (KGs) that requires emphno training, and is reminiscent of case-based reasoning in classical artificial intelligence (AI) Consider the task of finding a target entity given a source entity and a binary relation. Our non-parametric approach derives crisp logical rules for each query by finding multiple textitgraph path patterns that connect similar source entities through the given relation.
  arXiv  Detail & Related papers  (2020-06-25T06:28:09Z)
• A Methodology for Creating AI FactSheets [67.65802440158753]
  This paper describes a methodology for creating the form of AI documentation we call FactSheets. Within each step of the methodology, we describe the issues to consider and the questions to explore. This methodology will accelerate the broader adoption of transparent AI documentation.
  arXiv  Detail & Related papers  (2020-06-24T15:08:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.