Automatic Model Card Generation Using an LLM

• URL: http://arxiv.org/abs/2608.24807v1
• Date: Tue, 25 Aug 2026 16:49:37 GMT
• Title: Automatic Model Card Generation Using an LLM
• Abstract summary: Model cards are structured documents that summarize key information about machine learning models.<n>Many models provide no model cards, making comparison and interpretation difficult.<n>We propose MCTidy, an LLM-based approach that reorganizes existing model cards into a standardized template.<n>We also introduce MCGenie, an LLM-based system that generates model cards directly from model repository data.
• Score: 6.101895668857863
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Model cards are structured documents that summarize key information about machine learning models to improve transparency, usability, and accountability. However, they often lack a consistent structure, and many models provide no model cards, making comparison and interpretation difficult. This paper presents two contributions. First, we propose MCTidy, an LLM-based approach that reorganizes existing model cards into a standardized template to improve clarity and comparability. Second, we introduce MCGenie, an LLM-based system that generates model cards directly from model repository data. We apply MCTidy to 48 Hugging Face model cards and evaluate information retention, section alignment, hallucination, and stability. Our findings show high information retention with minimal textual loss, accurate section assignment, rare hallucinations primarily in descriptive sections, and strong stability across runs. We assess MCGenie by generating model cards for the same 48 models and assessing semantic similarity, factual correctness, and sensitivity to input resources. The generated model cards achieved high semantic similarity (mean around 0.9); over half were fully correct, and most remaining errors were minor. Generation quality depended strongly on the availability of supporting resources, particularly associated papers. Overall, our findings demonstrate the potential of LLM-based methods to enable scalable, standardized model card documentation.

Related papers

• Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit [16.056849135589324]
  Analyzing large-scale text corpora is a core challenge in machine learning.<n>We propose using sparse autoencoders (SAEs) to create SAE embeddings.<n>We show that SAE embeddings are more cost-effective and reliable than LLMs and more controllable than dense embeddings.
  arXiv  Detail & Related papers  (2025-12-10T21:26:24Z)
• Can LLM Annotations Replace User Clicks for Learning to Rank? [112.2254432364736]
  Large-scale supervised data is essential for training modern ranking models, but obtaining high-quality human annotations is costly.<n>Click data has been widely used as a low-cost alternative, and with recent advances in large language models (LLMs), LLM-based relevance annotation has emerged as another promising annotation.<n> Experiments on both a public dataset, TianGong-ST, and an industrial dataset, Baidu-Click, show that click-supervised models perform better on high-frequency queries.<n>We explore two training strategies -- data scheduling and frequency-aware multi-objective learning -- that integrate both supervision signals.
  arXiv  Detail & Related papers  (2025-11-10T02:26:14Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence [61.46575527504109]
  LimiX-16M and LimiX-2M treat structured data as a joint distribution over variables and missingness.<n>We evaluate LimiX models across 11 large structured-data benchmarks with broad regimes of sample size, feature dimensionality, class number, categorical-to-numerical feature ratio, missingness, and sample-to-feature ratios.
  arXiv  Detail & Related papers  (2025-09-03T17:39:08Z)
• Predicting the Performance of Black-box LLMs through Self-Queries [60.87193950962585]
  Large language models (LLMs) are increasingly relied on in AI systems, predicting when they make mistakes is crucial.<n>In this paper, we extract features of LLMs in a black-box manner by using follow-up prompts and taking the probabilities of different responses as representations.<n>We demonstrate that training a linear model on these low-dimensional representations produces reliable predictors of model performance at the instance level.
  arXiv  Detail & Related papers  (2025-01-02T22:26:54Z)
• MAPX: An explainable model-agnostic framework for the detection of false information on social media networks [1.5196326555431678]
  We introduce a novel model-agnostic framework, called MAPX, which allows evidence based aggregation of predictions. We perform extensive experiments on benchmarked fake news datasets to demonstrate the effectiveness of MAPX. Our empirical results show that the proposed framework consistently outperforms all state-of-the-art models evaluated.
  arXiv  Detail & Related papers  (2024-09-13T03:45:10Z)
• What's documented in AI? Systematic Analysis of 32K AI Model Cards [40.170354637778345]
  We conduct a comprehensive analysis of 32,111 AI model documentations on Hugging Face. Most of the AI models with substantial downloads provide model cards, though the cards have uneven informativeness. We find that sections addressing environmental impact, limitations, and evaluation exhibit the lowest filled-out rates, while the training section is the most consistently filled-out.
  arXiv  Detail & Related papers  (2024-02-07T18:04:32Z)
• Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning [79.53130089003986]
  Large Language Models (LLMs) have become a feasible solution for handling tasks in various domains. In this paper, we introduce how to fine-tune a LLM model that can be privately deployed for content moderation.
  arXiv  Detail & Related papers  (2023-10-05T09:09:44Z)
• Unlocking Model Insights: A Dataset for Automated Model Card Generation [4.167070553534516]
  We introduce a dataset of 500 question-answer pairs for 25 ML models. We employ annotators to extract the answers from the original paper. Our experiments with ChatGPT-3.5, LLaMa, and Galactica showcase a significant gap in the understanding of research papers by these LMs.
  arXiv  Detail & Related papers  (2023-09-22T04:46:11Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing [72.14557106085284]
  slice detection models (SDM) automatically identify underperforming groups of datapoints. This paper proposes a benchmark named "Discover, Explain, improve (DEIM)" for classification NLP tasks. Our evaluation shows that Edisa can accurately select error-prone datapoints with informative semantic features.
  arXiv  Detail & Related papers  (2022-11-08T19:00:00Z)
• Evaluation of HTR models without Ground Truth Material [2.4792948967354236]
  evaluation of Handwritten Text Recognition models during their development is straightforward. But the evaluation process becomes tricky as soon as we switch from development to application. We show that lexicon-based evaluation can compete with lexicon-based methods.
  arXiv  Detail & Related papers  (2022-01-17T01:26:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.