Modelpedia: A Catalog of Model Findings for the Meta-Science of AI

• URL: http://arxiv.org/abs/2609.01090v2
• Date: Thu, 03 Sep 2026 07:30:10 GMT
• Title: Modelpedia: A Catalog of Model Findings for the Meta-Science of AI
• Abstract summary: Modelpedia is an automated framework that extracts findings about models from published papers, links it to the model, dataset, method, and concept it concerns, and aggregates the result into a searchable public catalog.<n>Applying the prototype to accepted ICLR 2024 and 2025 papers, we extract over a thousand findings and, treating the catalog itself as an object of study, run a meta-analysis of how the community investigates models.
• Score: 16.336234329655372
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scientific knowledge about AI models is produced faster than the community can organize it. Every few months a new foundation model reshapes the field and hundreds of papers, blogs, and technical reports document how each behaves or fails. Yet, these findings remain scattered and effectively unretrievable. To address this gap we present Modelpedia, an automated, LLM-assisted framework that extracts findings about models from published papers, links it to the model, dataset, method, and concept it concerns, and aggregates the result into a searchable public catalog. Applying the prototype to accepted ICLR 2024 and 2025 papers, we extract over a thousand findings and, treating the catalog itself as an object of study, run a meta-analysis of how the community investigates models. Now, we invite the community to explore, contribute to, and build on the open catalog, and to help establish model findings as a shared foundation for the meta-science of AI.

Related papers

• Proceedings of the 7th Workshop on Models for Formal Analysis of Real Systems [2.277447144331876]
  Papers presented at the 7th Workshop on Models for Formal Analysis of Real Systems (MARS 2026)<n>MARS 2026 took place in Turin, Italy, on 12 April 2026.
  arXiv  Detail & Related papers  (2026-04-03T14:06:16Z)
• Affordances Enable Partial World Modeling with LLMs [68.52975612311575]
  We show that agents achieving task-agnostic, language-conditioned intents possess predictive partial-world models informed by affordances.<n>In the multi-task setting, we introduce distribution-robust affordances and show that partial models can be extracted to significantly improve search efficiency.
  arXiv  Detail & Related papers  (2026-02-11T00:25:25Z)
• SciArena: An Open Evaluation Platform for Foundation Models in Scientific Literature Tasks [87.29946641069068]
  We present SciArena, an open and collaborative platform for evaluating foundation models on scientific literature tasks.<n>By leveraging collective intelligence, SciArena offers a community-driven evaluation of model performance on open-ended scientific tasks.<n>We release SciArena-Eval, a meta-evaluation benchmark based on our collected preference data.
  arXiv  Detail & Related papers  (2025-07-01T17:51:59Z)
• Predicting New Research Directions in Materials Science using Large Language Models and Concept Graphs [30.813288388998256]
  We show that large language models (LLMs) can extract concepts more efficiently than automated keyword extraction methods.<n>A machine learning model is trained to predict emerging combinations of concepts, based on historical data.<n>We show that the model can inspire materials scientists in their creative thinking process by predicting innovative combinations of topics that have not yet been investigated.
  arXiv  Detail & Related papers  (2025-06-20T08:26:12Z)
• Unlocking the Potential of Past Research: Using Generative AI to Reconstruct Healthcare Simulation Models [0.0]
  This study explores the feasibility of using generative artificial intelligence (AI) to recreate published models using Free and Open Source Software (FOSS)<n>We successfully generated, tested and internally reproduced two DES models, including user interfaces.<n>The reported results were replicated for one model, but not the other, likely due to missing information on distributions.
  arXiv  Detail & Related papers  (2025-03-27T16:10:02Z)
• Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions [62.12545440385489]
  Large language models (LLMs) have brought substantial advancements in text generation, but their potential for enhancing classification tasks remains underexplored. We propose a framework for thoroughly investigating fine-tuning LLMs for classification, including both generation- and encoding-based approaches. We instantiate this framework in edit intent classification (EIC), a challenging and underexplored classification task.
  arXiv  Detail & Related papers  (2024-10-02T20:48:28Z)
• Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities [89.40778301238642]
  Model merging is an efficient empowerment technique in the machine learning community. There is a significant gap in the literature regarding a systematic and thorough review of these techniques.
  arXiv  Detail & Related papers  (2024-08-14T16:58:48Z)
• A Metascience Study of the Low-Code Scientific Field [2.031841135743809]
  "Low-code" has taken the modeling field by storm, largely due to its association with several highly popular development platforms.<n>We present a metascience study of Low-Code.<n>We hope to trigger a discussion on the current state and potential future trajectory of the low-code community.
  arXiv  Detail & Related papers  (2024-08-12T08:03:01Z)
• Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning [79.53130089003986]
  Large Language Models (LLMs) have become a feasible solution for handling tasks in various domains. In this paper, we introduce how to fine-tune a LLM model that can be privately deployed for content moderation.
  arXiv  Detail & Related papers  (2023-10-05T09:09:44Z)
• Replicability Study: Corpora For Understanding Simulink Models & Projects [8.261117235807607]
  The study reviews methodologies and data sources employed in prior Simulink model studies and replicates the previous analysis using SLNET. We found that open-source Simulink models follow good modeling practices and contain models comparable in size and properties to proprietary models.
  arXiv  Detail & Related papers  (2023-08-03T18:14:54Z)
• Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks [53.09649785009528]
  In this paper, we explore a paradigm that does not require training to obtain new models. Similar to the birth of CNN inspired by receptive fields in the biological visual system, we propose Model Disassembling and Assembling. For model assembling, we present the alignment padding strategy and parameter scaling strategy to construct a new model tailored for a specific task.
  arXiv  Detail & Related papers  (2022-03-25T05:27:28Z)
• Method and Dataset Entity Mining in Scientific Literature: A CNN + Bi-LSTM Model with Self-attention [21.93889297841459]
  We propose a novel entity recognition model, called MDER, which is able to effectively extract the method and dataset entities from scientific papers. We evaluate the proposed model on datasets constructed from the published papers of four research areas in computer science, i.e., NLP, CV, Data Mining and AI.
  arXiv  Detail & Related papers  (2020-10-26T13:38:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.