The Case for Model Science: Verify, Explore, Steer, Refine

• URL: http://arxiv.org/abs/2606.01189v1
• Date: Sun, 31 May 2026 12:11:47 GMT
• Title: The Case for Model Science: Verify, Explore, Steer, Refine
• Abstract summary: Complex AI models now serve billions of users, yet our understanding of how they work lags far behind our ability to deploy them.<n>We argue that the AI community is now ready to move beyond benchmarking and consolidate scattered efforts in model analysis into a systematic discipline, called Model Science.
• Score: 33.29513243549132
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We argue that the AI community is now ready to move beyond benchmarking and consolidate scattered efforts in model analysis into a systematic discipline, a direction we term Model Science. Complex AI models now serve billions of users, yet our understanding of how they work lags far behind our ability to deploy them. Decades of benchmark-driven research have delivered remarkable progress: extensive leaderboards, a wide range of performance metrics, tracking capability gains across diverse tasks; yet this success has also revealed the limits of benchmarks as they tell us whether models perform but not why they succeed or fail, they miss critical failure modes, such as hallucinations or shortcuts. Precedents from established sciences point the way forward: cognitive science shows that understanding complex systems requires complementary levels of analysis; neuroscience demonstrates that deep study of single cases reveals what population studies miss; medicine teaches that specialised training must develop alongside research practice; and agriculture models how shared infrastructure and principles enable cumulative progress. These lessons inform three foundations for Model Science. First, we propose to consolidate research around four functional perspectives: Verify, Explore, Steer, and Refine that address complementary questions about model behaviour. Second, we discuss the required infrastructure for cumulative knowledge: catalogues of datasets, models and findings. Third, we highlight the need for deep analysis of individual model instances, not just model families, because single cases can reveal what population studies miss.

Related papers

• Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond [209.35045331678043]
  We introduce a "levels x laws" taxonomy organized along two axes.<n>The first defines three capability levels: L1 Predictor, which learns one-step local transition operators; L2 Simulator, which composes them into multi-step, action-conditioned rollouts that respect domain laws; and L3 Evolver, which autonomously revises its own model when predictions fail against new evidence.<n>We synthesize over 400 works and summarize more than 100 representative systems spanning model-based reinforcement learning, video generation, web and GUI agents, multi-agent social simulation, and AI-driven scientific discovery.
  arXiv  Detail & Related papers  (2026-04-24T17:48:47Z)
• Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models [4.049850026698638]
  Many existing scholarly assistants depend on proprietary systems with tens or hundreds of billions of parameters.<n>We design a lightweight retrieval-augmented framework that performs task-aware routing to select specialized retrieval strategies.<n>The framework integrates evidence from full-text scientific papers and structured scholarly metadata, and employs compact instruction-tuned language models to generate responses with citations.
  arXiv  Detail & Related papers  (2026-04-02T12:28:51Z)
• UniG2U-Bench: Do Unified Models Advance Multimodal Understanding? [50.92401586025528]
  Unified multimodal models have recently demonstrated strong generative capabilities, yet whether and when generation improves understanding remains unclear.<n>We introduce UniG2U-Bench, a comprehensive benchmark categorizing generation-to-understanding (G2U) evaluation into 7 regimes and 30 subtasks.
  arXiv  Detail & Related papers  (2026-03-03T18:36:16Z)
• A Survey on Generative Model Unlearning: Fundamentals, Taxonomy, Evaluation, and Future Direction [21.966560704390716]
  We review current research on Generative Model Unlearning (GenMU)<n>We propose a unified analytical framework for categorizing unlearning objectives, methodological strategies, and evaluation metrics.<n>We highlight the potential practical value of unlearning techniques in real-world applications.
  arXiv  Detail & Related papers  (2025-07-26T09:49:57Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage.<n>Models may behave unreliably due to poorly explored failure modes.<n> causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field [0.0]
  This paper offers an analysis of the ability of large models to identify semantic relationships between different research topics.<n>We developed a gold standard based on the IEEE Thesaurus to evaluate the task.<n>Several models have achieved outstanding results, including Mixtral-8x7B, Dolphin-Mistral, and Claude 3-7B.
  arXiv  Detail & Related papers  (2024-12-11T10:11:41Z)
• Learning-based Models for Vulnerability Detection: An Extensive Study [3.1317409221921144]
  We extensively and comprehensively investigate two types of state-of-the-art learning-based approaches. We experimentally demonstrate the priority of sequence-based models and the limited abilities of both graph-based models.
  arXiv  Detail & Related papers  (2024-08-14T13:01:30Z)
• Recognizing Identities From Human Skeletons: A Survey on 3D Skeleton Based Person Re-Identification [60.939250172443586]
  Person re-identification via 3D skeletons is an important emerging research area that attracts increasing attention within the pattern recognition community.<n>We provide a comprehensive review and analysis of recent SRID advances.<n>A thorough evaluation of state-of-the-art SRID methods is conducted over various types of benchmarks and protocols to compare their effectiveness and efficiency.
  arXiv  Detail & Related papers  (2024-01-27T04:52:24Z)
• Generative Models as a Complex Systems Science: How can we make sense of large language model behavior? [75.79305790453654]
  Coaxing out desired behavior from pretrained models, while avoiding undesirable ones, has redefined NLP. We argue for a systematic effort to decompose language model behavior into categories that explain cross-task performance.
  arXiv  Detail & Related papers  (2023-07-31T22:58:41Z)
• On the Opportunities and Risks of Foundation Models [256.61956234436553]
  We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration.
  arXiv  Detail & Related papers  (2021-08-16T17:50:08Z)
• Computational model discovery with reinforcement learning [3.005240085945858]
  The motivation of this study is to leverage recent breakthroughs in artificial intelligence research to unlock novel solutions to scientific problems encountered in computational science. To address the human intelligence limitations in discovering reduced-order models, we propose to supplement human thinking with artificial intelligence.
  arXiv  Detail & Related papers  (2019-12-29T22:56:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.