Is Research Software Science a Metascience?

• URL: http://arxiv.org/abs/2509.13436v2
• Date: Fri, 10 Oct 2025 18:23:24 GMT
• Title: Is Research Software Science a Metascience?
• Authors: Evan Eisinger, Michael A. Heroux,
• Abstract summary: We define metascience and RSS, compare their principles and objectives, and examine their overlaps.<n>We argue RSS is best understood as a distinct interdisciplinary domain that aligns with metascience.<n>Regardless of classification, applying scientific rigor to research software ensures the tools of discovery meet the standards of the discoveries themselves.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As research increasingly relies on computational methods, the reliability of scientific results depends on the quality, reproducibility, and transparency of research software. Ensuring these qualities is critical for scientific integrity and discovery. This paper asks whether Research Software Science (RSS)--the empirical study of how research software is developed and used--should be considered a form of metascience, the science of science. Classification matters because it could affect recognition, funding, and integration of RSS into research improvement. We define metascience and RSS, compare their principles and objectives, and examine their overlaps. Arguments for classification highlight shared commitments to reproducibility, transparency, and empirical study of research processes. Arguments against portraying RSS as a specialized domain focused on a tool rather than the broader scientific enterprise. Our analysis finds RSS advances core goals of metascience, especially in computational reproducibility, and bridges technical, social, and cognitive aspects of research. Its classification depends on whether one adopts a broad definition of metascience--any empirical effort to improve science--or a narrow one focused on systemic and epistemological structures. We argue RSS is best understood as a distinct interdisciplinary domain that aligns with, and in some definitions fits within, metascience. Recognizing it as such can strengthen its role in improving reliability, justify funding, and elevate software development in research institutions. Regardless of classification, applying scientific rigor to research software ensures the tools of discovery meet the standards of the discoveries themselves.

Related papers

• Critically Engaged Pragmatism: A Scientific Norm and Social, Pragmatist Epistemology for AI Science Evaluation Tools [0.0]
  I caution that AI science evaluation tools are particularly prone to false ascent due to contestation about the purposes to which they should be put.<n>I argue for a social, pragmatist and a newly articulated norm of Critically Engaged to enjoin scientific communities to vigorously scrutinize the purposes and purpose-specific reliability of AI science evaluation tools.
  arXiv  Detail & Related papers  (2026-01-13T02:02:27Z)
• Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Network [46.56867772369597]
  Best practices for meta-algorithmic research exist, but they are scattered between different publications and fields.<n>This report collects good practices for empirical meta-algorithmic research across the subfields of the COSEAL community.<n>It establishes the current state-of-the-art practices within meta-algorithmic research and serves as a guideline to both new researchers and practitioners in meta-algorithmic fields.
  arXiv  Detail & Related papers  (2025-12-18T12:59:45Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [221.34650992288505]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning [53.82037883518254]
  We introduce SciReas, a diverse suite of existing benchmarks for scientific reasoning tasks.<n>We then propose KRUX, a probing framework for studying the distinct roles of reasoning and knowledge in scientific tasks.
  arXiv  Detail & Related papers  (2025-08-26T17:04:23Z)
• From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery [108.1082357960201]
  Agentic AI shows capabilities in hypothesis generation, experimental design, execution, analysis, and iterative refinement.<n>This survey provides a domain-oriented review of autonomous scientific discovery across life sciences, chemistry, materials science, and physics.
  arXiv  Detail & Related papers  (2025-08-18T05:25:54Z)
• The Evolving Role of Large Language Models in Scientific Innovation: Evaluator, Collaborator, and Scientist [3.7803247326675162]
  Scientific innovation is undergoing a paradigm shift driven by the rapid advancement of Large Language Models (LLMs)<n>This survey proposes a comprehensive framework to categorize the evolving roles of LLMs in scientific innovation across three hierarchical levels: Evaluator, Collaborator, and Scientist.
  arXiv  Detail & Related papers  (2025-07-16T00:11:01Z)
• From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery [67.07598263346591]
  Large Language Models (LLMs) are catalyzing a paradigm shift in scientific discovery.<n>This survey systematically charts this burgeoning field, placing a central focus on the changing roles and escalating capabilities of LLMs in science.
  arXiv  Detail & Related papers  (2025-05-19T15:41:32Z)
• Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation [58.064940977804596]
  A plethora of new AI models and tools has been proposed, promising to empower researchers and academics worldwide to conduct their research more effectively and efficiently.<n>Ethical concerns regarding shortcomings of these tools and potential for misuse take a particularly prominent place in our discussion.
  arXiv  Detail & Related papers  (2025-02-07T18:26:45Z)
• A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why? [84.46288849132634]
  We propose a systematic framework for analyzing the evolution of research topics in a scientific field using causal discovery and inference techniques. We define three variables to encompass diverse facets of the evolution of research topics within NLP. We utilize a causal discovery algorithm to unveil the causal connections among these variables using observational data.
  arXiv  Detail & Related papers  (2023-05-22T11:08:00Z)
• Towards a Benchmark for Scientific Understanding in Humans and Machines [2.714583452862024]
  We propose a framework to create a benchmark for scientific understanding, utilizing tools from philosophy of science. We adopt a behavioral notion according to which genuine understanding should be recognized as an ability to perform certain tasks.
  arXiv  Detail & Related papers  (2023-04-20T14:05:53Z)
• SciTweets -- A Dataset and Annotation Framework for Detecting Scientific Online Discourse [2.3371548697609303]
  Scientific topics, claims and resources are increasingly debated as part of online discourse. This has led to both significant societal impact and increased interest in scientific online discourse from various disciplines. Research across disciplines currently suffers from a lack of robust definitions of the various forms of science-relatedness.
  arXiv  Detail & Related papers  (2022-06-15T08:14:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.