The Fundamental Principles of Reproducibility
- URL: http://arxiv.org/abs/2011.10098v2
- Date: Mon, 22 Feb 2021 14:01:58 GMT
- Title: The Fundamental Principles of Reproducibility
- Authors: Odd Erik Gundersen
- Abstract summary: I take a fundamental view on rooted in the scientific method.
The scientific method is analysed and characterised in order to develop the terminology required to define.
- Score: 2.4671396651514983
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Reproducibility is a confused terminology. In this paper, I take a
fundamental view on reproducibility rooted in the scientific method. The
scientific method is analysed and characterised in order to develop the
terminology required to define reproducibility. Further, the literature on
reproducibility and replication is surveyed, and experiments are modeled as
tasks and problem solving methods. Machine learning is used to exemplify the
described approach. Based on the analysis, reproducibility is defined and three
different types of reproducibility as well as four degrees of reproducibility
are specified.
Related papers
- Can citations tell us about a paper's reproducibility? A case study of machine learning papers [3.5120846057971065]
Resource constraints and inadequate documentation can make running replications particularly challenging.
We introduce a sentiment analysis framework applied to citation contexts from papers involved in Machine Learning Reproducibility Challenges.
arXiv Detail & Related papers (2024-05-07T03:29:11Z) - Seeing Unseen: Discover Novel Biomedical Concepts via
Geometry-Constrained Probabilistic Modeling [53.7117640028211]
We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues.
We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space.
A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
arXiv Detail & Related papers (2024-03-02T00:56:05Z) - Representation Surgery: Theory and Practice of Affine Steering [72.61363182652853]
Language models often exhibit undesirable behavior, e.g., generating toxic or gender-biased text.
One natural (and common) approach to prevent the model from exhibiting undesirable behavior is to steer the model's representations.
This paper investigates the formal and empirical properties of steering functions.
arXiv Detail & Related papers (2024-02-15T00:20:30Z) - A Category-theoretical Meta-analysis of Definitions of Disentanglement [97.34033555407403]
Disentangling the factors of variation in data is a fundamental concept in machine learning.
This paper presents a meta-analysis of existing definitions of disentanglement.
arXiv Detail & Related papers (2023-05-11T15:24:20Z) - Missing Information, Unresponsive Authors, Experimental Flaws: The
Impossibility of Assessing the Reproducibility of Previous Human Evaluations
in NLP [84.08476873280644]
Just 13% of papers had (i) sufficiently low barriers to reproduction, and (ii) enough obtainable information, to be considered for reproduction.
As a result, we had to change our coordinated study design from a reproduce approach to a standardise-then-reproduce-twice approach.
arXiv Detail & Related papers (2023-05-02T17:46:12Z) - Reproducibility of Machine Learning: Terminology, Recommendations and
Open Issues [5.30596984761294]
A crisis has been recently acknowledged by scientists and this seems to affect even more Artificial Intelligence and Machine Learning.
We critically review the current literature on the topic and highlight the open issues.
We identify key elements often overlooked in modern Machine Learning and provide novel recommendations for them.
arXiv Detail & Related papers (2023-02-24T15:33:20Z) - Reproducibility in machine learning for medical imaging [3.1390096961027076]
This chapter intends at being an introduction to for researchers in the field of machine learning for medical imaging.
For each of them, we aim at defining it, at describing the requirements to achieve it and at discussing its utility.
The chapter ends with a discussion on the benefits of didactic and with a plea for a non-dogmatic approach to this concept and its implementation in research practice.
arXiv Detail & Related papers (2022-09-12T09:00:04Z) - Sequential Causal Imitation Learning with Unobserved Confounders [82.22545916247269]
"Monkey see monkey do" is an age-old adage, referring to na"ive imitation without a deep understanding of a system's underlying mechanics.
This paper investigates the problem of causal imitation learning in sequential settings, where the imitator must make multiple decisions per episode.
arXiv Detail & Related papers (2022-08-12T13:53:23Z) - A Novel Bio-Inspired Texture Descriptor based on Biodiversity and
Taxonomic Measures [81.08571247838206]
We propose a novel approach capable of quantifying a complex system of diverse patterns through species diversity and richness and taxonomic distinctiveness.
The proposed approach considers each image channel as a species ecosystem and computes species diversity and richness measures as well as taxonomic measures to describe the texture.
arXiv Detail & Related papers (2021-02-13T20:14:14Z) - Reproducibility in Evolutionary Computation [0.0]
We discuss, within the context of EC, the different types of as well as the concepts of artifact and measurement.
We identify cultural and technical obstacles to in the EC field.
We suggest tools that may help to overcome some of these obstacles.
arXiv Detail & Related papers (2021-02-05T19:06:35Z) - Research Reproducibility as a Survival Analysis [22.66983713481359]
We consider modeling a paper as a survival analysis problem.
We show how a survival analysis allows us to draw new insights that better explain prior longitudinal data.
arXiv Detail & Related papers (2020-12-17T20:56:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.