Reproducibility, Replicability, and Repeatability: A survey of reproducible research with a focus on high performance computing

• URL: http://arxiv.org/abs/2402.07530v2
• Date: Fri, 13 Sep 2024 11:44:35 GMT
• Title: Reproducibility, Replicability, and Repeatability: A survey of reproducible research with a focus on high performance computing
• Authors: Benjamin A. Antunes, David R. C. Hill,
• Abstract summary: Reproducibility is a fundamental principle in scientific research. Highperformance computing presents unique challenges. This paper provides a comprehensive review of these concerns and potential solutions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reproducibility is widely acknowledged as a fundamental principle in scientific research. Currently, the scientific community grapples with numerous challenges associated with reproducibility, often referred to as the ''reproducibility crisis.'' This crisis permeated numerous scientific disciplines. In this study, we examined the factors in scientific practices that might contribute to this lack of reproducibility. Significant focus is placed on the prevalent integration of computation in research, which can sometimes function as a black box in published papers. Our study primarily focuses on highperformance computing (HPC), which presents unique reproducibility challenges. This paper provides a comprehensive review of these concerns and potential solutions. Furthermore, we discuss the critical role of reproducible research in advancing science and identifying persisting issues within the field of HPC.

Related papers

• Targeted Reduction of Causal Models [55.11778726095353]
  Causal Representation Learning offers a promising avenue to uncover interpretable causal patterns in simulations. We introduce Targeted Causal Reduction (TCR), a method for condensing complex intervenable models into a concise set of causal factors. Its ability to generate interpretable high-level explanations from complex models is demonstrated on toy and mechanical systems.
  arXiv  Detail & Related papers  (2023-11-30T15:46:22Z)
• Causal machine learning for single-cell genomics [94.28105176231739]
  We discuss the application of machine learning techniques to single-cell genomics and their challenges. We first present the model that underlies most of current causal approaches to single-cell biology. We then identify open problems in the application of causal approaches to single-cell data.
  arXiv  Detail & Related papers  (2023-10-23T13:35:24Z)
• SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models [70.5763210869525]
  We introduce an expansive benchmark suite SciBench for Large Language Model (LLM) SciBench contains a dataset featuring a range of collegiate-level scientific problems from mathematics, chemistry, and physics domains. The results reveal that the current LLMs fall short of delivering satisfactory performance, with the best overall score of merely 43.22%.
  arXiv  Detail & Related papers  (2023-07-20T07:01:57Z)
• Computational Reproducibility in Computational Social Science [0.8930269507906258]
  We argue that computational-x disciplines such as computational social science are also susceptible for the symptoms of the crises. We provide solutions for Computational Social Science that hinder researchers from obtaining the highest level of data.
  arXiv  Detail & Related papers  (2023-07-04T21:04:18Z)
• A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why? [84.46288849132634]
  We propose a systematic framework for analyzing the evolution of research topics in a scientific field using causal discovery and inference techniques. We define three variables to encompass diverse facets of the evolution of research topics within NLP. We utilize a causal discovery algorithm to unveil the causal connections among these variables using observational data.
  arXiv  Detail & Related papers  (2023-05-22T11:08:00Z)
• Reproducibility of Machine Learning: Terminology, Recommendations and Open Issues [5.30596984761294]
  A crisis has been recently acknowledged by scientists and this seems to affect even more Artificial Intelligence and Machine Learning. We critically review the current literature on the topic and highlight the open issues. We identify key elements often overlooked in modern Machine Learning and provide novel recommendations for them.
  arXiv  Detail & Related papers  (2023-02-24T15:33:20Z)
• Social and environmental impact of recent developments in machine learning on biology and chemistry research [0.0]
  Recent developments in machine learning can potentially affect basic and applied research. These developments can potentially affect basic and applied research, such as drug discovery and development.
  arXiv  Detail & Related papers  (2022-10-01T20:29:01Z)
• Reproducibility in machine learning for medical imaging [3.1390096961027076]
  This chapter intends at being an introduction to for researchers in the field of machine learning for medical imaging. For each of them, we aim at defining it, at describing the requirements to achieve it and at discussing its utility. The chapter ends with a discussion on the benefits of didactic and with a plea for a non-dogmatic approach to this concept and its implementation in research practice.
  arXiv  Detail & Related papers  (2022-09-12T09:00:04Z)
• A Guide to Reproducible Research in Signal Processing and Machine Learning [9.69596041242667]
  In 2016 a survey conducted by the journal Nature found that 50% of researchers were unable to reproduce their own experiments. We aim to present signal processing researchers with a set of practical tools and strategies that can help mitigate many of the obstacles to producing reproducible computational experiments.
  arXiv  Detail & Related papers  (2021-08-27T16:42:32Z)
• Towards Continual Reinforcement Learning: A Review and Perspectives [69.48324517535549]
  We aim to provide a literature review of different formulations and approaches to continual reinforcement learning (RL) While still in its early days, the study of continual RL has the promise to develop better incremental reinforcement learners. These include applications such as those in the fields of healthcare, education, logistics, and robotics.
  arXiv  Detail & Related papers  (2020-12-25T02:35:27Z)
• Heterogeneous Representation Learning: A Review [66.12816399765296]
  Heterogeneous Representation Learning (HRL) brings some unique challenges. We present a unified learning framework which is able to model most existing learning settings with the heterogeneous inputs. We highlight the challenges that are less-touched in HRL and present future research directions.
  arXiv  Detail & Related papers  (2020-04-28T05:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.