Machine Learning State-of-the-Art with Uncertainties

• URL: http://arxiv.org/abs/2204.05173v2
• Date: Thu, 14 Apr 2022 15:27:15 GMT
• Title: Machine Learning State-of-the-Art with Uncertainties
• Authors: Peter Steinbach, Felicita Gernhardt, Mahnoor Tanveer, Steve Schmerler, Sebastian Starke
• Abstract summary: We conduct an exemplary image classification study in order to demonstrate how confidence intervals around accuracy measurements can greatly enhance the communication of research results. We make suggestions for improving the authoring and reviewing process of machine learning articles.
• Score: 3.4123736336071864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the availability of data, hardware, software ecosystem and relevant skill sets, the machine learning community is undergoing a rapid development with new architectures and approaches appearing at high frequency every year. In this article, we conduct an exemplary image classification study in order to demonstrate how confidence intervals around accuracy measurements can greatly enhance the communication of research results as well as impact the reviewing process. In addition, we explore the hallmarks and limitations of this approximation. We discuss the relevance of this approach reflecting on a spotlight publication of ICLR22. A reproducible workflow is made available as an open-source adjoint to this publication. Based on our discussion, we make suggestions for improving the authoring and reviewing process of machine learning articles.

Related papers

• Position: More Rigorous Software Engineering Would Improve Reproducibility in Machine Learning Research [1.0128808054306186]
  Experimental verification and falsification of scholarly work are part of the scientific method's core. To improve the Machine Learning (ML)-communities' ability to verify results from prior work, we argue for more robust software engineering.
  arXiv  Detail & Related papers  (2025-02-02T20:29:09Z)
• Machine Learning Information Retrieval and Summarisation to Support Systematic Review on Outcomes Based Contracting [7.081184240581488]
  This article presents a study that aims to address these challenges by enhancing the efficiency and scope of systematic reviews in the social sciences through advanced machine learning (ML) and natural language processing (NLP) tools. In particular, we focus on automating stages within the systematic reviewing process that are time-intensive and repetitive for human annotators and which lend themselves to immediate scalability through tools such as information retrieval and summarisation guided by expert advice. The article concludes with a summary of lessons learnt regarding the integrated approach towards systematic reviews and future directions for improvement, including explainability.
  arXiv  Detail & Related papers  (2024-12-11T17:54:01Z)
• Machine Learning Innovations in CPR: A Comprehensive Survey on Enhanced Resuscitation Techniques [52.71395121577439]
  This survey paper explores the transformative role of Machine Learning (ML) and Artificial Intelligence (AI) in Cardiopulmonary Resuscitation (CPR) It highlights the impact of predictive modeling, AI-enhanced devices, and real-time data analysis in improving resuscitation outcomes. The paper provides a comprehensive overview, classification, and critical analysis of current applications, challenges, and future directions in this emerging field.
  arXiv  Detail & Related papers  (2024-11-03T18:01:50Z)
• Retrieval-Enhanced Machine Learning: Synthesis and Opportunities [60.34182805429511]
  Retrieval-enhancement can be extended to a broader spectrum of machine learning (ML) This work introduces a formal framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by synthesizing the literature in various domains in ML with consistent notations which is missing from the current literature. The goal of this work is to equip researchers across various disciplines with a comprehensive, formally structured framework of retrieval-enhanced models, thereby fostering interdisciplinary future research.
  arXiv  Detail & Related papers  (2024-07-17T20:01:21Z)
• SyROCCo: Enhancing Systematic Reviews using Machine Learning [6.805429133535976]
  This paper explores the use of machine learning techniques to help navigate the systematic review process. The application of ML techniques to subsequent stages of a review, such as data extraction and evidence mapping, is in its infancy.
  arXiv  Detail & Related papers  (2024-06-24T11:04:43Z)
• Self-consistent Validation for Machine Learning Electronic Structure [81.54661501506185]
  Method integrates machine learning with self-consistent field methods to achieve both low validation cost and interpret-ability. This, in turn, enables exploration of the model's ability with active learning and instills confidence in its integration into real-world studies.
  arXiv  Detail & Related papers  (2024-02-15T18:41:35Z)
• Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach [77.61131357420201]
  We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs) We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date. We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
  arXiv  Detail & Related papers  (2022-11-07T16:19:42Z)
• Automated Machine Learning Techniques for Data Streams [91.3755431537592]
  This paper surveys the state-of-the-art open-source AutoML tools, applies them to data collected from streams, and measures how their performance changes over time. The results show that off-the-shelf AutoML tools can provide satisfactory results but in the presence of concept drift, detection or adaptation techniques have to be applied to maintain the predictive accuracy over time.
  arXiv  Detail & Related papers  (2021-06-14T11:42:46Z)
• Fantastic Features and Where to Find Them: Detecting Cognitive Impairment with a Subsequence Classification Guided Approach [6.063165888023164]
  We describe a new approach to feature engineering that leverages sequential machine learning models and domain knowledge to predict which features help enhance performance. We demonstrate that CI classification accuracy improves by 2.3% over a strong baseline when using features produced by this method.
  arXiv  Detail & Related papers  (2020-10-13T17:57:18Z)
• Bayesian active learning for production, a systematic study and a reusable library [85.32971950095742]
  In this paper, we analyse the main drawbacks of current active learning techniques. We do a systematic study on the effects of the most common issues of real-world datasets on the deep active learning process. We derive two techniques that can speed up the active learning loop such as partial uncertainty sampling and larger query size.
  arXiv  Detail & Related papers  (2020-06-17T14:51:11Z)
• Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) [43.55295847227261]
  Reproducibility is obtaining similar results as presented in a paper or talk, using the same code and data (when available) In 2019, the Neural Information Processing Systems (NeurIPS) conference introduced a program, designed to improve the standards across the community for how we conduct, communicate, and evaluate machine learning research. In this paper, we describe each of these components, how they were deployed, as well as what we were able to learn from this initiative.
  arXiv  Detail & Related papers  (2020-03-27T02:16:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.