Best Practices and Scoring System on Reviewing A.I. based Medical Imaging Papers: Part 1 Classification

• URL: http://arxiv.org/abs/2202.01863v1
• Date: Thu, 3 Feb 2022 21:46:59 GMT
• Title: Best Practices and Scoring System on Reviewing A.I. based Medical Imaging Papers: Part 1 Classification
• Authors: Timothy L. Kline, Felipe Kitamura, Ian Pan, Amine M. Korchi, Neil Tenenholtz, Linda Moy, Judy Wawira Gichoya, Igor Santos, Steven Blumer, Misha Ysabel Hwang, Kim-Ann Git, Abishek Shroff, Elad Walach, George Shih, Steve Langer
• Abstract summary: The Machine Learning Education Sub-Committee of SIIM has identified a knowledge gap and a serious need to establish guidelines for reviewing these studies. This first entry in the series focuses on the task of image classification. The goal of this series is to provide resources to help improve the review process for A.I.-based medical imaging papers.
• Score: 0.9428556282541211
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the recent advances in A.I. methodologies and their application to medical imaging, there has been an explosion of related research programs utilizing these techniques to produce state-of-the-art classification performance. Ultimately, these research programs culminate in submission of their work for consideration in peer reviewed journals. To date, the criteria for acceptance vs. rejection is often subjective; however, reproducible science requires reproducible review. The Machine Learning Education Sub-Committee of SIIM has identified a knowledge gap and a serious need to establish guidelines for reviewing these studies. Although there have been several recent papers with this goal, this present work is written from the machine learning practitioners standpoint. In this series, the committee will address the best practices to be followed in an A.I.-based study and present the required sections in terms of examples and discussion of what should be included to make the studies cohesive, reproducible, accurate, and self-contained. This first entry in the series focuses on the task of image classification. Elements such as dataset curation, data pre-processing steps, defining an appropriate reference standard, data partitioning, model architecture and training are discussed. The sections are presented as they would be detailed in a typical manuscript, with content describing the necessary information that should be included to make sure the study is of sufficient quality to be considered for publication. The goal of this series is to provide resources to not only help improve the review process for A.I.-based medical imaging papers, but to facilitate a standard for the information that is presented within all components of the research study. We hope to provide quantitative metrics in what otherwise may be a qualitative review process.

Related papers

• Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions [62.12545440385489]
  Large language models (LLMs) have brought substantial advancements in text generation, but their potential for enhancing classification tasks remains underexplored. We propose a framework for thoroughly investigating fine-tuning LLMs for classification, including both generation- and encoding-based approaches. We instantiate this framework in edit intent classification (EIC), a challenging and underexplored classification task.
  arXiv  Detail & Related papers  (2024-10-02T20:48:28Z)
• Why do you cite? An investigation on citation intents and decision-making classification processes [1.7812428873698407]
  This study emphasizes the importance of trustfully classifying citation intents. We present a study utilizing advanced Ensemble Strategies for Citation Intent Classification (CIC) One of our models sets as a new state-of-the-art (SOTA) with an 89.46% Macro-F1 score on the SciCite benchmark.
  arXiv  Detail & Related papers  (2024-07-18T09:29:33Z)
• Efficient Systematic Reviews: Literature Filtering with Transformers & Transfer Learning [0.0]
  It comes with an increasing burden in terms of the time required to identify the important articles of research for a given topic. We develop a method for building a general-purpose filtering system that matches a research question, posed as a natural language description of the required content. Our results demonstrate that transformer models, pre-trained on biomedical literature, and then fine tuned for the specific task, offer a promising solution to this problem.
  arXiv  Detail & Related papers  (2024-05-30T02:55:49Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [58.6354685593418]
  This paper proposes several article-level, field-normalized, and large language model-empowered bibliometric indicators to evaluate reviews. The newly emerging AI-generated literature reviews are also appraised. This work offers insights into the current challenges of literature reviews and envisions future directions for their development.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Zero-shot Generative Large Language Models for Systematic Review Screening Automation [55.403958106416574]
  This study investigates the effectiveness of using zero-shot large language models for automatic screening. We evaluate the effectiveness of eight different LLMs and investigate a calibration technique that uses a predefined recall threshold.
  arXiv  Detail & Related papers  (2024-01-12T01:54:08Z)
• Development and validation of a natural language processing algorithm to pseudonymize documents in the context of a clinical data warehouse [53.797797404164946]
  The study highlights the difficulties faced in sharing tools and resources in this domain. We annotated a corpus of clinical documents according to 12 types of identifying entities. We build a hybrid system, merging the results of a deep learning model as well as manual rules.
  arXiv  Detail & Related papers  (2023-03-23T17:17:46Z)
• Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search [31.797257552928336]
  We apply several pre-trained language models to the systematic review document ranking task. An empirical analysis compares how effective neural methods compare to traditional methods for this task. Our results show that BERT-based rankers outperform the current state-of-the-art screening prioritisation methods.
  arXiv  Detail & Related papers  (2022-12-18T05:26:40Z)
• Mirror Matching: Document Matching Approach in Seed-driven Document Ranking for Medical Systematic Reviews [31.3220495275256]
  Document ranking is an approach for assisting researchers by providing document rankings where relevant documents are ranked higher than irrelevant ones. We propose a document matching measure named Mirror Matching, which calculates matching scores between medical abstract texts by incorporating common writing patterns.
  arXiv  Detail & Related papers  (2021-12-28T22:27:52Z)
• A Systematic Literature Review of Empiricism and Norms of Reporting in Computing Education Research Literature [4.339510167603376]
  The goal of this study is to characterize the reporting of empiricism in Computing Education Research (CER) literature. We conducted an SLR of 427 papers published during 2014 and 2015 in five CER venues. Over 80% of papers had some form of empirical evaluation.
  arXiv  Detail & Related papers  (2021-07-02T16:37:29Z)
• What is the State of the Art of Computer Vision-Assisted Cytology? A Systematic Literature Review [47.42354724922676]
  We conducted a Systematic Literature Review to identify the state-of-art of computer vision techniques currently applied to cytology. The most used methods in the analyzed works are deep learning-based (70 papers), while fewer works employ classic computer vision only (101 papers) We conclude that there still is a lack of high-quality datasets for many types of stains and most of the works are not mature enough to be applied in a daily clinical diagnostic routine.
  arXiv  Detail & Related papers  (2021-05-24T13:50:45Z)
• A Survey on Text Classification: From Shallow to Deep Learning [83.47804123133719]
  The last decade has seen a surge of research in this area due to the unprecedented success of deep learning. This paper fills the gap by reviewing the state-of-the-art approaches from 1961 to 2021. We create a taxonomy for text classification according to the text involved and the models used for feature extraction and classification.
  arXiv  Detail & Related papers  (2020-08-02T00:09:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.