Labelling imaging datasets on the basis of neuroradiology reports: a validation study

• URL: http://arxiv.org/abs/2007.04226v5
• Date: Tue, 9 Mar 2021 01:39:06 GMT
• Title: Labelling imaging datasets on the basis of neuroradiology reports: a validation study
• Authors: David A. Wood, Sina Kafiabadi, Aisha Al Busaidi, Emily Guilhem, Jeremy Lynch, Matthew Townend, Antanas Montvila, Juveria Siddiqui, Naveen Gadapa, Matthew Benger, Gareth Barker, Sebastian Ourselin, James H. Cole, Thomas C. Booth
• Abstract summary: We show that, in our experience, assigning binary labels to images from reports alone is highly accurate. In contrast to the binary labels, however, the accuracy of more granular labelling is dependent on the category. We also show that downstream model performance is reduced when labelling of training reports is performed by a non-specialist.
• Score: 0.3871995016053975
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural language processing (NLP) shows promise as a means to automate the labelling of hospital-scale neuroradiology magnetic resonance imaging (MRI) datasets for computer vision applications. To date, however, there has been no thorough investigation into the validity of this approach, including determining the accuracy of report labels compared to image labels as well as examining the performance of non-specialist labellers. In this work, we draw on the experience of a team of neuroradiologists who labelled over 5000 MRI neuroradiology reports as part of a project to build a dedicated deep learning-based neuroradiology report classifier. We show that, in our experience, assigning binary labels (i.e. normal vs abnormal) to images from reports alone is highly accurate. In contrast to the binary labels, however, the accuracy of more granular labelling is dependent on the category, and we highlight reasons for this discrepancy. We also show that downstream model performance is reduced when labelling of training reports is performed by a non-specialist. To allow other researchers to accelerate their research, we make our refined abnormality definitions and labelling rules available, as well as our easy-to-use radiology report labelling app which helps streamline this process.

Related papers

• AutoRG-Brain: Grounded Report Generation for Brain MRI [57.22149878985624]
  Radiologists are tasked with interpreting a large number of images in a daily base, with the responsibility of generating corresponding reports. This demanding workload elevates the risk of human error, potentially leading to treatment delays, increased healthcare costs, revenue loss, and operational inefficiencies. We initiate a series of work on grounded Automatic Report Generation (AutoRG) This system supports the delineation of brain structures, the localization of anomalies, and the generation of well-organized findings.
  arXiv  Detail & Related papers  (2024-07-23T17:50:00Z)
• RaTEScore: A Metric for Radiology Report Generation [59.37561810438641]
  This paper introduces a novel, entity-aware metric, as Radiological Report (Text) Evaluation (RaTEScore) RaTEScore emphasizes crucial medical entities such as diagnostic outcomes and anatomical details, and is robust against complex medical synonyms and sensitive to negation expressions. Our evaluations demonstrate that RaTEScore aligns more closely with human preference than existing metrics, validated both on established public benchmarks and our newly proposed RaTE-Eval benchmark.
  arXiv  Detail & Related papers  (2024-06-24T17:49:28Z)
• A self-supervised text-vision framework for automated brain abnormality detection [1.1852378060341289]
  We present a text-vision framework that learns to detect clinically relevant abnormalities in brain MRI scans. Our framework could also serve as a clinical decision support tool.
  arXiv  Detail & Related papers  (2024-05-05T01:51:58Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation [92.73584302508907]
  We propose a knowledge graph with Dynamic structure and nodes to facilitate medical report generation with Contrastive Learning. In detail, the fundamental structure of our graph is pre-constructed from general knowledge. Each image feature is integrated with its very own updated graph before being fed into the decoder module for report generation.
  arXiv  Detail & Related papers  (2023-03-18T03:53:43Z)
• Weakly Supervised Learning Significantly Reduces the Number of Labels Required for Intracranial Hemorrhage Detection on Head CT [7.713240800142863]
  Machine learning pipelines, in particular those based on deep learning (DL) models, require large amounts of labeled data. This work studies the question of what kind of labels should be collected for the problem of intracranial hemorrhage detection in brain CT. We find that strong supervision (i.e., learning with local image-level annotations) and weak supervision (i.e., learning with only global examination-level labels) achieve comparable performance.
  arXiv  Detail & Related papers  (2022-11-29T04:42:41Z)
• Using Multi-modal Data for Improving Generalizability and Explainability of Disease Classification in Radiology [0.0]
  Traditional datasets for the radiological diagnosis tend to only provide the radiology image alongside the radiology report. This paper utilizes the recently published Eye-Gaze dataset to perform an exhaustive study on the impact on performance and explainability of deep learning (DL) classification. We find that the best classification performance of X-ray images is achieved with a combination of radiology report free-text and radiology image, with the eye-gaze data providing no performance boost.
  arXiv  Detail & Related papers  (2022-07-29T16:49:05Z)
• Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation [72.0276067144762]
  We discuss and define the two common types of label noise in medical images. We propose an uncertainty estimation-based framework to handle these two label noise amid the medical image classification task.
  arXiv  Detail & Related papers  (2021-02-28T14:56:45Z)
• Paying Per-label Attention for Multi-label Extraction from Radiology Reports [1.9601378412924186]
  We tackle the automated extraction of structured labels from head CT reports for imaging of suspected stroke patients. We propose a set of 31 labels which correspond to radiographic findings and clinical impressions related to neurological abnormalities. We are able to robustly extract many labels with a single model, classified according to the radiologist's reporting.
  arXiv  Detail & Related papers  (2020-07-31T16:11:09Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)
• Automated Labelling using an Attention model for Radiology reports of MRI scans (ALARM) [0.8163463207064016]
  We present a transformer-based network for magnetic resonance imaging (MRI) radiology report classification. Our model's performance is comparable to that of an expert radiologist, and better than that of an expert physician. We make code available online for researchers to label their own MRI datasets for medical imaging applications.
  arXiv  Detail & Related papers  (2020-02-16T15:04:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.