Reliability-based cleaning of noisy training labels with inductive conformal prediction in multi-modal biomedical data mining

• URL: http://arxiv.org/abs/2309.07332v1
• Date: Wed, 13 Sep 2023 22:04:50 GMT
• Title: Reliability-based cleaning of noisy training labels with inductive conformal prediction in multi-modal biomedical data mining
• Authors: Xianghao Zhan, Qinmei Xu, Yuanning Zheng, Guangming Lu, Olivier Gevaert
• Abstract summary: We propose a reliability-based training data cleaning method employing inductive conformal prediction (ICP) This method capitalizes on a small set of accurately labeled training data and leverages ICP-calculated reliability metrics to rectify mislabeled data and outliers. We show significant enhancements in classification performance in 86 out of 96 DILI experiments (up to 11.4%), AUROC and AUPRC enhancements in all 48 COVID-19 experiments (up to 23.8% and 69.8%), and accuracy and macro-average F1 score improvements in 47 out of 48 RNA-sequencing experiments (up to 74.6% and 89.0%)
• Score: 23.880097819466602
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurately labeling biomedical data presents a challenge. Traditional semi-supervised learning methods often under-utilize available unlabeled data. To address this, we propose a novel reliability-based training data cleaning method employing inductive conformal prediction (ICP). This method capitalizes on a small set of accurately labeled training data and leverages ICP-calculated reliability metrics to rectify mislabeled data and outliers within vast quantities of noisy training data. The efficacy of the method is validated across three classification tasks within distinct modalities: filtering drug-induced-liver-injury (DILI) literature with title and abstract, predicting ICU admission of COVID-19 patients through CT radiomics and electronic health records, and subtyping breast cancer using RNA-sequencing data. Varying levels of noise to the training labels were introduced through label permutation. Results show significant enhancements in classification performance: accuracy enhancement in 86 out of 96 DILI experiments (up to 11.4%), AUROC and AUPRC enhancements in all 48 COVID-19 experiments (up to 23.8% and 69.8%), and accuracy and macro-average F1 score improvements in 47 out of 48 RNA-sequencing experiments (up to 74.6% and 89.0%). Our method offers the potential to substantially boost classification performance in multi-modal biomedical machine learning tasks. Importantly, it accomplishes this without necessitating an excessive volume of meticulously curated training data.

Related papers

• TRUSWorthy: Toward Clinically Applicable Deep Learning for Confident Detection of Prostate Cancer in Micro-Ultrasound [3.8208601340697386]
  We propose TRUSWorthy, a carefully designed, tuned, and integrated system for reliable PCa detection. Our pipeline integrates self-supervised learning, multiple-instance learning aggregation using transformers, random-undersampled boosting and ensembling. Our method outperforms previous state-of-the-art deep learning methods in terms of accuracy and uncertainty calibration.
  arXiv  Detail & Related papers  (2025-02-20T16:31:24Z)
• Incremental Self-training for Semi-supervised Learning [56.57057576885672]
  IST is simple yet effective and fits existing self-training-based semi-supervised learning methods. We verify the proposed IST on five datasets and two types of backbone, effectively improving the recognition accuracy and learning speed.
  arXiv  Detail & Related papers  (2024-04-14T05:02:00Z)
• Multimodal Pretraining of Medical Time Series and Notes [45.89025874396911]
  Deep learning models show promise in extracting meaningful patterns, but they require extensive labeled data. We propose a novel approach employing self-supervised pretraining, focusing on the alignment of clinical measurements and notes. In downstream tasks, including in-hospital mortality prediction and phenotyping, our model outperforms baselines in settings where only a fraction of the data is labeled.
  arXiv  Detail & Related papers  (2023-12-11T21:53:40Z)
• Improving VTE Identification through Adaptive NLP Model Selection and Clinical Expert Rule-based Classifier from Radiology Reports [2.0637891440066363]
  Venous thromboembolism (VTE) is a severe cardiovascular condition including deep vein thrombosis (DVT) and pulmonary embolism (PE) automated methods have shown promising advancements in identifying VTE events from retrospective data cohorts or aiding clinical experts in identifying VTE events from radiology reports. However, effectively training Deep Learning (DL) and the NLP models is challenging due to limited labeled medical text data, the complexity and heterogeneity of radiology reports, and data imbalance. This study proposes novel method combinations of DL methods, along with data augmentation, adaptive pre-trained NLP model selection, and a clinical expert NLP rule-based
  arXiv  Detail & Related papers  (2023-09-21T17:29:37Z)
• Transfer Learning Based Diagnosis and Analysis of Lung Sound Aberrations [0.35232085374661276]
  This work attempts to develop a non-invasive technique for identifying respiratory sounds acquired by a stethoscope and voice recording software. A visual representation of each audio sample is constructed, allowing resource identification for classification using methods like those used to effectively describe visuals. Respiratory Sound Database obtained cutting-edge results, including accuracy of 95%, precision of 88%, recall score of 86%, and F1 score of 81%.
  arXiv  Detail & Related papers  (2023-03-15T04:46:57Z)
• Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher [54.50747989860957]
  We propose a semi-supervised learning algorithm named Progressive Teacher (PT) to utilize reliable FER datasets as well as large-scale unlabeled expression images for effective training. Experiments on widely-used databases RAF-DB and FERPlus validate the effectiveness of our method, which achieves state-of-the-art performance with accuracy of 89.57% on RAF-DB.
  arXiv  Detail & Related papers  (2022-05-28T07:47:53Z)
• Unsupervised Anomaly Detection in 3D Brain MRI using Deep Learning with impured training data [53.122045119395594]
  We study how unhealthy samples within the training data affect anomaly detection performance for brain MRI-scans. We evaluate a method to identify falsely labeled samples directly during training based on the reconstruction error of the AE.
  arXiv  Detail & Related papers  (2022-04-12T13:05:18Z)
• Towards Reducing Labeling Cost in Deep Object Detection [61.010693873330446]
  We propose a unified framework for active learning, that considers both the uncertainty and the robustness of the detector. Our method is able to pseudo-label the very confident predictions, suppressing a potential distribution drift.
  arXiv  Detail & Related papers  (2021-06-22T16:53:09Z)
• Uncertainty-Aware Semi-supervised Method using Large Unlabelled and Limited Labeled COVID-19 Data [14.530328267425638]
  We propose a Semi-supervised Classification using Limited Labelled Data (SCLLD) for automated COVID-19 detection. The proposed system is trained using 10,000 CT scans collected from Omid hospital. Our method significantly outperforms the supervised training of Convolutional Neural Network (CNN) in case labelled training data is scarce.
  arXiv  Detail & Related papers  (2021-02-12T08:20:20Z)
• Deep Learning Based Classification of Unsegmented Phonocardiogram Spectrograms Leveraging Transfer Learning [0.0]
  Heart murmurs are the most common abnormalities detected during the auscultation process. The two widely used publicly available phonocardiogram (PCG) datasets are from PhysioNet/CinC and PASCAL (2011) We propose a novel, less complex and relatively light custom CNN model for the classification of PhysioNet, combined and PASCAL datasets.
  arXiv  Detail & Related papers  (2020-12-15T16:32:29Z)
• Self-Training with Improved Regularization for Sample-Efficient Chest X-Ray Classification [80.00316465793702]
  We present a deep learning framework that enables robust modeling in challenging scenarios. Our results show that using 85% lesser labeled data, we can build predictive models that match the performance of classifiers trained in a large-scale data setting.
  arXiv  Detail & Related papers  (2020-05-03T02:36:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.