CheXpert++: Approximating the CheXpert labeler for Speed,Differentiability, and Probabilistic Output

• URL: http://arxiv.org/abs/2006.15229v1
• Date: Fri, 26 Jun 2020 22:14:11 GMT
• Title: CheXpert++: Approximating the CheXpert labeler for Speed,Differentiability, and Probabilistic Output
• Abstract summary: CheXpert is a labeler that produces diagnostic labels for chest X-ray radiology reports. CheXpert is non-differentiable so can't be used in any applications that require gradients to flow through the labeler. $texttCheXpert++$ achieves 99.81% parity with CheXpert.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is often infeasible or impossible to obtain ground truth labels for medical data. To circumvent this, one may build rule-based or other expert-knowledge driven labelers to ingest data and yield silver labels absent any ground-truth training data. One popular such labeler is CheXpert, a labeler that produces diagnostic labels for chest X-ray radiology reports. CheXpert is very useful, but is relatively computationally slow, especially when integrated with end-to-end neural pipelines, is non-differentiable so can't be used in any applications that require gradients to flow through the labeler, and does not yield probabilistic outputs, which limits our ability to improve the quality of the silver labeler through techniques such as active learning. In this work, we solve all three of these problems with $\texttt{CheXpert++}$, a BERT-based, high-fidelity approximation to CheXpert. $\texttt{CheXpert++}$ achieves 99.81\% parity with CheXpert, which means it can be reliably used as a drop-in replacement for CheXpert, all while being significantly faster, fully differentiable, and probabilistic in output. Error analysis of $\texttt{CheXpert++}$ also demonstrates that $\texttt{CheXpert++}$ has a tendency to actually correct errors in the CheXpert labels, with $\texttt{CheXpert++}$ labels being more often preferred by a clinician over CheXpert labels (when they disagree) on all but one disease task. To further demonstrate the utility of these advantages in this model, we conduct a proof-of-concept active learning study, demonstrating we can improve accuracy on an expert labeled random subset of report sentences by approximately 8\% over raw, unaltered CheXpert by using one-iteration of active-learning inspired re-training. These findings suggest that simple techniques in co-learning and active learning can yield high-quality labelers under minimal, and controllable human labeling demands.

Related papers

• Learning Sparse Label Couplings for Multilabel Chest X-Ray Diagnosis [0.5735035463793009]
  We study multilabel classification of chest X-rays and present a simple, strong pipeline built on SE-ResNeXt101 $(32 times 4d)$.<n>The backbone is finetuned for 14 thoracic findings with a sigmoid head, trained using Multilabel Iterative Stratification (MIS) for robust cross-validation.<n>We propose a lightweight Label-Graph Refinement module placed after the classifier: given per-label probabilities, it learns a sparse, trainable inter-label coupling matrix.
  arXiv  Detail & Related papers  (2025-11-11T03:38:01Z)
• Practical estimation of the optimal classification error with soft labels and calibration [52.1410307583181]
  We extend a previous work that utilizes soft labels for estimating the Bayes error, the optimal error rate.<n>We tackle a more challenging problem setting: estimation with corrupted soft labels.<n>Our method is instance-free, i.e., we do not assume access to any input instances.
  arXiv  Detail & Related papers  (2025-05-27T06:04:57Z)
• Robust Graph-Based Semi-Supervised Learning via $p$-Conductances [49.0776396776252]
  We study the problem of semi-supervised learning on graphs in the regime where data labels are scarce or possibly corrupted. We propose an approach called $p$-conductance learning that generalizes the $p$-Laplace and Poisson learning methods. Empirical results on computer vision and citation datasets demonstrate that our approach achieves state-of-the-art accuracy in low label-rate, corrupted-label, and partial-label regimes.
  arXiv  Detail & Related papers  (2025-02-13T01:11:25Z)
• Learning from Ambiguous Data with Hard Labels [34.06499138206804]
  Real-world data often contains intrinsic ambiguity that the common single-hard-label annotation paradigm ignores. Standard training using ambiguous data with hard labels may produce overly confident models and thus leading to poor generalization. We propose a novel framework called Quantized Label Learning (QLL) to alleviate this issue.
  arXiv  Detail & Related papers  (2025-01-03T14:54:49Z)
• Dual-Label Learning With Irregularly Present Labels [14.817794592309637]
  This work focuses on the two-label learning task, and proposes a novel training and inference framework, Dual-Label Learning (DLL) Our method makes consistently better predictions than baseline approaches by up to a 10% gain in F1-score or MAPE. Remarkably, our method provided with data at a label missing rate as high as 60% can achieve similar or even better results than baseline approaches at a label missing rate of only 10%.
  arXiv  Detail & Related papers  (2024-10-18T11:07:26Z)
• Retraining with Predicted Hard Labels Provably Increases Model Accuracy [77.71162068832108]
  Retraining can improve the population accuracy obtained by initially training with the given (noisy) labels. We empirically show that retraining selectively on the samples for which the predicted label matches the given label significantly improves label DP training.
  arXiv  Detail & Related papers  (2024-06-17T04:53:47Z)
• Enhancing Learning with Label Differential Privacy by Vector Approximation [12.212865127830872]
  Label differential privacy (DP) is a framework that protects the privacy of labels in training datasets, while the feature vectors are public. Existing approaches protect the privacy of labels by flipping them randomly, and then train a model to make the output approximate the privatized label. We propose a vector approximation approach, which is easy to implement and introduces little additional computational overhead.
  arXiv  Detail & Related papers  (2024-05-24T02:08:45Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• How many labelers do you have? A closer look at gold-standard labels [10.637125300701795]
  We show how access to non-aggregated label information can make training well-calibrated models more feasible than it is with gold-standard labels. We make several predictions for real-world datasets, including when non-aggregate labels should improve learning performance.
  arXiv  Detail & Related papers  (2022-06-24T02:33:50Z)
• Trustable Co-label Learning from Multiple Noisy Annotators [68.59187658490804]
  Supervised deep learning depends on massive accurately annotated examples. A typical alternative is learning from multiple noisy annotators. This paper proposes a data-efficient approach, called emphTrustable Co-label Learning (TCL)
  arXiv  Detail & Related papers  (2022-03-08T16:57:00Z)
• Debiased Pseudo Labeling in Self-Training [77.83549261035277]
  Deep neural networks achieve remarkable performances on a wide range of tasks with the aid of large-scale labeled datasets. To mitigate the requirement for labeled data, self-training is widely used in both academia and industry by pseudo labeling on readily-available unlabeled data. We propose Debiased, in which the generation and utilization of pseudo labels are decoupled by two independent heads.
  arXiv  Detail & Related papers  (2022-02-15T02:14:33Z)
• Unbiased Loss Functions for Multilabel Classification with Missing Labels [2.1549398927094874]
  Missing labels are a ubiquitous phenomenon in extreme multi-label classification (XMC) tasks. This paper derives the unique unbiased estimators for the different multilabel reductions.
  arXiv  Detail & Related papers  (2021-09-23T10:39:02Z)
• Self-Supervised Learning from Semantically Imprecise Data [7.24935792316121]
  Learning from imprecise labels such as "animal" or "bird" is an important capability when expertly labeled training data is scarce. CHILLAX is a recently proposed method to tackle this task. We extend CHILLAX with a self-supervised scheme using constrained extrapolation to generate pseudo-labels.
  arXiv  Detail & Related papers  (2021-04-22T07:26:14Z)
• Learning to Purify Noisy Labels via Meta Soft Label Corrector [49.92310583232323]
  Recent deep neural networks (DNNs) can easily overfit to biased training data with noisy labels. Label correction strategy is commonly used to alleviate this issue. We propose a meta-learning model which could estimate soft labels through meta-gradient descent step.
  arXiv  Detail & Related papers  (2020-08-03T03:25:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.