Calibrated Selective Prediction Using Deep Ensembles for ROI-Based Thyroid Nodule Ultrasound Classification Under Dataset Shift: A Retrospective Evaluation

• URL: http://arxiv.org/abs/2607.12075v1
• Date: Mon, 13 Jul 2026 18:50:57 GMT
• Title: Calibrated Selective Prediction Using Deep Ensembles for ROI-Based Thyroid Nodule Ultrasound Classification Under Dataset Shift: A Retrospective Evaluation
• Abstract summary: We developed a deterministic five-member deep ensemble for thyroid nodule classification and selective image-based triage.<n>On pooled out-of-fold TN5000 predictions, the ensemble achieved AUC-ROC 0.9395, AP 0.9715, ECE 0.0088, and Brier score 0.0813.<n>The framework showed strong internal discrimination and calibration, but limited external threshold transportability.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Background: Deep learning models can classify thyroid nodules on ultrasound, but reliable clinical decision support also requires calibrated probabilities, uncertainty estimation, and selective referral, particularly under dataset shift. Methods: We developed a calibrated deterministic five-member deep ensemble for ROI-based thyroid nodule classification and selective image-based triage. TN5000 was used for model development, five-fold cross-validation, member-wise vector-scaling calibration, and fold-specific threshold selection. TN3K served as an independent external dataset-shift evaluation. The framework used ConvNeXt-Tiny with squeeze-and-excitation attention, ensemble-mean malignancy probability, and mutual information (MI) as an ensemble-disagreement score. A three-tier policy assigned images to No-FNA suggestion, FNA recommendation, or radiologist review. Results: On pooled out-of-fold TN5000 predictions, the ensemble achieved AUC-ROC 0.9395, AP 0.9715, ECE 0.0088, and Brier score 0.0813. At 50% nominal MI retention, 7.2% of cases received a No-FNA suggestion, 39.9% an FNA recommendation, and 52.9% radiologist review, with 98.3% No-FNA NPV and 99.83% malignancy capture. On TN3K, AUC-ROC decreased to 0.7870, AP to 0.7254, ECE increased to 0.1899, and Brier score to 0.2281. The frozen TN5000 policy assigned 83.7% to review, 1.0% to No-FNA, and 15.3% to FNA recommendation. No malignant image entered the No-FNA pathway, but FNA-recommendation PPV fell to 76.6%. Conclusion: The framework showed strong internal discrimination and calibration, but limited external threshold transportability. Selective prediction may help identify images unsuitable for automated triage, but local recalibration, threshold validation, and prospective clinical evaluation are required before deployment.

Related papers

• Cross-dataset transportability of pediatric chest X-ray deep learning across three countries: discrimination, calibration, operating-point failure, and limited-label recovery [0.0]
  We evaluated a computational protocol that separately tests discrimination, probability calibration, fixed operatingpoint transport, and limited-label recoverability for pediatric pneumonia classification.<n>Cross-dataset shifts across countries affected ranking, probability alignment, and source-defined decision behavior differently.
  arXiv  Detail & Related papers  (2026-09-04T13:39:48Z)
• A Leakage-Aware Comparative Benchmark of Machine Learning, Deep Learning, and Transformer Models for Reliable Leukemia Detection [0.0]
  Automated classification of acute lymphoblastic leukemia (ALL) from peripheral blood smear images has often reported near-perfect performance on the C-NMC 2019 dataset.<n>We show that such results can be inflated by patient-level data leakage caused by random image-level partitioning.<n>We establish a leakage-aware benchmark under a strict subject-disjoint protocol.
  arXiv  Detail & Related papers  (2026-06-23T04:53:49Z)
• TriALS: Triphasic-Aided Liver Lesion Segmentation Benchmark in Non-Contrast CT [49.93685997940356]
  We describe the TriALS challenge for automated liver lesion segmentation under contrast-limited conditions.<n>The top-performing method achieved a mean venous-phase Dice of 0.754, consistent with human-level performance, yet dropped to 0.57 on NCCT.<n>Algorithm performance was most strongly predicted by training data scale and pre-training strategy.
  arXiv  Detail & Related papers  (2026-05-15T19:23:35Z)
• Opportunistic Bone-Loss Screening from Routine Knee Radiographs Using a Multi-Task Deep Learning Framework with Sensitivity-Constrained Threshold Optimization [9.995968038491128]
  Osteoporosis and osteopenia are often undiagnosed until fragility fractures occur.<n>Knee radiographs are obtained at high volume for osteoarthritis evaluation.<n>We developed STR-Net, a multi-task framework for single-channel grayscale knee radiographs.
  arXiv  Detail & Related papers  (2026-04-22T07:12:04Z)
• A Novel Attention-Augmented Wavelet YOLO System for Real-time Brain Vessel Segmentation on Transcranial Color-coded Doppler [49.03919553747297]
  We propose an AI-powered, real-time CoW auto-segmentation system capable of efficiently capturing cerebral arteries.<n>No prior studies have explored AI-driven cerebrovascular segmentation using Transcranial Color-coded Doppler (TCCD)<n>The proposed AAW-YOLO demonstrated strong performance in segmenting both ipsilateral and contralateral CoW vessels.
  arXiv  Detail & Related papers  (2025-08-19T14:41:22Z)
• ThyroidEffi 1.0: A Cost-Effective System for High-Performance Multi-Class Thyroid Carcinoma Classification [0.0]
  We develop and validate a deep learning system for multi-class thyroid FNAB image classification.<n>Benign, Indeterminate/Suspicious, and Malignant are three key categories directly guiding post-biopsy treatment.<n>The system processed 1000 cases in 30 seconds, demonstrating feasibility on widely accessible hardware.
  arXiv  Detail & Related papers  (2025-04-19T02:13:07Z)
• Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification [52.77024349608834]
  To investigate chest radiograph (CXR) classification performance of vision transformers (ViT) and interpretability of attention-based saliency. ViTs were fine-tuned for lung disease classification using four public data sets: CheXpert, Chest X-Ray 14, MIMIC CXR, and VinBigData. ViTs had comparable CXR classification AUCs compared with state-of-the-art CNNs.
  arXiv  Detail & Related papers  (2023-03-03T12:05:41Z)
• Performance of a deep learning system for detection of referable diabetic retinopathy in real clinical settings [0.0]
  RetCAD v.1.3.1 was developed to automatically detect referable diabetic retinopathy (DR) Analysed the reduction of workload that can be released incorporating this artificial intelligence-based technology.
  arXiv  Detail & Related papers  (2022-05-11T14:59:10Z)
• Osteoporosis Prescreening using Panoramic Radiographs through a Deep Convolutional Neural Network with Attention Mechanism [65.70943212672023]
  Deep convolutional neural network (CNN) with an attention module can detect osteoporosis on panoramic radiographs. dataset of 70 panoramic radiographs (PRs) from 70 different subjects of age between 49 to 60 was used.
  arXiv  Detail & Related papers  (2021-10-19T00:03:57Z)
• Controlling False Positive/Negative Rates for Deep-Learning-Based Prostate Cancer Detection on Multiparametric MR images [58.85481248101611]
  We propose a novel PCa detection network that incorporates a lesion-level cost-sensitive loss and an additional slice-level loss based on a lesion-to-slice mapping function. Our experiments based on 290 clinical patients concludes that 1) The lesion-level FNR was effectively reduced from 0.19 to 0.10 and the lesion-level FPR was reduced from 1.03 to 0.66 by changing the lesion-level cost.
  arXiv  Detail & Related papers  (2021-06-04T09:51:27Z)
• Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes [64.21642241351857]
  We curated and analyzed a chest computed tomography (CT) data set of 36,316 volumes from 19,993 unique patients. We developed a rule-based method for automatically extracting abnormality labels from free-text radiology reports. We also developed a model for multi-organ, multi-disease classification of chest CT volumes.
  arXiv  Detail & Related papers  (2020-02-12T00:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.