On Cross-Validation for Hyperparameter Optimization of Deep Learning Image Classifiers

• URL: http://arxiv.org/abs/2608.14705v1
• Date: Mon, 10 Aug 2026 22:44:42 GMT
• Title: On Cross-Validation for Hyperparameter Optimization of Deep Learning Image Classifiers
• Abstract summary: Hyper optimization (HPO) can materially affect the performance of deep learning (DL) image classifiers.<n>We compare three HPO protocols in terms of absolute performance- parameter errorestimation (AEE)<n>Every point estimate favored cross-validation over both holdout protocols on medical datasets.<n>For small-sample medical image classification, we recommend cross-validation-based HPO.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperparameter optimization (HPO) can materially affect the performance of deep learning (DL) image classifiers, but there is little empirical guidance on how to derive the validation signal that drives it, especially for the small sample sizes common in fields such as medical imaging. We compared three HPO protocols in terms of {\em absolute performance-estimation error} (AEE; the absolute difference between the winning configuration's validation AUROC and its test AUROC): fixed holdout (F), reshuffled holdout (R), and 5-fold cross-validation (C). The search space, sampler, training procedure, architecture, and test set were held identical across protocols. We evaluated the protocols on three public datasets spanning two regimes: binary medical imaging (RSNA pneumonia radiographs and binarized HAM10000 skin lesions) and 200-class natural imaging (Tiny ImageNet), across a range of development set sizes $n$ and two backbones (ResNet-18 on all datasets, Vision Transformer (ViT-S/16) on RSNA). On the medical datasets, every point estimate favored cross-validation over both holdout protocols, with reductions in AEE largest at small sample sizes and diminishing as $n$ increased. This pattern remained robust under conservative family-wise adjustment. On Tiny ImageNet, AEE was negligible under all three protocols. Test AUROC was generally similar among protocols. Fixed holdout had lower mean AEE than reshuffled holdout in 11 of 12 medical conditions, although this secondary finding was less uniformly supported. For small-sample medical image classification, we recommend cross-validation-based HPO when computational resources permit because it trades additional computation for a more reliable development-time estimate of subsequent test performance.

Related papers

• Beyond Static Anchors: Bounded Prototype Conditioning for Language-Free Medical Anomaly Detection [52.986874008247554]
  We propose ReCAP, a language-free framework that replaces static anchors with input-conditioned visual prototypes.<n>ReCAP re-centers separated normal and abnormal prototypes for each image through a bounded gated modulation.<n>It achieves the best image-level AUROC on all zero-shot and 23 of 24 few-shot settings, and the best zero-shot pixel-level AUROC on all three segmentation datasets.
  arXiv  Detail & Related papers  (2026-08-01T04:46:55Z)
• I Can't Believe TTA Is Not Better: When Test-Time Augmentation Hurts Medical Image Classification [0.0]
  Test-time augmentation (TTA) is widely assumed to improve classification accuracy in medical imaging.<n>We present a systematic empirical study challenging this assumption across three MedMNIST v2 benchmarks and four architectures.<n>Our principal finding is that TTA with standard augmentation pipelines consistently degrades accuracy relative to single-pass inference.
  arXiv  Detail & Related papers  (2026-04-06T23:52:10Z)
• Maximizing T2-Only Prostate Cancer Localization from Expected Diffusion Weighted Imaging [68.57480103120561]
  Multiparametric MRI is increasingly recommended as a first-line noninvasive approach to detect and localize prostate cancer.<n>Early machine learning attempts using only T2w images have shown promising diagnostic performance in segmenting radiologist-annotated lesions.<n>This work investigates an arguably more challenging application using only T2w at inference, but to localize individual cancers based on independent histopathology labels.
  arXiv  Detail & Related papers  (2026-04-01T14:50:18Z)
• A Heterogeneous Ensemble for Multi-Center COVID-19 Classification from Chest CT Scans [0.39998518782208775]
  We present a heterogeneous ensemble of nine models spanning three inference paradigms.<n>The ensemble achieves an average macro F1 of 0.9280 across four hospital centres, outperforming the best single model.
  arXiv  Detail & Related papers  (2026-03-15T21:34:59Z)
• T3: Test-Time Model Merging in VLMs for Zero-Shot Medical Imaging Analysis [15.624549727053475]
  Existing model-merging techniques fail to deliver consistent gains across diverse medical modalities.<n>We introduce Test-Time Task adaptive merging (T3), a backpropagation-free framework that computes per-sample coefficients.<n>We present a rigorous cross-evaluation protocol spanning in-domain, base-to-novel, and corruptions across four modalities.
  arXiv  Detail & Related papers  (2025-10-31T08:05:40Z)
• The Protocol Genome A Self Supervised Learning Framework from DICOM Headers [0.5076419064097734]
  Protocol Genome is a self-supervised learning system that learns correlations from DICOM headers.<n>Our method also improves calibration and robustness across modalities (CT, MRI, CXR) and vendors.
  arXiv  Detail & Related papers  (2025-09-03T07:38:32Z)
• AHDMIL: Asymmetric Hierarchical Distillation Multi-Instance Learning for Fast and Accurate Whole-Slide Image Classification [51.525891360380285]
  AHDMIL is an Asymmetric Hierarchical Distillation Multi-Instance Learning framework.<n>It eliminates irrelevant patches through a two-step training process.<n>It consistently outperforms previous state-of-the-art methods in both classification performance and inference speed.
  arXiv  Detail & Related papers  (2025-08-07T07:47:16Z)
• Domain Transfer Through Image-to-Image Translation for Uncertainty-Aware Prostate Cancer Classification [42.75911994044675]
  We present a novel approach for unpaired image-to-image translation of prostate MRIs and an uncertainty-aware training approach for classifying clinically significant PCa. Our approach involves a novel pipeline for translating unpaired 3.0T multi-parametric prostate MRIs to 1.5T, thereby augmenting the available training data. Our experiments demonstrate that the proposed method significantly improves the Area Under ROC Curve (AUC) by over 20% compared to the previous work.
  arXiv  Detail & Related papers  (2023-07-02T05:26:54Z)
• Recurrence With Correlation Network for Medical Image Registration [66.63200823918429]
  We present Recurrence with Correlation Network (RWCNet), a medical image registration network with multi-scale features and a cost volume layer. We demonstrate that these architectural features improve medical image registration accuracy in two image registration datasets.
  arXiv  Detail & Related papers  (2023-02-05T02:41:46Z)
• Feature-enhanced Adversarial Semi-supervised Semantic Segmentation Network for Pulmonary Embolism Annotation [6.142272540492936]
  This study established a feature-enhanced adversarial semi-supervised semantic segmentation model to automatically annotate pulmonary embolism lesion areas. In current studies, all of the PEA image segmentation methods are trained by supervised learning. This study proposed a semi-supervised learning method to make the model applicable to different datasets by adding a small amount of unlabeled images.
  arXiv  Detail & Related papers  (2022-04-08T04:21:02Z)
• Cross-Site Severity Assessment of COVID-19 from CT Images via Domain Adaptation [64.59521853145368]
  Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features.
  arXiv  Detail & Related papers  (2021-09-08T07:56:51Z)
• Collaborative Boundary-aware Context Encoding Networks for Error Map Prediction [65.44752447868626]
  We propose collaborative boundaryaware context encoding networks called AEP-Net for error prediction task. Specifically, we propose a collaborative feature transformation branch for better feature fusion between images and masks, and precise localization of error regions. The AEP-Net achieves an average DSC of 0.8358, 0.8164 for error prediction task, and shows a high Pearson correlation coefficient of 0.9873.
  arXiv  Detail & Related papers  (2020-06-25T12:42:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.