I Can't Believe TTA Is Not Better: When Test-Time Augmentation Hurts Medical Image Classification

• URL: http://arxiv.org/abs/2604.09697v1
• Date: Mon, 06 Apr 2026 23:52:10 GMT
• Title: I Can't Believe TTA Is Not Better: When Test-Time Augmentation Hurts Medical Image Classification
• Abstract summary: Test-time augmentation (TTA) is widely assumed to improve classification accuracy in medical imaging.<n>We present a systematic empirical study challenging this assumption across three MedMNIST v2 benchmarks and four architectures.<n>Our principal finding is that TTA with standard augmentation pipelines consistently degrades accuracy relative to single-pass inference.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Test-time augmentation (TTA)--aggregating predictions over multiple augmented copies of a test input--is widely assumed to improve classification accuracy, particularly in medical imaging where it is routinely deployed in production systems and competition solutions. We present a systematic empirical study challenging this assumption across three MedMNIST v2 benchmarks and four architectures spanning three orders of magnitude in parameter count (21K to 11M). Our principal finding is that TTA with standard augmentation pipelines consistently degrades accuracy relative to single-pass inference, with drops as severe as 31.6 percentage points for ResNet-18 on pathology images. This degradation affects all architectures, including convolutional models, and worsens with more augmented views. The sole exception is ResNet-18 on dermatology images, which gains a modest +1.6%. We identify the distribution shift between augmented and training-time inputs--amplified by batch normalization statistics mismatch--as the primary mechanism. Our ablation studies show that augmentation strategy matters critically: intensity-only augmentations preserve more performance than geometric transforms, and including the original unaugmented image partially mitigates but does not eliminate the accuracy drop. These findings serve as a cautionary note for practitioners: TTA should not be applied as a default post-hoc improvement but must be validated on the specific model-dataset combination.

Related papers

• On Cross-Validation for Hyperparameter Optimization of Deep Learning Image Classifiers [0.0]
  Hyper optimization (HPO) can materially affect the performance of deep learning (DL) image classifiers.<n>We compare three HPO protocols in terms of absolute performance- parameter errorestimation (AEE)<n>Every point estimate favored cross-validation over both holdout protocols on medical datasets.<n>For small-sample medical image classification, we recommend cross-validation-based HPO.
  arXiv  Detail & Related papers  (2026-08-10T22:44:42Z)
• Robust by Design: A Continuous Monitoring and Data Integration Framework for Medical AI [0.779547724026871]
  We propose an autonomous continuous monitoring and data integration framework that maintains robust performance over time.<n>Our method uses multi-metric feature analysis and Monte Carlo dropout-based uncertainty gating to decide when to retrain on new data.<n>In experiments with a ResNet18 ensemble on a multi-center dataset, the framework prevents performance degradation.
  arXiv  Detail & Related papers  (2026-04-10T06:22:00Z)
• Rethinking Long-tailed Dataset Distillation: A Uni-Level Framework with Unbiased Recovery and Relabeling [105.8570596633629]
  We rethink long-tailed dataset distillation by revisiting the limitations of trajectory-based methods.<n>We adopt the statistical alignment perspective to jointly model bias and restore fair supervision.<n>Our approach improves top-1 accuracy by 15.6% on CIFAR-100-LT and 11.8% on Tiny-ImageNet-LT.
  arXiv  Detail & Related papers  (2025-11-24T07:57:01Z)
• T3: Test-Time Model Merging in VLMs for Zero-Shot Medical Imaging Analysis [15.624549727053475]
  Existing model-merging techniques fail to deliver consistent gains across diverse medical modalities.<n>We introduce Test-Time Task adaptive merging (T3), a backpropagation-free framework that computes per-sample coefficients.<n>We present a rigorous cross-evaluation protocol spanning in-domain, base-to-novel, and corruptions across four modalities.
  arXiv  Detail & Related papers  (2025-10-31T08:05:40Z)
• MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation [55.37355146924576]
  MedSeqFT is a sequential fine-tuning framework for medical image analysis.<n>It adapts pre-trained models to new tasks while refining their representational capacity.<n>It consistently outperforms state-of-the-art fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-09-07T15:22:53Z)
• Calibrated and Robust Foundation Models for Vision-Language and Medical Image Tasks Under Distribution Shift [2.292525568003776]
  Foundation models like CLIP and SAM have advanced computer vision and medical imaging via low-shot transfer learning, aiding CADD with limited data.<n>We propose textbfStaRFM, a fusion of Fisher information penalty (FIP) and confidence misalignment penalty (CMP) tackling both challenges.<n>It applies FIP, extended to 3D via patch-wise regularization, to reduce embedding shift, and CMP, reformulated for voxel-level predictions, to segmentation uncertainty.
  arXiv  Detail & Related papers  (2025-07-12T09:39:07Z)
• Brain Tumor Classification on MRI in Light of Molecular Markers [56.99710477905796]
  Co-deletion of the 1p/19q gene is associated with clinical outcomes in low-grade gliomas.<n>This study aims to utilize a specially MRI-based convolutional neural network for brain cancer detection.
  arXiv  Detail & Related papers  (2024-09-29T07:04:26Z)
• Medical Image Segmentation with InTEnt: Integrated Entropy Weighting for Single Image Test-Time Adaptation [6.964589353845092]
  Test-time adaptation (TTA) refers to adapting a trained model to a new domain during testing. Here, we propose to adapt a medical image segmentation model with only a single unlabeled test image. Our method, validated on 24 source/target domain splits across 3 medical image datasets surpasses the leading method by 2.9% Dice coefficient on average.
  arXiv  Detail & Related papers  (2024-02-14T22:26:07Z)
• Cross-Site Severity Assessment of COVID-19 from CT Images via Domain Adaptation [64.59521853145368]
  Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features.
  arXiv  Detail & Related papers  (2021-09-08T07:56:51Z)
• Vision Transformers for femur fracture classification [59.99241204074268]
  The Vision Transformer (ViT) was able to correctly predict 83% of the test images. Good results were obtained in sub-fractures with the largest and richest dataset ever.
  arXiv  Detail & Related papers  (2021-08-07T10:12:42Z)
• COVID-19 Classification Using Staked Ensembles: A Comprehensive Analysis [0.0]
  COVID-19, increasing with a massive mortality rate, led to the WHO declaring it as a pandemic. It is crucial to perform efficient and fast diagnosis. The reverse transcript polymerase chain reaction (RTPCR) test is conducted to detect the presence of SARS-CoV-2. Instead chest CT (or Chest X-ray) can be used for a fast and accurate diagnosis.
  arXiv  Detail & Related papers  (2020-10-07T07:43:57Z)
• Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning [91.3755431537592]
  We propose the use of Multi-Source Transfer Learning to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
  arXiv  Detail & Related papers  (2020-09-22T11:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.