Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging

• URL: http://arxiv.org/abs/2606.02339v1
• Date: Mon, 01 Jun 2026 14:48:16 GMT
• Title: Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging
• Abstract summary: Entropy minimization is the dominant objective for test-time adaptation, yet its failure mode, model collapse, remains poorly understood.<n>We propose Distribution Shift Bias Reduction (DSBR), a bias-correcting objective that specifically targets this failure mode.<n>We find that DSBR consistently stabilizes test-time adaptation, prevents model collapse, and matches or outperforms state-of-the-art methods.
• Score: 2.651701169890824
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Entropy minimization (EM) is the dominant objective for test-time adaptation, yet its failure mode, model collapse, remains poorly understood. In this work, we show that distribution shifts can cause feature clusters corresponding to distinct classes in the model's representation space to merge, while the decision boundary remains fixed. This induces a systematic skew in the predicted class distribution, referred to as prediction bias. Prediction bias refers to a shift in the predicted class distribution, with some classes overrepresented and others suppressed. We show that entropy minimization amplifies this prediction bias by tightening the existing clusters, reinforcing the incorrect groupings until all predictions collapse to a trivial solution. Next, to demonstrate the significance of prediction bias and mitigate it, we further propose Distribution Shift Bias Reduction (DSBR), a bias-correcting objective that specifically targets this failure mode by equalizing the contribution of each predicted class to the unsupervised entropy minimization loss. To study this failure mode, we design suitable adaptation settings using four medical-imaging datasets and additionally evaluate on ImageNet-C. We find that DSBR consistently stabilizes test-time adaptation, prevents model collapse, and matches or outperforms state-of-the-art methods. Moreover, DSBR operates solely at test-time.

Related papers

• Rethinking the Test-Time Prompt Tuning Objective from the Perspective of Calibration [14.593876228926185]
  Test-time prompt tuning (TPT) has emerged as a powerful paradigm, refining prompts for each test sample via entropy minimization (EM) over multiple augmented views.<n>We propose a new objective that replaces the conventional EM loss by aligning the original-view prediction with a target distribution derived from augmented views via cross-entropy.<n>We apply confidence-aware temperature scaling to each augmented-view prediction according to its confidence, sharpening confident predictions while softening uncertain ones.
  arXiv  Detail & Related papers  (2026-08-31T04:33:05Z)
• Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction [74.05279072166697]
  Exponential-Gaussian Mixture Network (EGMN) models the full conditional WT distribution.<n>EGMN is vulnerable to variance collapse, component redundancy, and inactive components.<n>We propose a Hierarchical Exponential-Gaussian Mixture (HEGM) model that addresses these failure modes.
  arXiv  Detail & Related papers  (2026-08-24T15:06:18Z)
• Mind the Residual Gap: Probabilistic Downscaling under Real-World Bias [11.400279490725042]
  Probabilistic downscaling is a central challenge to atmospheric science, climate modeling, and other multiscale physical systems.<n>A widely used paradigm decomposes the problem into a deterministic mean predictor followed by a residual generator.<n>We show that the root cause is residual target miscalibration, the residual distribution induced during training differs systematically from the one required at test time due to downscaling bias.
  arXiv  Detail & Related papers  (2026-06-29T18:47:34Z)
• Dual Strategies for Test-Time Adaptation [37.36806938657306]
  Conventional test-time adaptation approaches typically adapt the model using only a small fraction of test samples.<n>This paper introduces DualTTA, a novel framework that improves performance under distribution shifts by utilizing a larger and more diverse set of test samples.
  arXiv  Detail & Related papers  (2026-04-19T17:04:36Z)
• Micro-Diffusion Compression - Binary Tree Tweedie Denoising for Online Probability Estimation [0.0]
  We introduce a micro-diffusion denoising layer for improving probability estimates produced by adaptive statistical models.<n>Midicoth combines five fully online components: an adaptive PPM model, a long-range match model, a trie-based word model, a high-order context model, and the micro-diffusion denoiser applied as the final stage.
  arXiv  Detail & Related papers  (2026-03-09T16:59:24Z)
• Nonparametric Distribution Regression Re-calibration [3.0204520109309847]
  Minimizing overall prediction error encourages models to prioritize informativeness over calibration.<n>In safety-critical settings, trustworthy uncertainty estimates are often more valuable than narrow intervals.<n>We propose a novel non-parametric re-calibration algorithm based on conditional kernel mean embeddings.
  arXiv  Detail & Related papers  (2026-02-13T11:48:43Z)
• Causal Inference as Distribution Adaptation: Optimizing ATE Risk under Propensity Uncertainty [0.0]
  We reframing ATE estimation as a textitdomain adaptation problem under distribution shift.<n>We propose the textbfJoint Robust Estimator (JRE) to train outcome models jointly.
  arXiv  Detail & Related papers  (2025-12-19T21:40:46Z)
• Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization [85.50560211492898]
  Test-time adaptation (TTA) may fail to improve or even harm the model performance when test data have mixed distribution shifts.<n>This is often a key obstacle preventing existing TTA methods from being deployed in the real world.<n>We propose a sharpness-aware and reliable entropy minimization method, called SAR, for stabilizing TTA from two aspects.
  arXiv  Detail & Related papers  (2025-09-05T10:03:00Z)
• BayesTTA: Continual-Temporal Test-Time Adaptation for Vision-Language Models via Gaussian Discriminant Analysis [41.09181390655176]
  Vision-language models (VLMs) such as CLIP achieve strong zero-shot recognition but degrade significantly under textittemporally evolving distribution shifts common in real-world scenarios.<n>We formalize this practical problem as textitContinual-Temporal Test-Time Adaptation (CT-TTA), where test distributions evolve gradually over time.<n>We propose textitBayesTTA, a Bayesian adaptation framework that enforces temporally consistent predictions and dynamically aligns visual representations.
  arXiv  Detail & Related papers  (2025-07-11T14:02:54Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions.<n>We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance.<n>Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [65.21599711087538]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample.<n>Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications.<n>We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Source-Free Unsupervised Domain Adaptation with Hypothesis Consolidation of Prediction Rationale [53.152460508207184]
  Source-Free Unsupervised Domain Adaptation (SFUDA) is a challenging task where a model needs to be adapted to a new domain without access to target domain labels or source domain data. This paper proposes a novel approach that considers multiple prediction hypotheses for each sample and investigates the rationale behind each hypothesis. To achieve the optimal performance, we propose a three-step adaptation process: model pre-adaptation, hypothesis consolidation, and semi-supervised learning.
  arXiv  Detail & Related papers  (2024-02-02T05:53:22Z)
• Patch-level Gaze Distribution Prediction for Gaze Following [49.93340533068501]
  We introduce the patch distribution prediction ( PDP) method for gaze following training. We show that our model regularizes the MSE loss by predicting better heatmap distributions on images with larger annotation variances. Experiments show that our model bridging the gap between the target prediction and in/out prediction subtasks, showing a significant improvement on both subtasks on public gaze following datasets.
  arXiv  Detail & Related papers  (2022-11-20T19:25:15Z)
• Ensembling over Classifiers: a Bias-Variance Perspective [13.006468721874372]
  We build upon the extension to the bias-variance decomposition by Pfau (2013) in order to gain crucial insights into the behavior of ensembles of classifiers. We show that conditional estimates necessarily incur an irreducible error. Empirically, standard ensembling reducesthe bias, leading us to hypothesize that ensembles of classifiers may perform well in part because of this unexpected reduction.
  arXiv  Detail & Related papers  (2022-06-21T17:46:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.