Learning Robustness at Test-Time from a Non-Robust Teacher

• URL: http://arxiv.org/abs/2604.11590v1
• Date: Mon, 13 Apr 2026 15:05:21 GMT
• Title: Learning Robustness at Test-Time from a Non-Robust Teacher
• Abstract summary: This work studies how adversarial training strategies behave when integrated into adaptation schemes for the unsupervised test-time setting.<n>We propose a label-free framework that uses the predictions of a non-robust teacher model as a semantic anchor for both the clean and adversarial objectives.
• Score: 3.723585565253798
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Nowadays, pretrained models are increasingly used as general-purpose backbones and adapted at test-time to downstream environments where target data are scarce and unlabeled. While this paradigm has proven effective for improving clean accuracy on the target domain, adversarial robustness has received far less attention, especially when the original pretrained model is not explicitly designed to be robust. This raises a practical question: \emph{can a pretrained, non-robust model be adapted at test-time to improve adversarial robustness on a target distribution?} To face this question, this work studies how adversarial training strategies behave when integrated into adaptation schemes for the unsupervised test-time setting, where only a small set of unlabeled target samples is available. It first analyzes how classical adversarial training formulations can be extended to this scenario, showing that straightforward distillation-based adaptations remain unstable and highly sensitive to hyperparameter tuning, particularly when the teacher itself is non-robust. To address these limitations, the work proposes a label-free framework that uses the predictions of a non-robust teacher model as a semantic anchor for both the clean and adversarial objectives during adaptation. We further provide theoretical insights showing that our formulation yields a more stable alternative to the self-consistency-based regularization commonly used in classical adversarial training. Experiments evaluate the proposed approach on CIFAR-10 and ImageNet under induced photometric transformations. The results support the theoretical insights by showing that the proposed approach achieves improved optimization stability, lower sensitivity to parameter choices, and a better robustness-accuracy trade-off than existing baselines in this post-deployment test-time setting.

Related papers

• Smart predict-then-robustly-optimize [0.5161531917413708]
  We propose and study a robust variant of the smart predict-then-optimize approach.<n>Our framework consistently yields significant performance improvements over standard methods.
  arXiv  Detail & Related papers  (2026-07-23T19:37:17Z)
• The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation [67.26315138466312]
  We propose a calibration-aware OPD framework, CaOPD, that estimates empirical confidence from model rollouts.<n>Our findings highlight that capability distillation does not imply confidence, and that confidence should be treated as an essential objective in post-training.
  arXiv  Detail & Related papers  (2026-04-18T04:43:40Z)
• Adversarial Robustification via Text-to-Image Diffusion Models [56.37291240867549]
  Adrial robustness has been conventionally believed as a challenging property to encode for neural networks. We develop a scalable and model-agnostic solution to achieve adversarial robustness without using any data.
  arXiv  Detail & Related papers  (2024-07-26T10:49:14Z)
• Selective Learning: Towards Robust Calibration with Dynamic Regularization [79.92633587914659]
  Miscalibration in deep learning refers to there is a discrepancy between the predicted confidence and performance. We introduce Dynamic Regularization (DReg) which aims to learn what should be learned during training thereby circumventing the confidence adjusting trade-off.
  arXiv  Detail & Related papers  (2024-02-13T11:25:20Z)
• Robust Feature Inference: A Test-time Defense Strategy using Spectral Projections [12.807619042576018]
  We propose a novel test-time defense strategy called Robust Feature Inference (RFI) RFI is easy to integrate with any existing (robust) training procedure without additional test-time computation. We show that RFI improves robustness across adaptive and transfer attacks consistently.
  arXiv  Detail & Related papers  (2023-07-21T16:18:58Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• Semantic Self-adaptation: Enhancing Generalization with a Single Sample [45.111358665370524]
  We propose a self-adaptive approach for semantic segmentation. It fine-tunes the parameters of convolutional layers to the input image using consistency regularization. Our empirical study suggests that self-adaptation may complement the established practice of model regularization at training time.
  arXiv  Detail & Related papers  (2022-08-10T12:29:01Z)
• Learning Domain Adaptive Object Detection with Probabilistic Teacher [93.76128726257946]
  We present a simple yet effective framework, termed as Probabilistic Teacher (PT) PT aims to capture the uncertainty of unlabeled target data from a gradually evolving teacher and guides the learning of a student in a mutually beneficial manner. We also present a novel Entropy Focal Loss (EFL) to further facilitate the uncertainty-guided self-training.
  arXiv  Detail & Related papers  (2022-06-13T16:24:22Z)
• MEMO: Test Time Robustness via Adaptation and Augmentation [131.28104376280197]
  We study the problem of test time robustification, i.e., using the test input to improve model robustness. Recent prior works have proposed methods for test time adaptation, however, they each introduce additional assumptions. We propose a simple approach that can be used in any test setting where the model is probabilistic and adaptable.
  arXiv  Detail & Related papers  (2021-10-18T17:55:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.