ASAP: Unsupervised Post-training with Label Distribution Shift Adaptive Learning Rate

• URL: http://arxiv.org/abs/2508.13445v1
• Date: Tue, 19 Aug 2025 01:59:24 GMT
• Title: ASAP: Unsupervised Post-training with Label Distribution Shift Adaptive Learning Rate
• Authors: Heewon Park, Mugon Joe, Miru Kim, Minhae Kwon,
• Abstract summary: ASAP adjusts the learning rate by computing the cosine distance between current and previous unlabeled outputs and mapping it within a bounded range.<n>Experiments show ASAP consistently improves accuracy and efficiency, making it practical for unsupervised model adaptation.
• Score: 3.187381965457262
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In real-world applications, machine learning models face online label shift, where label distributions change over time. Effective adaptation requires careful learning rate selection: too low slows adaptation and too high causes instability. We propose ASAP (Adaptive Shift Aware Post-training), which dynamically adjusts the learning rate by computing the cosine distance between current and previous unlabeled outputs and mapping it within a bounded range. ASAP requires no labels, model ensembles, or past inputs, using only the previous softmax output for fast, lightweight adaptation. Experiments across multiple datasets and shift scenarios show ASAP consistently improves accuracy and efficiency, making it practical for unsupervised model adaptation.

Related papers

• Guess-and-Learn (G&L): Measuring the Cumulative Error Cost of Cold-Start Adaptation [0.11102988539107704]
  Evaluation of machine learning models typically emphasizes final accuracy, overlooking the cost of adaptation cumulative errors incurred while learning from scratch.<n>Guess-and- Learn (G&L) v1.0 addresses this gap by measuring cold-start adaptability - the total mistakes a model makes while sequentially labeling an unlabeled dataset.
  arXiv  Detail & Related papers  (2025-08-29T00:13:02Z)
• Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptation [16.657929958093824]
  Test-time adaptation is an approach to adjust models to a new data distribution during inference. Test-time batch normalization is a simple and popular method that achieved compelling performance on domain shift benchmarks. We propose to tackle this challenge by only selectively adapting channels in a deep network, minimizing drastic adaptation that is sensitive to label shifts.
  arXiv  Detail & Related papers  (2024-02-07T15:41:01Z)
• Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening [51.34904967046097]
  We introduce an extension to the selective synaptic dampening unlearning method that removes the need for parameter tuning. We demonstrate the performance of this extension, adaptive selective synaptic dampening (ASSD) on various ResNet18 and Vision Transformer unlearning tasks. The application of this approach is particularly compelling in industrial settings, such as supply chain management.
  arXiv  Detail & Related papers  (2024-02-06T14:04:31Z)
• Online Feature Updates Improve Online (Generalized) Label Shift Adaptation [51.328801874640675]
  Our novel method, Online Label Shift adaptation with Online Feature Updates (OLS-OFU), leverages self-supervised learning to refine the feature extraction process. By carefully designing the algorithm, OLS-OFU maintains the similar online regret convergence to the results in the literature while taking the improved features into account.
  arXiv  Detail & Related papers  (2024-02-05T22:03:25Z)
• Online Label Shift: Optimal Dynamic Regret meets Practical Algorithms [33.61487362513345]
  This paper focuses on supervised and unsupervised online label shift, where the class marginals $Q(y)$ varies but the class-conditionals $Q(x|y)$ remain invariant. In the unsupervised setting, our goal is to adapt a learner, trained on some offline labeled data, to changing label distributions given unlabeled online data. We develop novel algorithms that reduce the adaptation problem to online regression and guarantee optimal dynamic regret without any prior knowledge of the extent of drift in the label distribution.
  arXiv  Detail & Related papers  (2023-05-31T05:39:52Z)
• ASPEST: Bridging the Gap Between Active Learning and Selective Prediction [56.001808843574395]
  Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain. Active learning aims to lower the overall labeling effort, and hence human dependence, by querying the most informative examples. In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain.
  arXiv  Detail & Related papers  (2023-04-07T23:51:07Z)
• Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification [57.36281142038042]
  We present a new adaptive block called Contextual Squeeze-and-Excitation (CaSE) that adjusts a pretrained neural network on a new task to significantly improve performance. We also present a new training protocol based on Coordinate-Descent called UpperCaSE that exploits meta-trained CaSE blocks and fine-tuning routines for efficient adaptation.
  arXiv  Detail & Related papers  (2022-06-20T15:25:08Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Online Adaptation to Label Distribution Shift [37.91472909652585]
  We propose adaptation algorithms inspired by classical online learning techniques such as Follow The Leader (FTL) and Online Gradient Descent (OGD) We empirically verify our findings under both simulated and real world label distribution shifts and show that OGD is particularly effective and robust to a variety of challenging label shift scenarios.
  arXiv  Detail & Related papers  (2021-07-09T16:12:19Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.