Order Matters: Improving Domain Adaptation by Reordering Data

• URL: http://arxiv.org/abs/2605.05084v1
• Date: Wed, 06 May 2026 16:20:24 GMT
• Title: Order Matters: Improving Domain Adaptation by Reordering Data
• Abstract summary: Domain shift is a key challenge in deploying machine learning models to the real world.<n>Unsupervised domain adaptation aims to address this by minimising domain discrepancy during training.<n>This paper proposes a novel unbiased variance reduction technique which reduces the discrepancy estimation error.
• Score: 1.7188280334580195
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain shift remains a key challenge in deploying machine learning models to the real world. Unsupervised domain adaptation (UDA) aims to address this by minimising domain discrepancy during training, but the discrepancy estimates suffer from high variance in stochastic settings, which can stifle the theoretical benefits of the method. This paper proposes Optimal Reordering of Data for Error-Reduced Estimation of Discrepancy (ORDERED), a novel unbiased stochastic variance reduction technique which reduces the discrepancy estimation error by optimising the order in which the training data are sampled. We consider two specific domain discrepancy losses (correlation alignment and the maximum mean discrepancy), formulate their stochastic estimation error as a function of the data sampling order, and propose a practical optimisation algorithm. Our simulations demonstrate reduced variance compared to related methods, and experiments on two domain shift image classification benchmarks show improved target domain accuracy.

Related papers

• Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting [63.86964530167364]
  This paper presents a Bayesian domain weighting method to infer the weights from a Dirichlet distribution via introducing Gamma prior information learned from observations.<n> Experimental results demonstrate that proposed method could achieve stable and efficient domain weights learning, and identifies optimal mixtures while consuming substantially less data than search-based function-fitting methods, revitalizing optimization-based domain weighting for large-scale applications.
  arXiv  Detail & Related papers  (2026-07-30T09:41:02Z)
• Variance-reduced Domain Adaptation using Paired Sampling [2.0305676256390934]
  Correlation alignment and the maximum mean discrepancy are widely used for unsupervised domain adaptation.<n>High variance in these losses has been shown to undermine their effectiveness in minibatch settings.<n>This paper proposes Paired Sampling for Domain Adaptation (PSDA), a novel SVR technique tailored to such objectives.
  arXiv  Detail & Related papers  (2026-07-22T16:58:45Z)
• Understanding the Generalization of Bilevel Programming in Hyperparameter Optimization: A Tale of Bias-Variance Decomposition [53.68517860700599]
  We propose an ensemble hypergradient strategy to reduce the variance in HPO algorithms effectively.<n>We establish a connection between excess error and hypergradient estimation, offering some understanding of empirical observations.
  arXiv  Detail & Related papers  (2026-02-20T02:52:13Z)
• Improving Domain Generalization in Contrastive Learning using Adaptive Temperature Control [6.29137812995328]
  Self-supervised pre-training with contrastive learning is a powerful method for learning from sparsely labeled data.<n>Performance can drop considerably when there is a shift in the distribution of data from training to test time.<n>We present a new method for contrastive learning that incorporates domain labels to increase the domain invariance of learned representations.
  arXiv  Detail & Related papers  (2026-01-12T17:32:24Z)
• Variance Matters: Improving Domain Adaptation via Stratified Sampling [1.7188280334580195]
  This paper proposes Variance-Reduced Adaptation via Domain Stratified Sampling (VaRDASS)<n>VaRDASS is the first specialised variance reduction technique for UDA.<n> Experiments on three domain shift datasets improved discrepancy estimation accuracy and target domain performance.
  arXiv  Detail & Related papers  (2025-12-04T20:01:04Z)
• Target-specific Adaptation and Consistent Degradation Alignment for Cross-Domain Remaining Useful Life Prediction [24.676267074769537]
  We propose a novel domain adaptation approach for cross-domain RUL prediction named TACDA.<n>We develop a novel clustering and pairing strategy for consistent alignment between similar degradation stages.<n>Our results demonstrate the remarkable performance of our proposed TACDA method.
  arXiv  Detail & Related papers  (2025-12-02T10:15:14Z)
• Improving Distribution Alignment with Diversity-based Sampling [0.0]
  Domain shifts are ubiquitous in machine learning, and can substantially degrade a model's performance when deployed to real-world data. This paper proposes to improve these estimates by inducing diversity in each sampled minibatch. It simultaneously balances the data and reduces the variance of the gradients, thereby enhancing the model's generalisation ability.
  arXiv  Detail & Related papers  (2024-10-05T17:26:03Z)
• Enhanced Online Test-time Adaptation with Feature-Weight Cosine Alignment [7.991720491452191]
  Online Test-Time Adaptation (OTTA) has emerged as an effective strategy to handle distributional shifts. This paper introduces a novel cosine alignment optimization approach with a dual-objective loss function. Our method outperforms state-of-the-art techniques and sets a new benchmark in multiple datasets.
  arXiv  Detail & Related papers  (2024-05-12T05:57:37Z)
• Bi-discriminator Domain Adversarial Neural Networks with Class-Level Gradient Alignment [87.8301166955305]
  We propose a novel bi-discriminator domain adversarial neural network with class-level gradient alignment. BACG resorts to gradient signals and second-order probability estimation for better alignment of domain distributions. In addition, inspired by contrastive learning, we develop a memory bank-based variant, i.e. Fast-BACG, which can greatly shorten the training process.
  arXiv  Detail & Related papers  (2023-10-21T09:53:17Z)
• Domain-Specific Risk Minimization for Out-of-Distribution Generalization [104.17683265084757]
  We first establish a generalization bound that explicitly considers the adaptivity gap. We propose effective gap estimation methods for guiding the selection of a better hypothesis for the target. The other method is minimizing the gap directly by adapting model parameters using online target samples.
  arXiv  Detail & Related papers  (2022-08-18T06:42:49Z)
• Selective Pseudo-Labeling with Reinforcement Learning for Semi-Supervised Domain Adaptation [116.48885692054724]
  We propose a reinforcement learning based selective pseudo-labeling method for semi-supervised domain adaptation. We develop a deep Q-learning model to select both accurate and representative pseudo-labeled instances. Our proposed method is evaluated on several benchmark datasets for SSDA, and demonstrates superior performance to all the comparison methods.
  arXiv  Detail & Related papers  (2020-12-07T03:37:38Z)
• Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation [120.69747175899421]
  Optimal Transport (OT) distances such as Wasserstein have been used in several areas such as GANs and domain adaptation. We propose a computationally-efficient dual form of the robust OT optimization that is amenable to modern deep learning applications. Our approach can train state-of-the-art GAN models on noisy datasets corrupted with outlier distributions.
  arXiv  Detail & Related papers  (2020-10-12T17:13:40Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.