DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalization

• URL: http://arxiv.org/abs/2601.19394v1
• Date: Tue, 27 Jan 2026 09:24:51 GMT
• Title: DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalization
• Authors: Xudong Han, Senkang Hu, Yihang Tao, Yu Guo, Philip Birch, Sam Tak Wu Kwong, Yuguang Fang,
• Abstract summary: Domain Generalization is a critical area that focuses on developing models capable of performing well on data from unseen distributions.<n>Existing approaches primarily concentrate on learning domain-invariant features, which assume that a model robust to variations in the source domains will generalize well to unseen target domains.<n>We propose Domain-Sensitive Regularization (DSP-Reg), a principled framework that guides model optimization by a soft regularization technique.
• Score: 21.0252973774713
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Domain Generalization (DG) is a critical area that focuses on developing models capable of performing well on data from unseen distributions, which is essential for real-world applications. Existing approaches primarily concentrate on learning domain-invariant features, which assume that a model robust to variations in the source domains will generalize well to unseen target domains. However, these approaches neglect a deeper analysis at the parameter level, which makes the model hard to explicitly differentiate between parameters sensitive to domain shifts and those robust, potentially hindering its overall ability to generalize. In order to address these limitations, we first build a covariance-based parameter sensitivity analysis framework to quantify the sensitivity of each parameter in a model to domain shifts. By computing the covariance of parameter gradients across multiple source domains, we can identify parameters that are more susceptible to domain variations, which serves as our theoretical foundation. Based on this, we propose Domain-Sensitive Parameter Regularization (DSP-Reg), a principled framework that guides model optimization by a soft regularization technique that encourages the model to rely more on domain-invariant parameters while suppressing those that are domain-specific. This approach provides a more granular control over the model's learning process, leading to improved robustness and generalization to unseen domains. Extensive experiments on benchmarks, such as PACS, VLCS, OfficeHome, and DomainNet, demonstrate that DSP-Reg outperforms state-of-the-art approaches, achieving an average accuracy of 66.7\% and surpassing all baselines.

Related papers

• Preserving Domain Generalization in Fine-Tuning via Joint Parameter Selection [26.366275954455514]
  We introduce Joint Selection (JPS), a novel method that restricts updates to a small subset of parameters, thereby retaining and harnessing the strength of pre-trained models.<n>JPS achieves superior performance compared to state-of-the-art domain generalization methods, substantiating both the efficiency and efficacy of the proposed approach.
  arXiv  Detail & Related papers  (2025-08-23T10:00:45Z)
• TAROT: Towards Essentially Domain-Invariant Robustness with Theoretical Justification [2.3718468096686802]
  TAROT is designed to enhance both domain adaptability and robustness.<n>It achieves superior performance on the challenging DomainNet dataset.<n>Results highlight the broader applicability of our approach in real-world domain adaptation scenarios.
  arXiv  Detail & Related papers  (2025-05-10T09:43:04Z)
• FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation [65.93276461982093]
  Existing approaches either selectively fine-tune parameters or freeze the VFMs and update only the adapters.<n>We propose textbfFisherTune, a robust fine-tuning method guided by the Domain-Related Fisher Information Matrix (DR-FIM)<n>DR-FIM measures parameter sensitivity across tasks and domains, enabling selective updates that preserve generalization and enhance DGSS adaptability.
  arXiv  Detail & Related papers  (2025-03-23T04:47:15Z)
• Normalization Perturbation: A Simple Domain Generalization Method for Real-World Domain Shifts [133.99270341855728]
  Real-world domain styles can vary substantially due to environment changes and sensor noises. Deep models only know the training domain style. We propose Normalization Perturbation to overcome this domain style overfitting problem.
  arXiv  Detail & Related papers  (2022-11-08T17:36:49Z)
• Learning to Learn Domain-invariant Parameters for Domain Generalization [29.821634033299855]
  Domain generalization (DG) aims to overcome this issue by capturing domain-invariant representations from source domains. We propose two modules of Domain Decoupling and Combination (DDC) and Domain-invariance-guided Backpropagation (DIGB) Our proposed method has achieved state-of-the-art performance with strong generalization capability.
  arXiv  Detail & Related papers  (2022-11-04T07:19:34Z)
• Variational Model Perturbation for Source-Free Domain Adaptation [64.98560348412518]
  We introduce perturbations into the model parameters by variational Bayesian inference in a probabilistic framework. We demonstrate the theoretical connection to learning Bayesian neural networks, which proves the generalizability of the perturbed model to target domains.
  arXiv  Detail & Related papers  (2022-10-19T08:41:19Z)
• Adaptive Domain Generalization via Online Disagreement Minimization [17.215683606365445]
  Domain Generalization aims to safely transfer a model to unseen target domains. AdaODM adaptively modifies the source model at test time for different target domains. Results show AdaODM stably improves the generalization capacity on unseen domains.
  arXiv  Detail & Related papers  (2022-08-03T11:51:11Z)
• Generalizing to Unseen Domains with Wasserstein Distributional Robustness under Limited Source Knowledge [22.285156929279207]
  Domain generalization aims at learning a universal model that performs well on unseen target domains. We propose a novel domain generalization framework called Wasserstein Distributionally Robust Domain Generalization (WDRDG)
  arXiv  Detail & Related papers  (2022-07-11T14:46:50Z)
• Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
  We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions. We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space. Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
  arXiv  Detail & Related papers  (2022-03-24T11:54:59Z)
• Model-Based Domain Generalization [96.84818110323518]
  We propose a novel approach for the domain generalization problem called Model-Based Domain Generalization. Our algorithms beat the current state-of-the-art methods on the very-recently-proposed WILDS benchmark by up to 20 percentage points.
  arXiv  Detail & Related papers  (2021-02-23T00:59:02Z)
• Dual Distribution Alignment Network for Generalizable Person Re-Identification [174.36157174951603]
  Domain generalization (DG) serves as a promising solution to handle person Re-Identification (Re-ID) We present a Dual Distribution Alignment Network (DDAN) which handles this challenge by selectively aligning distributions of multiple source domains. We evaluate our DDAN on a large-scale Domain Generalization Re-ID (DG Re-ID) benchmark.
  arXiv  Detail & Related papers  (2020-07-27T00:08:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.