Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning

• URL: http://arxiv.org/abs/2309.10302v2
• Date: Sun, 18 Feb 2024 02:58:17 GMT
• Title: Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning
• Authors: Ximei Wang, Junwei Pan, Xingzhuo Guo, Dapeng Liu, Jie Jiang
• Abstract summary: Multi-domain learning aims to train a model with minimal average risk across multiple overlapping but non-identical domains. We propose Decoupled Training (D-Train) as a frustratingly easy and hyper parameter-free multi-domain learning method. D-Train is a tri-phase general-to-specific training strategy that first pre-trains on all domains to warm up a root model, then post-trains on each domain by splitting into multi-heads, and finally fine-tunes the heads by fixing the backbone.
• Score: 20.17925272562433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-domain learning (MDL) aims to train a model with minimal average risk across multiple overlapping but non-identical domains. To tackle the challenges of dataset bias and domain domination, numerous MDL approaches have been proposed from the perspectives of seeking commonalities by aligning distributions to reduce domain gap or reserving differences by implementing domain-specific towers, gates, and even experts. MDL models are becoming more and more complex with sophisticated network architectures or loss functions, introducing extra parameters and enlarging computation costs. In this paper, we propose a frustratingly easy and hyperparameter-free multi-domain learning method named Decoupled Training (D-Train). D-Train is a tri-phase general-to-specific training strategy that first pre-trains on all domains to warm up a root model, then post-trains on each domain by splitting into multi-heads, and finally fine-tunes the heads by fixing the backbone, enabling decouple training to achieve domain independence. Despite its extraordinary simplicity and efficiency, D-Train performs remarkably well in extensive evaluations of various datasets from standard benchmarks to applications of satellite imagery and recommender systems.

Related papers

• Crocodile: Cross Experts Covariance for Disentangled Learning in Multi-Domain Recommendation [23.010588147317623]
  We propose a novel Cross-experts Covariance Loss for Disentangled Learning model (Crocodile) It employs multiple embedding tables to make the model domain-aware at the embeddings which consist most parameters in the model. Crocodile achieves 0.72% CTR lift and 0.73% GMV lift on a primary advertising scenario.
  arXiv  Detail & Related papers  (2024-05-21T11:54:16Z)
• OurDB: Ouroboric Domain Bridging for Multi-Target Domain Adaptive Semantic Segmentation [8.450397069717727]
  Multi-target domain adaptation (MTDA) for semantic segmentation poses a significant challenge, as it involves multiple target domains with varying distributions. Previous MTDA approaches typically employ multiple teacher architectures, where each teacher specializes in one target domain to simplify the task. We propose an ouroboric domain bridging (OurDB) framework, offering an efficient solution to the MTDA problem using a single teacher architecture.
  arXiv  Detail & Related papers  (2024-03-18T08:55:48Z)
• Adapting Self-Supervised Representations to Multi-Domain Setups [47.03992469282679]
  Current state-of-the-art self-supervised approaches, are effective when trained on individual domains but show limited generalization on unseen domains. We propose a general-purpose, lightweight Domain Disentanglement Module that can be plugged into any self-supervised encoder.
  arXiv  Detail & Related papers  (2023-09-07T20:05:39Z)
• NormAUG: Normalization-guided Augmentation for Domain Generalization [60.159546669021346]
  We propose a simple yet effective method called NormAUG (Normalization-guided Augmentation) for deep learning. Our method introduces diverse information at the feature level and improves the generalization of the main path. In the test stage, we leverage an ensemble strategy to combine the predictions from the auxiliary path of our model, further boosting performance.
  arXiv  Detail & Related papers  (2023-07-25T13:35:45Z)
• Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation [86.02485817444216]
  We introduce Multi-Prompt Alignment (MPA), a simple yet efficient framework for multi-source UDA. MPA denoises the learned prompts through an auto-encoding process and aligns them by maximizing the agreement of all the reconstructed prompts. Experiments show that MPA achieves state-of-the-art results on three popular datasets with an impressive average accuracy of 54.1% on DomainNet.
  arXiv  Detail & Related papers  (2022-09-30T03:40:10Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• Multi-Domain Adversarial Feature Generalization for Person Re-Identification [52.835955258959785]
  We propose a multi-dataset feature generalization network (MMFA-AAE) It is capable of learning a universal domain-invariant feature representation from multiple labeled datasets and generalizing it to unseen' camera systems. It also surpasses many state-of-the-art supervised methods and unsupervised domain adaptation methods by a large margin.
  arXiv  Detail & Related papers  (2020-11-25T08:03:15Z)
• Multi-path Neural Networks for On-device Multi-domain Visual Classification [55.281139434736254]
  This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths.
  arXiv  Detail & Related papers  (2020-10-10T05:13:49Z)
• Mutual Learning Network for Multi-Source Domain Adaptation [73.25974539191553]
  We propose a novel multi-source domain adaptation method, Mutual Learning Network for Multiple Source Domain Adaptation (ML-MSDA) Under the framework of mutual learning, the proposed method pairs the target domain with each single source domain to train a conditional adversarial domain adaptation network as a branch network. The proposed method outperforms the comparison methods and achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-29T04:31:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.