Adapting Self-Supervised Representations to Multi-Domain Setups
- URL: http://arxiv.org/abs/2309.03999v2
- Date: Wed, 13 Dec 2023 00:06:18 GMT
- Title: Adapting Self-Supervised Representations to Multi-Domain Setups
- Authors: Neha Kalibhat, Sam Sharpe, Jeremy Goodsitt, Bayan Bruss, Soheil Feizi
- Abstract summary: Current state-of-the-art self-supervised approaches, are effective when trained on individual domains but show limited generalization on unseen domains.
We propose a general-purpose, lightweight Domain Disentanglement Module that can be plugged into any self-supervised encoder.
- Score: 47.03992469282679
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Current state-of-the-art self-supervised approaches, are effective when
trained on individual domains but show limited generalization on unseen
domains. We observe that these models poorly generalize even when trained on a
mixture of domains, making them unsuitable to be deployed under diverse
real-world setups. We therefore propose a general-purpose, lightweight Domain
Disentanglement Module (DDM) that can be plugged into any self-supervised
encoder to effectively perform representation learning on multiple, diverse
domains with or without shared classes. During pre-training according to a
self-supervised loss, DDM enforces a disentanglement in the representation
space by splitting it into a domain-variant and a domain-invariant portion.
When domain labels are not available, DDM uses a robust clustering approach to
discover pseudo-domains. We show that pre-training with DDM can show up to 3.5%
improvement in linear probing accuracy on state-of-the-art self-supervised
models including SimCLR, MoCo, BYOL, DINO, SimSiam and Barlow Twins on
multi-domain benchmarks including PACS, DomainNet and WILDS. Models trained
with DDM show significantly improved generalization (7.4%) to unseen domains
compared to baselines. Therefore, DDM can efficiently adapt self-supervised
encoders to provide high-quality, generalizable representations for diverse
multi-domain data.
Related papers
- Virtual Classification: Modulating Domain-Specific Knowledge for
Multidomain Crowd Counting [67.38137379297717]
Multidomain crowd counting aims to learn a general model for multiple diverse datasets.
Deep networks prefer modeling distributions of the dominant domains instead of all domains, which is known as domain bias.
We propose a Modulating Domain-specific Knowledge Network (MDKNet) to handle the domain bias issue in multidomain crowd counting.
arXiv Detail & Related papers (2024-02-06T06:49:04Z) - Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning [20.17925272562433]
Multi-domain learning aims to train a model with minimal average risk across multiple overlapping but non-identical domains.
We propose Decoupled Training (D-Train) as a frustratingly easy and hyper parameter-free multi-domain learning method.
D-Train is a tri-phase general-to-specific training strategy that first pre-trains on all domains to warm up a root model, then post-trains on each domain by splitting into multi-heads, and finally fine-tunes the heads by fixing the backbone.
arXiv Detail & Related papers (2023-09-19T04:06:41Z) - MEnsA: Mix-up Ensemble Average for Unsupervised Multi Target Domain
Adaptation on 3D Point Clouds [9.568577396815602]
Unlabelled domain adaptation (UDA) addresses the problem of distribution shift between the unsupervised target domain and labelled source domain.
We propose to mix the feature representations from all domains together to achieve better domain adaptation performance by an ensemble average.
With the mixed representation, we use a domain classifier to improve at distinguishing the feature representations of source domain from those of target domains in a shared latent space.
arXiv Detail & Related papers (2023-04-04T06:13:33Z) - Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation [86.02485817444216]
We introduce Multi-Prompt Alignment (MPA), a simple yet efficient framework for multi-source UDA.
MPA denoises the learned prompts through an auto-encoding process and aligns them by maximizing the agreement of all the reconstructed prompts.
Experiments show that MPA achieves state-of-the-art results on three popular datasets with an impressive average accuracy of 54.1% on DomainNet.
arXiv Detail & Related papers (2022-09-30T03:40:10Z) - Dynamic Instance Domain Adaptation [109.53575039217094]
Most studies on unsupervised domain adaptation assume that each domain's training samples come with domain labels.
We develop a dynamic neural network with adaptive convolutional kernels to generate instance-adaptive residuals to adapt domain-agnostic deep features to each individual instance.
Our model, dubbed DIDA-Net, achieves state-of-the-art performance on several commonly used single-source and multi-source UDA datasets.
arXiv Detail & Related papers (2022-03-09T20:05:54Z) - META: Mimicking Embedding via oThers' Aggregation for Generalizable
Person Re-identification [68.39849081353704]
Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time.
This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
arXiv Detail & Related papers (2021-12-16T08:06:50Z) - Dual Distribution Alignment Network for Generalizable Person
Re-Identification [174.36157174951603]
Domain generalization (DG) serves as a promising solution to handle person Re-Identification (Re-ID)
We present a Dual Distribution Alignment Network (DDAN) which handles this challenge by selectively aligning distributions of multiple source domains.
We evaluate our DDAN on a large-scale Domain Generalization Re-ID (DG Re-ID) benchmark.
arXiv Detail & Related papers (2020-07-27T00:08:07Z) - Unified Multi-Domain Learning and Data Imputation using Adversarial
Autoencoder [5.933303832684138]
We present a novel framework that can combine multi-domain learning (MDL), data imputation (DI) and multi-task learning (MTL)
The core of our method is an adversarial autoencoder that can: (1) learn to produce domain-invariant embeddings to reduce the difference between domains; (2) learn the data distribution for each domain and correctly perform data imputation on missing data.
arXiv Detail & Related papers (2020-03-15T19:55:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.