Related papers: Source-free Domain Adaptation via Distributional Alignment by Matching Batch Normalization Statistics

Source-free Domain Adaptation via Distributional Alignment by Matching Batch Normalization Statistics

URL: http://arxiv.org/abs/2101.10842v1
Date: Tue, 19 Jan 2021 14:22:33 GMT
Title: Source-free Domain Adaptation via Distributional Alignment by Matching Batch Normalization Statistics
Authors: Masato Ishii and Masashi Sugiyama
Abstract summary: We propose a novel domain adaptation method for the source-free setting. We use batch normalization statistics stored in the pretrained model to approximate the distribution of unobserved source data. Our method achieves competitive performance with state-of-the-art domain adaptation methods.
Score: 85.75352990739154
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: In this paper, we propose a novel domain adaptation method for the source-free setting. In this setting, we cannot access source data during adaptation, while unlabeled target data and a model pretrained with source data are given. Due to lack of source data, we cannot directly match the data distributions between domains unlike typical domain adaptation algorithms. To cope with this problem, we propose utilizing batch normalization statistics stored in the pretrained model to approximate the distribution of unobserved source data. Specifically, we fix the classifier part of the model during adaptation and only fine-tune the remaining feature encoder part so that batch normalization statistics of the features extracted by the encoder match those stored in the fixed classifier. Additionally, we also maximize the mutual information between the features and the classifier's outputs to further boost the classification performance. Experimental results with several benchmark datasets show that our method achieves competitive performance with state-of-the-art domain adaptation methods even though it does not require access to source data.