Target to Source: Guidance-Based Diffusion Model for Test-Time Adaptation

• URL: http://arxiv.org/abs/2312.05274v1
• Date: Fri, 8 Dec 2023 02:31:36 GMT
• Title: Target to Source: Guidance-Based Diffusion Model for Test-Time Adaptation
• Authors: Kaiyu Song, Hanjiang Lai
• Abstract summary: We propose a novel guidance-based diffusion-driven adaptation (GDDA) to overcome the data shift. GDDA significantly performs better than the state-of-the-art baselines.
• Score: 8.695439655048634
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most recent works of test-time adaptation (TTA) aim to alleviate domain shift problems by re-training source classifiers in each domain. On the other hand, the emergence of the diffusion model provides another solution to TTA, which directly maps the test data from the target domain to the source domain based on a diffusion model pre-trained in the source domain. The source classifier does not need to be fine-tuned. However, 1) the semantic information loss from test data to the source domain and 2) the model shift between the source classifier and diffusion model would prevent the diffusion model from mapping the test data back to the source domain correctly. In this paper, we propose a novel guidance-based diffusion-driven adaptation (GDDA) to overcome the data shift and let the diffusion model find a better way to go back to the source. Concretely, we first propose detail and global guidance to better keep the common semantics of the test and source data. The two guidance include a contrastive loss and mean squared error to alleviate the information loss by fully exploring the diffusion model and the test data. Meanwhile, we propose a classifier-aware guidance to reduce the bias caused by the model shift, which can incorporate the source classifier's information into the generation process of the diffusion model. Extensive experiments on three image datasets with three classifier backbones demonstrate that GDDA significantly performs better than the state-of-the-art baselines. On CIFAR-10C, CIFAR-100C, and ImageNetC, GDDA achieves 11.54\%, 19.05\%, and 11.63\% average accuracy improvements, respectively. GDDA even achieves equal performance compared with methods of re-training classifiers. The code is available in the supplementary material.

Related papers

• Everything to the Synthetic: Diffusion-driven Test-time Adaptation via Synthetic-Domain Alignment [76.44483062571611]
  Test-time adaptation (TTA) aims to enhance the performance of source-domain pretrained models when tested on unknown shifted target domains. Traditional TTA methods primarily adapt model weights based on target data streams, making model performance sensitive to the amount and order of target data. Recent diffusion-driven TTA methods have demonstrated strong performance by using an unconditional diffusion model.
  arXiv  Detail & Related papers  (2024-06-06T17:39:09Z)
• Source-Free Domain Adaptation with Diffusion-Guided Source Data Generation [6.087274577167399]
  This paper introduces a novel approach to leverage the generalizability of Diffusion Models for Source-Free Domain Adaptation (DM-SFDA) Our proposed DMSFDA method involves fine-tuning a pre-trained text-to-image diffusion model to generate source domain images. We validate our approach through comprehensive experiments across a range of datasets, including Office-31, Office-Home, and VisDA.
  arXiv  Detail & Related papers  (2024-02-07T14:56:13Z)
• Turn Down the Noise: Leveraging Diffusion Models for Test-time Adaptation via Pseudo-label Ensembling [2.5437028043490084]
  The goal of test-time adaptation is to adapt a source-pretrained model to a continuously changing target domain without relying on any source data. We introduce an approach that leverages a pre-trained diffusion model to project the target domain images closer to the source domain.
  arXiv  Detail & Related papers  (2023-11-29T20:35:32Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Back to the Source: Diffusion-Driven Test-Time Adaptation [77.4229736436935]
  Test-time adaptation harnesses test inputs to improve accuracy of a model trained on source data when tested on shifted target data. We instead update the target data, by projecting all test inputs toward the source domain with a generative diffusion model.
  arXiv  Detail & Related papers  (2022-07-07T17:14:10Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Distill and Fine-tune: Effective Adaptation from a Black-box Source Model [138.12678159620248]
  Unsupervised domain adaptation (UDA) aims to transfer knowledge in previous related labeled datasets (source) to a new unlabeled dataset (target) We propose a novel two-step adaptation framework called Distill and Fine-tune (Dis-tune)
  arXiv  Detail & Related papers  (2021-04-04T05:29:05Z)
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation [102.67010690592011]
  Unsupervised adaptationUDA (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model. This work tackles a practical setting where only a trained source model is available and how we can effectively utilize such a model without source data to solve UDA problems.
  arXiv  Detail & Related papers  (2020-02-20T03:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.