Everything to the Synthetic: Diffusion-driven Test-time Adaptation via Synthetic-Domain Alignment

• URL: http://arxiv.org/abs/2406.04295v1
• Date: Thu, 6 Jun 2024 17:39:09 GMT
• Title: Everything to the Synthetic: Diffusion-driven Test-time Adaptation via Synthetic-Domain Alignment
• Authors: Jiayi Guo, Junhao Zhao, Chunjiang Ge, Chaoqun Du, Zanlin Ni, Shiji Song, Humphrey Shi, Gao Huang,
• Abstract summary: Test-time adaptation (TTA) aims to enhance the performance of source-domain pretrained models when tested on unknown shifted target domains. Traditional TTA methods primarily adapt model weights based on target data streams, making model performance sensitive to the amount and order of target data. Recent diffusion-driven TTA methods have demonstrated strong performance by using an unconditional diffusion model.
• Score: 76.44483062571611
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Test-time adaptation (TTA) aims to enhance the performance of source-domain pretrained models when tested on unknown shifted target domains. Traditional TTA methods primarily adapt model weights based on target data streams, making model performance sensitive to the amount and order of target data. Recently, diffusion-driven TTA methods have demonstrated strong performance by using an unconditional diffusion model, which is also trained on the source domain to transform target data into synthetic data as a source domain projection. This allows the source model to make predictions without weight adaptation. In this paper, we argue that the domains of the source model and the synthetic data in diffusion-driven TTA methods are not aligned. To adapt the source model to the synthetic domain of the unconditional diffusion model, we introduce a Synthetic-Domain Alignment (SDA) framework to fine-tune the source model with synthetic data. Specifically, we first employ a conditional diffusion model to generate labeled samples, creating a synthetic dataset. Subsequently, we use the aforementioned unconditional diffusion model to add noise to and denoise each sample before fine-tuning. This process mitigates the potential domain gap between the conditional and unconditional models. Extensive experiments across various models and benchmarks demonstrate that SDA achieves superior domain alignment and consistently outperforms existing diffusion-driven TTA methods. Our code is available at https://github.com/SHI-Labs/Diffusion-Driven-Test-Time-Adaptation-via-Synthetic-Domain-Alignment.

Related papers

• Constrained Diffusion Models via Dual Training [80.03953599062365]
  Diffusion processes are prone to generating samples that reflect biases in a training dataset. We develop constrained diffusion models by imposing diffusion constraints based on desired distributions. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• Transfer Learning for Diffusion Models [43.10840361752551]
  Diffusion models consistently produce high-quality synthetic samples. They can be impractical in real-world applications due to high collection costs or associated risks. This paper introduces the Transfer Guided Diffusion Process (TGDP), a novel approach distinct from conventional finetuning and regularization methods.
  arXiv  Detail & Related papers  (2024-05-27T06:48:58Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Target to Source: Guidance-Based Diffusion Model for Test-Time Adaptation [8.695439655048634]
  We propose a novel guidance-based diffusion-driven adaptation (GDDA) to overcome the data shift. GDDA significantly performs better than the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-12-08T02:31:36Z)
• Transcending Domains through Text-to-Image Diffusion: A Source-Free Approach to Domain Adaptation [6.649910168731417]
  Domain Adaptation (DA) is a method for enhancing a model's performance on a target domain with inadequate annotated data. We propose a novel framework for SFDA that generates source data using a text-to-image diffusion model trained on the target domain samples.
  arXiv  Detail & Related papers  (2023-10-02T23:38:17Z)
• Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolation [56.620403243640396]
  Deep neural networks achieve superior performance for learning from independent and identically distributed (i.i.d.) data. However, their performance deteriorates significantly when handling out-of-distribution (OoD) data. We develop a simple yet effective method called Generative Interpolation to fuse generative models trained from multiple domains for synthesizing diverse OoD samples.
  arXiv  Detail & Related papers  (2023-07-23T03:53:53Z)
• Back to the Source: Diffusion-Driven Test-Time Adaptation [77.4229736436935]
  Test-time adaptation harnesses test inputs to improve accuracy of a model trained on source data when tested on shifted target data. We instead update the target data, by projecting all test inputs toward the source domain with a generative diffusion model.
  arXiv  Detail & Related papers  (2022-07-07T17:14:10Z)
• Domain Adaptation without Source Data [20.64875162351594]
  We introduce Source data-Free Domain Adaptation (SFDA) to avoid accessing source data that may contain sensitive information. Our key idea is to leverage a pre-trained model from the source domain and progressively update the target model in a self-learning manner. Our PrDA outperforms conventional domain adaptation methods on benchmark datasets.
  arXiv  Detail & Related papers  (2020-07-03T07:21:30Z)
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation [102.67010690592011]
  Unsupervised adaptationUDA (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model. This work tackles a practical setting where only a trained source model is available and how we can effectively utilize such a model without source data to solve UDA problems.
  arXiv  Detail & Related papers  (2020-02-20T03:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.