DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning

• URL: http://arxiv.org/abs/2607.16090v1
• Date: Fri, 17 Jul 2026 16:20:08 GMT
• Title: DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning
• Abstract summary: Transferring policies across domains poses a vital challenge in reinforcement learning.<n>We introduce DADiff, a diffusion-based framework that leverages the discrepancy between source and target domain generative trajectories.<n>Our method provides superior performance compared to existing approaches.
• Score: 6.441873210441909
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transferring policies across domains poses a vital challenge in reinforcement learning, due to the dynamics mismatch between the source and target domains. In this paper, we consider the setting of online dynamics adaptation, where policies are trained in the source domain with sufficient data, while only limited interactions with the target domain are allowed. There are a few existing works that address the dynamics mismatch by employing domain classifiers, value-guided data filtering, or representation learning. Instead, we study the domain adaptation problem from a generative modeling perspective. Specifically, we introduce DADiff, a diffusion-based framework that leverages the discrepancy between source and target domain generative trajectories in the generation process of the next state to estimate the dynamics mismatch. Both reward modification and data selection variants are developed to adapt the policy to the target domain. We also provide a theoretical analysis to show that the performance difference of a given policy between the two domains is bounded by the generative trajectory deviation. More discussions on the applicability of the variants and the connection between our theoretical analysis and the prior work are further provided. We conduct extensive experiments in environments with various shifts to validate the effectiveness of our method. The results demonstrate that our method provides superior performance compared to existing approaches, effectively addressing the dynamics mismatch. We provide the code of our method at https://github.com/hanyang-chen/DADiff-release

Related papers

• Cross-Domain Offline Policy Adaptation via Selective Transition Correction [29.251685312287155]
  It remains a critical challenge to adapt policies across domains with mismatched dynamics in reinforcement learning (RL)<n>We study cross-domain offline RL, where an offline dataset from another similar source domain can be accessed to enhance policy learning upon a target domain dataset.<n>We propose the Selective Transition Correction (STC) algorithm, which enables reliable usage of source domain data for policy adaptation.
  arXiv  Detail & Related papers  (2026-02-05T15:37:29Z)
• xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing [21.37585797507323]
  Cross-domain policy transfer methods mostly aim at learning domain correspondences or corrections to facilitate policy learning.<n>We propose the Cross-Domain Trajectory EDiting framework that employs a specially designed diffusion model for cross-domain trajectory adaptation.<n>Our proposed model architecture effectively captures the intricate dependencies among states, actions, and rewards, as well as the dynamics patterns within target data.
  arXiv  Detail & Related papers  (2024-09-13T10:07:28Z)
• Cross-Domain Policy Adaptation by Capturing Representation Mismatch [53.087413751430255]
  It is vital to learn effective policies that can be transferred to different domains with dynamics discrepancies in reinforcement learning (RL) In this paper, we consider dynamics adaptation settings where there exists dynamics mismatch between the source domain and the target domain. We perform representation learning only in the target domain and measure the representation deviations on the transitions from the source domain.
  arXiv  Detail & Related papers  (2024-05-24T09:06:12Z)
• Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer [69.82229895838577]
  Active Domain Adaptation (ADA) aims to maximally boost model adaptation in a new target domain by actively selecting a limited number of target data to annotate. This setting neglects the more practical scenario where training data are collected from multiple sources. This motivates us to target a new and challenging setting of knowledge transfer that extends ADA from a single source domain to multiple source domains.
  arXiv  Detail & Related papers  (2023-11-21T13:12:21Z)
• Cross-Domain Policy Adaptation via Value-Guided Data Filtering [57.62692881606099]
  Generalizing policies across different domains with dynamics mismatch poses a significant challenge in reinforcement learning. We present the Value-Guided Data Filtering (VGDF) algorithm, which selectively shares transitions from the source domain based on the proximity of paired value targets.
  arXiv  Detail & Related papers  (2023-05-28T04:08:40Z)
• A Comprehensive Survey on Source-free Domain Adaptation [69.17622123344327]
  The research of Source-Free Domain Adaptation (SFDA) has drawn growing attention in recent years. We provide a comprehensive survey of recent advances in SFDA and organize them into a unified categorization scheme. We compare the results of more than 30 representative SFDA methods on three popular classification benchmarks.
  arXiv  Detail & Related papers  (2023-02-23T06:32:09Z)
• FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation [26.929772844572213]
  We introduce a fixed ratio-based mixup to augment multiple intermediate domains between the source and target domain. We train the source-dominant model and the target-dominant model that have complementary characteristics. Through our proposed methods, the models gradually transfer domain knowledge from the source to the target domain.
  arXiv  Detail & Related papers  (2020-11-18T11:58:19Z)
• Adaptively-Accumulated Knowledge Transfer for Partial Domain Adaptation [66.74638960925854]
  Partial domain adaptation (PDA) deals with a realistic and challenging problem when the source domain label space substitutes the target domain. We propose an Adaptively-Accumulated Knowledge Transfer framework (A$2$KT) to align the relevant categories across two domains.
  arXiv  Detail & Related papers  (2020-08-27T00:53:43Z)
• Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits [101.68525259222164]
  We present a study of various distance-based measures in the context of NLP tasks, that characterize the dissimilarity between domains based on sample estimates. We develop a DistanceNet model which uses these distance measures as an additional loss function to be minimized jointly with the task's loss function. We extend this model to a novel DistanceNet-Bandit model, which employs a multi-armed bandit controller to dynamically switch between multiple source domains.
  arXiv  Detail & Related papers  (2020-01-13T15:53:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.