Bridging Domain Gaps with Target-Aligned Generation for Offline Reinforcement Learning

• URL: http://arxiv.org/abs/2605.13054v1
• Date: Wed, 13 May 2026 06:23:51 GMT
• Title: Bridging Domain Gaps with Target-Aligned Generation for Offline Reinforcement Learning
• Abstract summary: Cross-domain offline reinforcement learning aims to adapt a policy from a source domain to a target domain using only pre-collected datasets.<n>We propose Target-aligned Coverage Expansion (TCE), a framework that decides how source data should be used.<n>TCE builds on a dual score-based generative model to synthesize target-consistent transitions over an expanded state region.
• Score: 4.404003578416113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cross-domain offline reinforcement learning aims to adapt a policy from a source domain to a target domain using only pre-collected datasets, where environment dynamics may differ. A key challenge is to leverage source data while reducing distributional mismatch, particularly when the target dataset is extremely limited. To address this, we propose Target-aligned Coverage Expansion (TCE), a framework that decides how source data should be used, either by directly incorporating target-near transitions or by expanding state coverage through target-aligned generation, guided by theoretical analysis. TCE builds on a dual score-based generative model to synthesize target-consistent transitions over an expanded state region. Extensive experiments across diverse cross-domain environments show that TCE consistently outperforms state-of-the-art cross-domain offline RL baselines.

Related papers

• Cross-Domain Offline Policy Adaptation via Selective Transition Correction [29.251685312287155]
  It remains a critical challenge to adapt policies across domains with mismatched dynamics in reinforcement learning (RL)<n>We study cross-domain offline RL, where an offline dataset from another similar source domain can be accessed to enhance policy learning upon a target domain dataset.<n>We propose the Selective Transition Correction (STC) algorithm, which enables reliable usage of source domain data for policy adaptation.
  arXiv  Detail & Related papers  (2026-02-05T15:37:29Z)
• A Turn Toward Better Alignment: Few-Shot Generative Adaptation with Equivariant Feature Rotation [67.2019317630466]
  Few-shot image generation aims to effectively adapt a source generative model to a target domain using very few training images.<n>We propose Equivariant Feature Rotation (EFR), a novel adaptation strategy that aligns source and target domains at two complementary levels.<n>Our method significantly enhances the generative performance within the targeted domain.
  arXiv  Detail & Related papers  (2025-12-24T13:48:22Z)
• Cross-Domain Offline Policy Adaptation with Dynamics- and Value-Aligned Data Filtering [71.07094489719034]
  Cross-Domain Offline Reinforcement Learning aims to train an agent deployed in the target environment.<n>Recent advances address this issue by selectively sharing source domain samples that exhibit dynamics alignment with the target domain.<n>These approaches focus solely on dynamics alignment and overlook textitvalue alignment, i.e., selecting high-quality, high-value samples from the source domain.<n>We present our textbfunderlineDynamics- and textbfunderlineValue-aligned textbfunderlineData
  arXiv  Detail & Related papers  (2025-12-02T05:45:40Z)
• DmC: Nearest Neighbor Guidance Diffusion Model for Offline Cross-domain Reinforcement Learning [11.290019540058625]
  Cross-domain offline reinforcement learning (RL) seeks to enhance sample efficiency by utilizing additional offline source datasets.<n>DmC is a novel framework for cross-domain offline RL with limited target samples.
  arXiv  Detail & Related papers  (2025-07-28T03:34:15Z)
• Tackling Dimensional Collapse toward Comprehensive Universal Domain Adaptation [11.875619863954238]
  Partial domain matching (PDM) struggles in extreme cases where many source classes are absent in the target domain.<n>In this work, we identify that the failure of PDM for extreme UniDA stems from dimensional collapse (DC) in target representations.<n>We propose to use the de-collapse techniques in self-supervised learning on the unlabeled target data to preserve the intrinsic structure of the learned representations.
  arXiv  Detail & Related papers  (2024-10-15T04:51:37Z)
• Self-training through Classifier Disagreement for Cross-Domain Opinion Target Extraction [62.41511766918932]
  Opinion target extraction (OTE) or aspect extraction (AE) is a fundamental task in opinion mining. Recent work focus on cross-domain OTE, which is typically encountered in real-world scenarios. We propose a new SSL approach that opts for selecting target samples whose model output from a domain-specific teacher and student network disagrees on the unlabelled target data.
  arXiv  Detail & Related papers  (2023-02-28T16:31:17Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)
• Discriminative Cross-Domain Feature Learning for Partial Domain Adaptation [70.45936509510528]
  Partial domain adaptation aims to adapt knowledge from a larger and more diverse source domain to a smaller target domain with less number of classes. Recent practice on domain adaptation manages to extract effective features by incorporating the pseudo labels for the target domain. It is essential to align target data with only a small set of source data.
  arXiv  Detail & Related papers  (2020-08-26T03:18:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.