Domain Adaptation with Adaptive Imagination for Visual Reinforcement Learning under Limited Target Data

• URL: http://arxiv.org/abs/2606.30192v1
• Date: Mon, 29 Jun 2026 12:08:08 GMT
• Title: Domain Adaptation with Adaptive Imagination for Visual Reinforcement Learning under Limited Target Data
• Abstract summary: AIDA is a framework for visual reinforcement learning that addresses sim-to-real transfer under scarce target data.<n>Our key idea is adaptive imagination: generating reliable and semantic imagination rollouts to augment limited target data.<n>AIDA learns semantically meaningful state representations and outperforms baselines across five MuJoCo tasks and two Gymnasium-Robotics tasks.
• Score: 16.16206336187501
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sim-to-real transfer remains a major obstacle for reinforcement learning (RL), especially for vision-based control where image observations exacerbate the state-distribution shift between simulation and the real world. Domain adaptation (DA) is a promising remedy for this challenge. Prior sim-to-real DA works have demonstrated encouraging results, yet these approaches typically assume substantially more target data, which is not available in practice. Indeed, their performance degrades significantly when the target data budget is reduced. To address this challenge, we propose AIDA (Adaptive Imagination for Domain Adaptation), a domain adaptation framework for visual reinforcement learning that addresses sim-to-real transfer under scarce target data without requiring additional interaction with the target environment. Our key idea is adaptive imagination: generating reliable and semantic imagination rollouts to augment limited target data. Specifically, AIDA employs a distribution-shift-aware discriminator that truncates rollouts when imagined transitions drift into low-confidence regions, so that only reliable transitions contribute to the augmentation. On these reliable transitions, AIDA introduces a self-consistency loss that cycles through state -> image observation -> state, penalizing discrepancies between the original and reconstructed states. This provides additional adaptation signals beyond the scarce target data. Our experiments demonstrate that adaptive imagination effectively truncates unreliable rollouts. By enforcing a self-consistency loss on the resulting reliable transitions, AIDA learns semantically meaningful state representations and outperforms baselines across five MuJoCo tasks and two Gymnasium-Robotics tasks.

Related papers

• WOMBET: World Model-based Experience Transfer for Robust and Sample-efficient Reinforcement Learning [8.729531978655737]
  Reinforcement learning in robotics is often limited by the cost and risk of data collection.<n>We propose textitWorld Model-based Experience Transfer (WOMBET), a framework that jointly generates and utilizes prior data.
  arXiv  Detail & Related papers  (2026-04-10T04:57:54Z)
• Attention Residual Fusion Network with Contrast for Source-free Domain Adaptation [7.195870730342018]
  Source-free domain adaptation (SFDA) involves training a model on source domain and then applying it to a related target domain.<n>Recent studies have shown promising results, but many approaches to domain adaptation concentrate on domain shift.<n>We propose a novel framework of Attention Residual Fusion Network (ARFNet) based on contrast learning for SFDA to alleviate negative transfer and domain shift.
  arXiv  Detail & Related papers  (2025-10-25T03:27:26Z)
• Disentangled Source-Free Personalization for Facial Expression Recognition with Neutral Target Data [49.25159192831934]
  This paper introduces the Disentangled SFDA (DSFDA) method to address the challenge posed by adapting models with missing target expression data.<n>Our method learns to disentangle features related to expressions and identity while generating the missing non-neutral expression data for the target subject.
  arXiv  Detail & Related papers  (2025-03-26T17:53:53Z)
• Cooperative Students: Navigating Unsupervised Domain Adaptation in Nighttime Object Detection [1.6624384368855527]
  Unsupervised Domain Adaptation (UDA) has shown significant advancements in object detection under well-lit conditions.<n>UDA's performance degrades notably in low-visibility scenarios, especially at night.<n>To address this problem, we propose a textbfCooperative textbfStudents (textbfCoS) framework.
  arXiv  Detail & Related papers  (2024-04-02T14:26:18Z)
• Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer [60.31021888394358]
  Unsupervised Domain Adaptation (UDA) can effectively address domain gap issues in real-world image Super-Resolution (SR) We propose a SOurce-free Domain Adaptation framework for image SR (SODA-SR) to address this issue, i.e., adapt a source-trained model to a target domain with only unlabeled target data.
  arXiv  Detail & Related papers  (2023-03-31T03:14:44Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• Lifelong Unsupervised Domain Adaptive Person Re-identification with Coordinated Anti-forgetting and Adaptation [127.6168183074427]
  We propose a new task, Lifelong Unsupervised Domain Adaptive (LUDA) person ReID. This is challenging because it requires the model to continuously adapt to unlabeled data of the target environments. We design an effective scheme for this task, dubbed CLUDA-ReID, where the anti-forgetting is harmoniously coordinated with the adaptation.
  arXiv  Detail & Related papers  (2021-12-13T13:19:45Z)
• Attention-based Adversarial Appearance Learning of Augmented Pedestrians [49.25430012369125]
  We propose a method to synthesize realistic data for the pedestrian recognition task. Our approach utilizes an attention mechanism driven by an adversarial loss to learn domain discrepancies. Our experiments confirm that the proposed adaptation method is robust to such discrepancies and reveals both visual realism and semantic consistency.
  arXiv  Detail & Related papers  (2021-07-06T15:27:00Z)
• Transformer-Based Source-Free Domain Adaptation [134.67078085569017]
  We study the task of source-free domain adaptation (SFDA), where the source data are not available during target adaptation. We propose a generic and effective framework based on Transformer, named TransDA, for learning a generalized model for SFDA.
  arXiv  Detail & Related papers  (2021-05-28T23:06:26Z)
• Intervention Design for Effective Sim2Real Transfer [48.9711031777803]
  This work addresses the recent success of domain randomization and data augmentation for the sim2real setting. We explain this success through the lens of causal inference, positioning domain randomization and data augmentation as interventions on the environment.
  arXiv  Detail & Related papers  (2020-12-03T16:38:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.