Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning

• URL: http://arxiv.org/abs/2605.24810v1
• Date: Sun, 24 May 2026 01:44:57 GMT
• Title: Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning
• Abstract summary: Off-dynamics offline reinforcement learning seeks to learn a target-domain policy from a large source dataset and a limited target dataset.<n>We propose CEDGE, a Cross-domain Energy-guided Diffusion GEneration framework.<n>CEDGE trains a trajectory diffusion model on source-domain trajectories and adapts the generated samples to the target domain through energy guidance.
• Score: 25.497449531415125
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Off-dynamics offline reinforcement learning seeks to learn a target-domain policy from a large source dataset and a limited target dataset under mismatched transition dynamics. Existing approaches such as reward augmentation and data filtering are constrained to the source dataset and cannot synthesize new target behavior to improve coverage beyond the collected source trajectories. While recent model-based methods attempt to address this by learning target-aware dynamics, the generated experience is constructed only at the transition level, which leads to accumulated errors over long horizons. These limitations necessitate a shift toward trajectory-level generation for off-dynamics offline RL. We propose CEDGE, a Cross-domain Energy-guided Diffusion GEneration framework. CEDGE trains a trajectory diffusion model on source-domain trajectories and adapts the generated samples to the target domain through energy guidance. This guidance is derived by minimizing the distribution mismatch between the source and desired target-domain trajectories and is decomposed into return, domain, and behavior energy components. The resulting energy-guided trajectories are useful both for direct planning and as synthetic data for policy learning. Since target adaptation is achieved via energy guidance rather than retraining the diffusion model, CEDGE can be efficiently adapted to new target dynamics compared to previous methods. Experiments on the ODRL benchmark demonstrate that trajectory-level energy-guided generation improves diffusion planning under dynamics shifts and produces synthetic data that improves downstream target policy learning.

Related papers

• Bridging Domain Gaps with Target-Aligned Generation for Offline Reinforcement Learning [4.404003578416113]
  Cross-domain offline reinforcement learning aims to adapt a policy from a source domain to a target domain using only pre-collected datasets.<n>We propose Target-aligned Coverage Expansion (TCE), a framework that decides how source data should be used.<n>TCE builds on a dual score-based generative model to synthesize target-consistent transitions over an expanded state region.
  arXiv  Detail & Related papers  (2026-05-13T06:23:51Z)
• HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving [51.268878540511054]
  We propose HAD, an end-to-end planning framework with a Hierarchical Diffusion Policy.<n>We show that HAD achieves new state-of-the-art performance on both NAVSIM and HUGSIM.
  arXiv  Detail & Related papers  (2026-04-04T04:12:47Z)
• Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making [48.998030470623384]
  offline decision-making requires reliable behaviors from fixed datasets without further interaction.<n>We propose a compositional model-based diffusion framework consisting of: (i) a planner that generates diverse, task-aligned trajectories; (ii) a dynamics model that enforces consistency with the underlying system dynamics; and (iii) a ranker module that selects behaviors aligned with the task objectives.
  arXiv  Detail & Related papers  (2025-12-09T06:26:02Z)
• ASTRO: Adaptive Stitching via Dynamics-Guided Trajectory Rollouts [22.46606397400043]
  We propose ASTRO, a data augmentation framework that generates distributionally novel and dynamics-consistent trajectories for offline RL.<n>ASTRO first learns a temporal-distance representation to identify distinct and reachable stitch targets.<n>We then employ a dynamics-guided stitch planner that adaptively generates connecting action sequences via Rollout Deviation Feedback.
  arXiv  Detail & Related papers  (2025-11-28T18:35:37Z)
• TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning [63.73629127832652]
  We introduce TD-JEPA, which leverages TD-based latent-predictive representations into unsupervised RL.<n> TD-JEPA trains explicit state and task encoders, a policy-conditioned multi-step predictor, and a set of parameterized policies directly in latent space.<n> Empirically, TD-JEPA matches or outperforms state-of-the-art baselines on locomotion, navigation, and manipulation tasks across 13 datasets.
  arXiv  Detail & Related papers  (2025-10-01T10:21:18Z)
• MOBODY: Model Based Off-Dynamics Offline Reinforcement Learning [25.497449531415125]
  We study off-dynamics offline reinforcement learning, where the goal is to learn a policy from offline source and limited target datasets.<n>We propose MOBODY, a Model-Based Off-Dynamics Offline RL algorithm that optimize a policy using learned target dynamics transitions.<n>We evaluate MOBODY on a wide range of MuJoCo and Adroit benchmarks, demonstrating that it outperforms state-of-the-art off-dynamics RL baselines.
  arXiv  Detail & Related papers  (2025-06-10T05:36:54Z)
• Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data [43.61074482938135]
  CompFlow is a method grounded in the theoretical connection between flow matching and optimal transport.<n>We model the target dynamics as a conditional flow built upon the output distribution of the source-domain flow.<n>We show that CompFlow outperforms strong baselines across several RL benchmarks with shifted dynamics.
  arXiv  Detail & Related papers  (2025-05-29T04:09:19Z)
• Prior-Guided Diffusion Planning for Offline Reinforcement Learning [5.819784482811376]
  Prior Guidance (PG) is a novel guided sampling framework that replaces the standard Gaussian prior of a behavior-cloned diffusion model.<n>We present an efficient training strategy that applies behavior regularization in latent space, and empirically demonstrate that PG outperforms state-the-art diffusion policies and planners across diverse long-horizon offline RL benchmarks.
  arXiv  Detail & Related papers  (2025-05-16T05:39:02Z)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models [79.2162092822111]
  We systematically evaluate reinforcement learning (RL) and control-based methods on a suite of navigation tasks.<n>We employ a latent dynamics model using the Joint Embedding Predictive Architecture (JEPA) and employ it for planning.<n>Our results show that model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts.
  arXiv  Detail & Related papers  (2025-02-20T18:39:41Z)
• Progressive Conservative Adaptation for Evolving Target Domains [76.9274842289221]
  Conventional domain adaptation typically transfers knowledge from a source domain to a stationary target domain. Restoring and adapting to such target data results in escalating computational and resource consumption over time. We propose a simple yet effective approach, termed progressive conservative adaptation (PCAda)
  arXiv  Detail & Related papers  (2024-02-07T04:11:25Z)
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective [52.497514255040514]
  This work develops a new approach that enables data-driven methods to continuously learn and optimize resource allocation strategies in a dynamic environment. We propose to build the notion of continual learning into wireless system design, so that the learning model can incrementally adapt to the new episodes. Our design is based on a novel bilevel optimization formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2021-05-03T07:23:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.