Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning

• URL: http://arxiv.org/abs/2605.25740v1
• Date: Mon, 25 May 2026 11:54:04 GMT
• Title: Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning
• Abstract summary: We propose Latent-Aligned Value Learning (LAVL), an offline goal-conditioned reinforcement learning algorithm.<n>Experiments on OGBench demonstrate that LAVL consistently outperforms existing offline GCRL methods.<n>LAVL exhibits strong performance in long-horizon tasks and trajectory stitching datasets.
• Score: 33.21859985861439
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Offline goal-conditioned reinforcement learning (GCRL) provides a practical framework for obtaining goal-reaching policies from fixed datasets. However, learning a reliable goal-conditioned value function in long-horizon tasks remains challenging. In this paper, we identify erroneous generalization in goal-conditioned value functions as a fundamental bottleneck, and demonstrate that appropriate inductive bias in the value function is crucial for addressing the bottleneck. Building on these findings, we propose Latent-Aligned Value Learning (LAVL), an offline GCRL algorithm that integrates latent-representation-based value generalization with hierarchical planning in a unified framework. Extensive experiments on OGBench demonstrate that LAVL consistently outperforms existing offline GCRL methods, achieving the highest performance on 20 out of 22 datasets. Notably, LAVL exhibits strong performance in long-horizon tasks and trajectory stitching datasets, where prior methods suffer significant performance degradation. Our code is available at https://github.com/oh-lab/LAVL.git.

Related papers

• Nudging Beyond the Comfort Zone: Efficient Strategy-Guided Exploration for RLVR [53.27792011950384]
  Reinforcement learning with verifiable rewards (RLVR) has emerged as a scalable paradigm for improving the reasoning capabilities of large language models.<n>We propose NudgeRL, a framework for structured and diversity-driven exploration in RLVR.<n>Our approach introduces Strategy Nudging, which conditions each rollout on lightweight, strategy-level contexts.
  arXiv  Detail & Related papers  (2026-05-15T08:22:59Z)
• ALOE: Action-Level Off-Policy Evaluation for Vision-Language-Action Model Post-Training [15.70383059978939]
  We study how to improve large foundation vision--action (VLA) systems through online reinforcement learning (RL) in real-world settings.<n>In practice, the value function is estimated from trajectory fragments collected from different data sources.<n>We propose ALOE, an action-level off-policy evaluation framework for VLA post-training.
  arXiv  Detail & Related papers  (2026-02-13T07:46:37Z)
• COVLM-RL: Critical Object-Oriented Reasoning for Autonomous Driving Using VLM-Guided Reinforcement Learning [55.83415345423854]
  We propose COVLM-RL, a novel end-to-end driving framework that integrates Critical Object-oriented (CO) reasoning with reinforcement learning (RL)<n>Experiments conducted in the CARLA simulator demonstrate that COVLM-RL significantly improves the success rate by 30% in trained driving environments.
  arXiv  Detail & Related papers  (2025-12-10T06:18:16Z)
• Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective [85.06838178922791]
  Reinforcement Learning (RL) has proven highly effective for autoregressive language models.<n>But adapting these methods to diffusion large language models (dLLMs) presents fundamental challenges.<n>We propose a principled RL framework that treats entire sequence generation as a single action and uses the ELBO as a tractable sequence-level likelihood proxy.
  arXiv  Detail & Related papers  (2025-12-03T13:05:32Z)
• Test-time Offline Reinforcement Learning on Goal-related Experience [50.94457794664909]
  Research in foundation models has shown that performance can be substantially improved through test-time training.<n>We propose a novel self-supervised data selection criterion, which selects transitions from an offline dataset according to their relevance to the current state.<n>Our goal-conditioned test-time training (GC-TTT) algorithm applies this routine in a receding-horizon fashion during evaluation, adapting the policy to the current trajectory as it is being rolled out.
  arXiv  Detail & Related papers  (2025-07-24T21:11:39Z)
• Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learning [5.274804664403783]
  Strict Subgoal Execution (SSE) is a graph-based hierarchical RL framework that enforces single-step subgoal reachability.<n>We show that SSE consistently outperforms existing goal-conditioned RL and hierarchical RL approaches in both efficiency and success rate.
  arXiv  Detail & Related papers  (2025-06-26T06:35:42Z)
• Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning [19.341894845618445]
  offline goal-conditioned reinforcement learning (GCRL) offers a practical learning paradigm in which goal-reaching policies are trained from abundant state-action trajectory datasets.<n>In this paper, we propose a simple yet effective solution: Option-aware Temporally Abstracted value learning, dubbed OTA, which incorporates temporal abstraction into the temporal-difference learning process.<n>We experimentally show that the high-level policy learned using OTA achieves strong performance on complex tasks from OGBench, a recently proposed offline GCRL benchmark.
  arXiv  Detail & Related papers  (2025-05-19T05:51:11Z)
• Stitching Sub-Trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL [18.31263353823447]
  We propose a model-based offline Goal-Conditioned Reinforcement Learning (Offline GCRL) method to acquire diverse goal-oriented skills. In this paper, we use the diffusion model that generates future plans conditioned on the target goal and value, with the target value estimated from the goal-relabeled offline dataset. We report state-of-the-art performance in the standard benchmark set of GCRL tasks, and demonstrate the capability to successfully stitch the segments of suboptimal trajectories in the offline data to generate high-quality plans.
  arXiv  Detail & Related papers  (2024-02-11T15:23:13Z)
• HIQL: Offline Goal-Conditioned RL with Latent States as Actions [81.67963770528753]
  We propose a hierarchical algorithm for goal-conditioned RL from offline data. We show how this hierarchical decomposition makes our method robust to noise in the estimated value function. Our method can solve long-horizon tasks that stymie prior methods, can scale to high-dimensional image observations, and can readily make use of action-free data.
  arXiv  Detail & Related papers  (2023-07-22T00:17:36Z)
• ConStruct-VL: Data-Free Continual Structured VL Concepts Learning [57.86651057895222]
  We introduce the first Continual Data-Free Structured VL Concepts Learning (ConStruct-VL) benchmark. We propose a data-free method comprised of a new approach of Adrial Pseudo-Replay (APR) which generates adversarial reminders of past tasks from past task models. We show this approach outperforms all data-free methods by as much as 7% while even matching some levels of experience-replay.
  arXiv  Detail & Related papers  (2022-11-17T18:57:03Z)
• Offline Reinforcement Learning with Value-based Episodic Memory [19.12430651038357]
  offline reinforcement learning (RL) shows promise of applying RL to real-world problems. We propose Expectile V-Learning (EVL), which smoothly interpolates between the optimal value learning and behavior cloning. We present a new offline method called Value-based Episodic Memory (VEM)
  arXiv  Detail & Related papers  (2021-10-19T08:20:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.