PathBridger: Subgoal Bridges for Offline Goal-Conditioned Reinforcement Learning

• URL: http://arxiv.org/abs/2608.29061v1
• Date: Sat, 29 Aug 2026 05:33:01 GMT
• Title: PathBridger: Subgoal Bridges for Offline Goal-Conditioned Reinforcement Learning
• Abstract summary: We propose PathBridger, a hierarchical offline GCRL method that explicitly connects subgoal selection to short-horizon execution.<n> Experiments across the evaluated OGBench tasks demonstrate strong aggregate performance, with particularly large gains on the multi-object Cube manipulation tasks.
• Score: 15.88392414749782
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Offline goal-conditioned reinforcement learning (GCRL) aims to learn policies for reaching diverse goals entirely from fixed trajectory data. Long-horizon offline GCRL remains challenging because sparse goal-reaching signals must be propagated over many steps, while execution errors cannot be corrected through additional environment interaction. Existing methods address these challenges by improving long-range value estimation or reducing the effective decision horizon through subgoals, options, and action chunks. In several hierarchical methods, however, a selected subgoal specifies where to go, while the intervening state-space path remains implicit in an endpoint-conditioned low-level policy. To address this interface, we propose PathBridger, a hierarchical offline GCRL method that explicitly connects subgoal selection to short-horizon execution. PathBridger constructs a state-space bridge toward the selected intermediate endpoint and decodes it into a short executable action chunk using an inverse dynamics model. Experiments across the evaluated OGBench tasks demonstrate strong aggregate performance, with particularly large gains on the multi-object Cube manipulation tasks. Code: https://github.com/SChoish/PathBridger

Related papers

• Extreme Motion Generation via Hybrid Null-Space Control for Straight-Line Path Following [10.975583425374532]
  This paper studies extreme motion generation'', which aims to maximize the path length along a pre-defined trajectory within the manipulator's workspace.<n>We propose a step-level hybrid controller that switches between an RL-based and a model-based controller according to the normalized joint-limit distance.<n>We evaluate the proposed framework on 10,000 straight-line path-following tasks with a 7-DoF Franka FR3, extending the average rollout length by 27% over the model-based baseline.
  arXiv  Detail & Related papers  (2026-06-02T09:33:08Z)
• Adaptive Coarse-to-Fine Subgoal Refinement for Long-Horizon Offline Goal-Conditioned Reinforcement Learning [16.15046948057208]
  We propose a fully offline goal-conditioned reinforcement learning framework that adaptively refines distant goals before execution.<n>Experiments on OGBench show substantial gains on several long-horizon tasks, with ablations validating the proposed refinement and stopping mechanisms.
  arXiv  Detail & Related papers  (2026-05-27T08:17:01Z)
• Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RL [25.40364932514488]
  We propose a novel framework that reformulates hierarchical decision-making as autoregressive sequence modeling.<n>CoGHP consistently outperforms strong offline baselines, demonstrating improved performance on long-horizon tasks.
  arXiv  Detail & Related papers  (2026-02-03T11:11:03Z)
• Push Smarter, Not Harder: Hierarchical RL-Diffusion Policy for Efficient Nonprehensile Manipulation [8.7216199131049]
  HeRD is a hierarchical reinforcement learning-diffusion policy that decomposes pushing tasks into two levels: high-level goal selection and low-level trajectory generation.<n>We employ a high-level reinforcement learning agent to select intermediate spatial goals, and a low-level goal-conditioned diffusion model to generate feasible, efficient trajectories to reach them.<n>Our results suggest that hierarchical control with generative low-level planning is a promising direction for scalable, goal-directed nonprehensile manipulation.
  arXiv  Detail & Related papers  (2025-12-10T21:40:22Z)
• Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learning [72.24831946301613]
  Key question is how to estimate the temporal distance between pairs of observations.<n>We show how these approaches can be integrated into a practical GCRL method that fits a quasimetric distance.<n>We also demonstrate that our method can enable stitching in the real-world robotic manipulation domain.
  arXiv  Detail & Related papers  (2025-11-11T01:28:10Z)
• Test-Time Graph Search for Goal-Conditioned Reinforcement Learning [56.13800388912632]
  offline goal-conditioned reinforcement learning (GCRL) trains policies that reach user-specified goals at test time.<n>We introduce Test-Time Graph Search (TTGS), a lightweight planning approach to solve the GCRL task.<n>TTGS accepts any state-space distance or cost signal, builds a weighted graph over dataset states, and performs fast search to assemble a sequence of subgoals that a frozen policy executes.
  arXiv  Detail & Related papers  (2025-10-08T17:20:53Z)
• Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations [72.24831946301613]
  Approaches for goal-conditioned reinforcement learning (GCRL) often use learned state representations to extract goal-reaching policies.<n>We propose an approach that unifies these two frameworks, using the structure of a quasimetric representation space (triangle inequality) with the right additional constraints to learn successor representations that enable optimal goal-reaching.<n>Our approach is able to exploit a **quasimetric** distance parameterization to learn **optimal** goal-reaching distances, even with **suboptimal** data and in **stochastic** environments.
  arXiv  Detail & Related papers  (2025-09-24T18:45:32Z)
• Flattening Hierarchies with Policy Bootstrapping [5.528896840956629]
  We introduce an algorithm to train a flat (non-hierarchical) goal-conditioned policy by bootstrapping on subgoal-conditioned policies with advantage-weighted importance sampling.<n>Our approach eliminates the need for a generative model over the (sub)goal space, which we find is key for scaling to high-dimensional control in large state spaces.
  arXiv  Detail & Related papers  (2025-05-20T23:31:30Z)
• Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning [68.16998247593209]
  offline reinforcement learning (RL) paradigm provides recipe to convert static behavior datasets into policies that can perform better than the policy that collected the data. In this paper, we propose an adaptive scheme for action quantization. We show that several state-of-the-art offline RL methods such as IQL, CQL, and BRAC improve in performance on benchmarks when combined with our proposed discretization scheme.
  arXiv  Detail & Related papers  (2023-10-18T06:07:10Z)
• HIQL: Offline Goal-Conditioned RL with Latent States as Actions [81.67963770528753]
  We propose a hierarchical algorithm for goal-conditioned RL from offline data. We show how this hierarchical decomposition makes our method robust to noise in the estimated value function. Our method can solve long-horizon tasks that stymie prior methods, can scale to high-dimensional image observations, and can readily make use of action-free data.
  arXiv  Detail & Related papers  (2023-07-22T00:17:36Z)
• Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space [76.46113138484947]
  General-purpose robots require diverse repertoires of behaviors to complete challenging tasks in real-world unstructured environments. To address this issue, goal-conditioned reinforcement learning aims to acquire policies that can reach goals for a wide range of tasks on command. We propose Planning to Practice, a method that makes it practical to train goal-conditioned policies for long-horizon tasks.
  arXiv  Detail & Related papers  (2022-05-17T06:58:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.