Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning

• URL: http://arxiv.org/abs/2607.09336v1
• Date: Fri, 10 Jul 2026 12:17:43 GMT
• Title: Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning
• Abstract summary: Shortcut Trajectory Planning (STP) is an offline model-based reinforcement learning framework that incorporates shortcut models as efficient trajectory generators.<n>STP achieves strong performance while simplifying the training pipeline for fast generative planning.
• Score: 8.985173544901796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion-based trajectory planners have shown strong performance in offline reinforcement learning, but their iterative denoising process often incurs high inference cost. Consistency-based planners reduce the number of sampling steps, yet they typically rely on a two-stage teacher--student distillation pipeline that increases training cost and may introduce instability. We propose Shortcut Trajectory Planning (STP), an offline model-based reinforcement learning framework that incorporates shortcut models as efficient trajectory generators. STP trains a conditional shortcut trajectory model in a single stage, supports adjustable one-step and few-step inference through step-size conditioning, and selects candidate plans using a critic augmented with feasibility-aware correction. Across standard D4RL benchmarks, including locomotion, navigation, manipulation, and dexterous control tasks, STP achieves strong performance while simplifying the training pipeline for fast generative planning.

Related papers

• Fast and Highly Expressive Policy Learning for Offline Reinforcement Learning via Bootstrapped Flow Q-Learning [27.992471782191014]
  Bootstrapped Flow Q-Learning (BFQ) is a novel framework that enables accurate single-step action generation during both training and inference.<n>It eliminates multi-step denoising, resulting in a learning procedure that is substantially faster, simpler, and more robust.<n>Extensive D4RL evaluations show that BFQ improves performance while significantly reducing computational cost.
  arXiv  Detail & Related papers  (2026-06-09T09:12:12Z)
• Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models [65.89572755202245]
  Diffusion Language Models (DLMs) offer stronger global awareness and highly parallel generation.<n>Post-training DLMs with standard Negative Evidence Lower Bound (NELBO)-based supervised fine-tuning remains inefficient.<n>We propose a self-distilled trajectory-based post-training framework that aligns training with the easy-to-hard structure of inference.
  arXiv  Detail & Related papers  (2026-05-12T09:39:06Z)
• Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance [47.605498477489306]
  Some advanced finetuning methods with auxiliary training objectives can improve performance and reduce the number of convergence steps.<n>We propose a novel approach to address the challenge that pretrained VLA models often fail to effectively improve performance and reduce adaptation costs during standard supervised finetuning.
  arXiv  Detail & Related papers  (2026-03-26T17:14:57Z)
• PlannerRFT: Reinforcing Diffusion Planners through Closed-Loop and Sample-Efficient Fine-Tuning [17.21182051618103]
  Diffusion-based planners have emerged as a promising approach for human-like trajectory generation in autonomous driving.<n>Recent works incorporate reinforcement fine-tuning to enhance the robustness of diffusion planners.<n>We propose PlannerRFT, a sample-efficient reinforcement fine-tuning framework for diffusion-based planners.
  arXiv  Detail & Related papers  (2026-01-19T09:56:05Z)
• PILOT: Planning via Internalized Latent Optimization Trajectories for Large Language Models [51.43746425777865]
  Large Language Models (LLMs) often lack the capacity to formulate global strategies, leading to error propagation in long-horizon tasks.<n>We propose PILOT, a framework designed to internalize the strategic oversight of large models into intrinsic Latent Guidance.
  arXiv  Detail & Related papers  (2026-01-07T12:38:56Z)
• Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter [52.111923076688505]
  Training Large Language Models (LLMs) with strong reasoning capabilities marks a significant milestone, unlocking new frontiers in complex problem-solving.<n>We propose TLT, a system that accelerates reasoning RL training losslessly by integrating adaptive speculative decoding.
  arXiv  Detail & Related papers  (2025-11-20T18:59:25Z)
• Improved Training Technique for Shortcut Models [12.527716901034694]
  Shortcut models are a promising, non-adversarial paradigm for generative modeling.<n>Shortcut models support one-step, few-step, and multi-step sampling from a single trained network.<n>This paper tackles the five core issues that held shortcut models back.
  arXiv  Detail & Related papers  (2025-10-24T08:35:04Z)
• Consistency Trajectory Planning: High-Quality and Efficient Trajectory Optimization for Offline Model-Based Reinforcement Learning [16.126373530332245]
  This paper introduces Consistency Trajectory Planning (CTP), a novel offline model-based reinforcement learning method.<n>CTP supports fast, single-step trajectory generation without significant degradation in policy quality.
  arXiv  Detail & Related papers  (2025-07-13T08:31:11Z)
• Fast T2T: Optimization Consistency Speeds Up Diffusion-Based Training-to-Testing Solving for Combinatorial Optimization [83.65278205301576]
  We propose to learn direct mappings from different noise levels to the optimal solution for a given instance, facilitating high-quality generation with minimal shots.<n>This is achieved through an optimization consistency training protocol, which minimizes the difference among samples.<n>Experiments on two popular tasks, the Traveling Salesman Problem (TSP) and Maximal Independent Set (MIS), demonstrate the superiority of Fast T2T regarding both solution quality and efficiency.
  arXiv  Detail & Related papers  (2025-02-05T07:13:43Z)
• Efficient Stagewise Pretraining via Progressive Subnetworks [53.00045381931778]
  The prevailing view suggests that stagewise dropping strategies, such as layer dropping, are ineffective when compared to stacking-based approaches. This paper challenges this notion by demonstrating that, with proper design, dropping strategies can be competitive, if not better, than stacking methods. We propose an instantiation of this framework - Random Part Training (RAPTR) - that selects and trains only a random subnetwork at each step, progressively increasing the size in stages.
  arXiv  Detail & Related papers  (2024-02-08T18:49:09Z)
• Online Convolutional Re-parameterization [51.97831675242173]
  We present online convolutional re- parameterization (OREPA), a two-stage pipeline, aiming to reduce the huge training overhead by squeezing the complex training-time block into a single convolution. Compared with the state-of-the-art re-param models, OREPA is able to save the training-time memory cost by about 70% and accelerate the training speed by around 2x. We also conduct experiments on object detection and semantic segmentation and show consistent improvements on the downstream tasks.
  arXiv  Detail & Related papers  (2022-04-02T09:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.