HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization

• URL: http://arxiv.org/abs/2606.16480v1
• Date: Mon, 15 Jun 2026 09:49:14 GMT
• Title: HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization
• Abstract summary: We present HOLO-MPPI, a multi-scenario motion planning framework that combines high-level policy learning with low-level optimal control.<n>Offline, we learn a high-level policy that proposes scenario-robust plans in an abstract action space, with a learned world model for online rollout.<n>Online, the policy serves as a data-driven prior generator that parameterizes MPPI's sampling distribution conditioned on the current observation and goal.<n>MPPI then optimize low-level control sequences around this prior in real time to adapt to local disturbances.
• Score: 8.210283573775568
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Robots deployed in the real world must plan motions across diverse scenarios without per-scenario retuning. End-to-end reinforcement learning (RL) can generalize across scenarios but often becomes brittle under distribution shift, reward misspecification, and stochastic interactions. Model predictive path integral (MPPI) control enables strong real-time refinement without gradients, but its performance depends on a well-shaped sampling prior, while manually designing the priors does not scale to multi-scenario deployment. We present HOLO-MPPI (High-level Offline, Low-level Online MPPI), a multi-scenario motion planning framework that combines high-level policy learning with low-level stochastic optimal control. Offline, we learn a high-level policy that proposes scenario-robust plans in an abstract action space, with a learned world model for online rollout. Online, the policy serves as a data-driven prior generator that parameterizes MPPI's sampling distribution conditioned on the current observation and goal. MPPI then optimizes low-level control sequences around this prior in real time to adapt to local disturbances. We instantiate HOLO-MPPI in autonomous driving by designing an effective high-level action space and tailored model architectures. Our evaluation across diverse driving scenarios shows that HOLO-MPPI improves upon MPPI and end-to-end RL baselines while maintaining real-time control.

Related papers

• Unifying Object-Centric World Models and Diffusion Policy: A Hierarchical Framework for Multi-Stage Robotic Tasks [44.06996146648851]
  We introduce WorldDP, a world model framework designed for multi-stage robotic manipulation.<n>Our hierarchical approach utilizes a high-level world model as a transition function to optimize for feasible subgoals during runtime.<n>We incorporate object-centric representations that decouple environmental entities and enable us to plan sequentially with respect to each.
  arXiv  Detail & Related papers  (2026-06-07T18:39:45Z)
• HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving [51.268878540511054]
  We propose HAD, an end-to-end planning framework with a Hierarchical Diffusion Policy.<n>We show that HAD achieves new state-of-the-art performance on both NAVSIM and HUGSIM.
  arXiv  Detail & Related papers  (2026-04-04T04:12:47Z)
• Soft MPCritic: Amortized Model Predictive Value Iteration [2.217971863441725]
  We propose soft MPCritic, an RL-MPC framework that learns in (soft) value space.<n> soft MPCritic instantiates MPC through model predictive path integral control (MPPI)<n>We introduce an amortized warm-start strategy that recycles planned open-loop action sequences from online observations.
  arXiv  Detail & Related papers  (2026-04-01T23:35:00Z)
• Model Predictive Control with Differentiable World Models for Offline Reinforcement Learning [17.293286354647716]
  We introduce an inference time adaptation framework inspired by model predictive control (MPC)<n>We show that exploiting inference-time information to optimize the policy parameters yields consistent gains over strong offline RL baselines.
  arXiv  Detail & Related papers  (2026-03-23T18:05:29Z)
• DDP-WM: Disentangled Dynamics Prediction for Efficient World Models [79.53092337527382]
  We introduce DDP-WM, a novel world model centered on the principle of Disentangled Dynamics Prediction.<n>DDP-WM realizes this decomposition through an architecture that integrates efficient historical processing with dynamic localization.<n>Experiments demonstrate that DDP-WM achieves significant efficiency and performance across diverse tasks.
  arXiv  Detail & Related papers  (2026-02-02T08:04:25Z)
• Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Driving [54.46325690390831]
  We propose Model-based Policy Adaptation (MPA), a general framework that enhances the robustness and safety of pretrained E2E driving agents during deployment.<n>MPA first generates diverse counterfactual trajectories using a geometry-consistent simulation engine.<n>MPA trains a diffusion-based policy adapter to refine the base policy's predictions and a multi-step Q value model to evaluate long-term outcomes.
  arXiv  Detail & Related papers  (2025-11-26T17:01:41Z)
• Bootstrap Off-policy with World Model [59.129118672069644]
  We propose BOOM, a framework that tightly integrates planning and off-policy learning through a bootstrap loop.<n>BOOM achieves state-of-the-art results in both training stability and final performance.
  arXiv  Detail & Related papers  (2025-11-01T06:33:04Z)
• A KL-regularization framework for learning to plan with adaptive priors [1.0246259631050245]
  We introduce Policy Optimization-Model Predictive Control (PO-MPC)<n>PO-MPC integrates the planner's action distribution as a prior in policy optimization.<n>Our experiments show that these extended configurations yield significant performance improvements.
  arXiv  Detail & Related papers  (2025-10-05T16:45:38Z)
• Drive As You Like: Strategy-Level Motion Planning Based on A Multi-Head Diffusion Model [7.3078271605135114]
  We propose a diffusion-based multi-head trajectory planner(M-diffusion planner)<n>During the early training stage, all output heads share weights to learn to generate high-quality trajectories.<n>We incorporate a large language model (LLM) to guide strategy selection, enabling dynamic, instruction-aware planning.
  arXiv  Detail & Related papers  (2025-08-23T08:33:11Z)
• Diverse Policy Optimization for Structured Action Space [59.361076277997704]
  We propose Diverse Policy Optimization (DPO) to model the policies in structured action space as the energy-based models (EBM) A novel and powerful generative model, GFlowNet, is introduced as the efficient, diverse EBM-based policy sampler. Experiments on ATSC and Battle benchmarks demonstrate that DPO can efficiently discover surprisingly diverse policies.
  arXiv  Detail & Related papers  (2023-02-23T10:48:09Z)
• Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic [59.94347858883343]
  This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states.
  arXiv  Detail & Related papers  (2021-02-24T01:11:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.