Decompose and Reorganize: Planning with Primitives and Visuomotor Policies Learned from Demonstrations

• URL: http://arxiv.org/abs/2607.25397v1
• Date: Tue, 28 Jul 2026 07:57:09 GMT
• Title: Decompose and Reorganize: Planning with Primitives and Visuomotor Policies Learned from Demonstrations
• Abstract summary: We propose DR-LfD, a framework that seamlessly integrates visuomotor policies into a TAMP-gated decision-making system.<n>DR-LfD decomposes human demonstrations into atomic skills, which are reproduced as visuomotor policies or object-centric primitives.<n>We demonstrate the strong performance of DR-LfD on tasks involving multiple steps, unseen setups, and physical constraints.
• Score: 37.34419635595643
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Successfully automating dexterous, long-horizon robotic manipulation requires frameworks capable of both high-level reasoning and fine-grained execution. Traditional task and motion planning (TAMP), while excellent at symbolic planning, is often brittle in contact-rich operations. Simultaneously, imitation learning (IL), while effective in manipulation tasks with visual feedback, is limited by its low capability in spatial generalization and multi-stage operation. To reconcile their complementary strengths and limitations, we propose DR-LfD (Decomposed and Reorganized Skills Learned from Demonstrations), a framework that seamlessly integrates visuomotor policies into a TAMP-gated decision-making system. Based on contact relationships, DR-LfD decomposes human demonstrations into atomic skills, which are reproduced as visuomotor policies or object-centric primitives. The initiation, termination, and constraints of the visuomotor policies are carefully modeled and implemented in a TAMP-compatible form, enabling reorganization of skills learned from different sources. DR-LfD transforms the learning problem from one requiring exponential demonstration data over possible skill sequences to one whose demonstration burden scales with the number of distinct skill types, with limited data for each skill. Through comprehensive real-world and simulation benchmarking across diverse scenarios, we demonstrate the strong performance of DR-LfD on tasks involving multiple steps, unseen setups, and physical constraints. Project website: https://dr-lfd.github.io/DR-LfD-website.

Related papers

• LUCID: Latent-Skill Unified Control via Imagined Dynamics for Long-Horizon Humanoid Loco-Manipulation [16.259708213720383]
  Long-horizon humanoid loco-manipulation requires versatile whole-body skills and reliable high-level decision making.<n>Existing methods often coordinate pretrained skills with scripted planners, finite-state machines or task-specific model-free policies.<n>We propose textbfLUCID, a hierarchical model-based reinforcement learning framework that plans over reusable skills through imagined rollouts of a learned dynamics model.
  arXiv  Detail & Related papers  (2026-08-07T20:26:34Z)
• SkillMemo: Expert-guided Skill Memory Framework for Compositional Embodied Manipulation [72.14179740056073]
  Embodied visuomotor models, including Diffusion Policy (DP) and Vision-Language-Action (VLA) models, have demonstrated promising performance on robotic manipulation benchmarks.<n>We propose Skill-Based Memory (SkillMemo) framework that implicitly decomposes long-horizon demonstrations into latent atomic skills.
  arXiv  Detail & Related papers  (2026-08-06T12:46:13Z)
• DLO-Lab: Benchmarking Deformable Linear Object Manipulations with Differentiable Physics [30.589042316457093]
  We introduce a differentiable simulator explicitly designed for versatile DLO manipulation.<n>Our simulator models a wide range of material properties-including (in)extensibility, elasticity, bending plasticity, and complex interactions.<n>We propose a benchmark suite of representative tasks that highlight the unique challenges of DLO manipulation.
  arXiv  Detail & Related papers  (2026-06-02T20:49:44Z)
• Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories [14.475289544849007]
  ENAP (Emergent Neural Automaton Policy) is a framework that allows a bi-level neuro-symbolic policy adaptively emerge from visuomotor demonstrations.<n>By explicitly modeling the task structure with discrete transitions and continuous residuals, ENAP achieves high sample efficiency and interpretability without requiring task-specific labels.
  arXiv  Detail & Related papers  (2026-03-26T20:50:36Z)
• Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation [90.90219129619344]
  This paper presents a novel R-prior-S, Recurrent Geometric-priormodal Policy with Spiking features.<n>To ground high-level reasoning in physical reality, we leverage lightweight 2D geometric inductive biases.<n>For the data efficiency issue in robotic action generation, we introduce a Recursive Adaptive Spiking Network.
  arXiv  Detail & Related papers  (2026-01-13T23:36:30Z)
• Latent Diffusion Planning for Imitation Learning [78.56207566743154]
  Latent Diffusion Planning (LDP) is a modular approach consisting of a planner and inverse dynamics model.<n>By separating planning from action prediction, LDP can benefit from the denser supervision signals of suboptimal and action-free data.<n>On simulated visual robotic manipulation tasks, LDP outperforms state-of-the-art imitation learning approaches.
  arXiv  Detail & Related papers  (2025-04-23T17:53:34Z)
• MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation [62.854649499866774]
  Large Language Models (LLMs) have demonstrated remarkable planning abilities across various domains, including robotics manipulation and navigation.<n>We propose a novel multi-agent LLM framework that distributes high-level planning and low-level control code generation across specialized LLM agents.<n>We evaluate our approach on nine RLBench tasks, including long-horizon tasks, and demonstrate its ability to solve robotics manipulation in a zero-shot setting.
  arXiv  Detail & Related papers  (2024-11-26T17:53:44Z)
• Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulation [12.377289165111028]
  Reinforcement learning (RL) often necessitates a meticulous Markov Decision Process (MDP) design tailored to each task. This work proposes a systematic approach to behavior synthesis and control for multi-contact loco-manipulation tasks. We define a task-independent MDP to train RL policies using only a single demonstration per task generated from a model-based trajectory.
  arXiv  Detail & Related papers  (2024-10-17T17:46:27Z)
• NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptors [16.475094344344512]
  We propose to combine two paradigms: Neural Object Descriptors (NODs) that produce generalizable object-centric features and Task and Motion Planning (TAMP) frameworks that chain short-horizon skills to solve multi-step tasks. We introduce NOD-TAMP, a TAMP-based framework that extracts short manipulation trajectories from a handful of human demonstrations, adapts these trajectories using NOD features, and composes them to solve broad long-horizon, contact-rich tasks.
  arXiv  Detail & Related papers  (2023-11-02T18:26:28Z)
• Learning Neuro-Symbolic Skills for Bilevel Planning [63.388694268198655]
  Decision-making is challenging in robotics environments with continuous object-centric states, continuous actions, long horizons, and sparse feedback. Hierarchical approaches, such as task and motion planning (TAMP), address these challenges by decomposing decision-making into two or more levels of abstraction. Our main contribution is a method for learning parameterized polices in combination with operators and samplers.
  arXiv  Detail & Related papers  (2022-06-21T19:01:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.