Learning Diffusion Policy from Primitive Skills for Robot Manipulation

• URL: http://arxiv.org/abs/2601.01948v1
• Date: Mon, 05 Jan 2026 09:56:24 GMT
• Title: Learning Diffusion Policy from Primitive Skills for Robot Manipulation
• Abstract summary: Diffusion policies (DP) have recently shown great promise for generating actions in robotic manipulation.<n>We propose SDP, a skill-conditioned DP that integrates interpretable skill learning with conditional action planning.
• Score: 36.95867683028485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion policies (DP) have recently shown great promise for generating actions in robotic manipulation. However, existing approaches often rely on global instructions to produce short-term control signals, which can result in misalignment in action generation. We conjecture that the primitive skills, referred to as fine-grained, short-horizon manipulations, such as ``move up'' and ``open the gripper'', provide a more intuitive and effective interface for robot learning. To bridge this gap, we propose SDP, a skill-conditioned DP that integrates interpretable skill learning with conditional action planning. SDP abstracts eight reusable primitive skills across tasks and employs a vision-language model to extract discrete representations from visual observations and language instructions. Based on them, a lightweight router network is designed to assign a desired primitive skill for each state, which helps construct a single-skill policy to generate skill-aligned actions. By decomposing complex tasks into a sequence of primitive skills and selecting a single-skill policy, SDP ensures skill-consistent behavior across diverse tasks. Extensive experiments on two challenging simulation benchmarks and real-world robot deployments demonstrate that SDP consistently outperforms SOTA methods, providing a new paradigm for skill-based robot learning with diffusion policies.

Related papers

• Adapting Generalist Robot Policies with Semantic Reinforcement Learning [59.009122725071386]
  Generalist robot policies learn a diverse repertoire of behaviors from large-scale pretraining.<n>We propose Semantic Action Reinforcement Learning (SARL) to learn to optimize this prompt space through online interaction.<n>We show SARL unlocks fundamentally new capabilities -- adapting VLA behavior to solve complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-06-30T17:00:33Z)
• Build on Priors: Vision--Language--Guided Neuro-Symbolic Imitation Learning for Data-Efficient Real-World Robot Manipulation [4.118262876469644]
  We present a scalable neuro-symbolic framework that autonomously constructs symbolic planning domains and data-efficient control policies.<n>Our method segments demonstrations into skills and employs a Vision-Language Model (VLM) to classify skills.<n>Known controllers can be leveraged for real-world data augmentation by projecting a single demonstration onto other objects in the scene.
  arXiv  Detail & Related papers  (2026-04-04T15:17:59Z)
• Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies [13.826293542795938]
  Diffusion-based policies have recently shown strong results in robot manipulation, but their extension to multi-task scenarios is hindered by the high cost of scaling model size and demonstrations.<n>We introduce Skill Mixture-of-Experts Policy (SMP), a diffusion-based mixture-of-experts policy that learns a compact skill basis and uses sticky routing to compose actions from a small, task-relevant subset of experts at each step.<n>We validate SMP in simulation and on a real dual-arm platform with multi-task learning and transfer learning tasks, where SMP achieves higher success rates and markedly lower inference cost than large diffusion baselines.
  arXiv  Detail & Related papers  (2026-01-29T04:17:56Z)
• RoBridge: A Hierarchical Architecture Bridging Cognition and Execution for General Robotic Manipulation [90.81956345363355]
  RoBridge is a hierarchical intelligent architecture for general robotic manipulation.<n>It consists of a high-level cognitive planner (HCP) based on a large-scale pre-trained vision-language model (VLM)<n>It unleashes the procedural skill of reinforcement learning, effectively bridging the gap between cognition and execution.
  arXiv  Detail & Related papers  (2025-05-03T06:17:18Z)
• Few-Shot Vision-Language Action-Incremental Policy Learning [55.07841353049953]
  Transformer-based robotic manipulation methods utilize multi-view spatial representations and language instructions to learn robot motion trajectories.<n>Existing methods lack the capability for continuous learning on new tasks with only a few demonstrations.<n>We develop a Task-prOmpt graPh evolutIon poliCy (TOPIC) to address these issues.
  arXiv  Detail & Related papers  (2025-04-22T01:30:47Z)
• Trajectory Adaptation using Large Language Models [0.8704964543257245]
  Adapting robot trajectories based on human instructions as per new situations is essential for achieving more intuitive and scalable human-robot interactions.<n>This work proposes a flexible language-based framework to adapt generic robotic trajectories produced by off-the-shelf motion planners.<n>We utilize pre-trained LLMs to adapt trajectory waypoints by generating code as a policy for dense robot manipulation.
  arXiv  Detail & Related papers  (2025-04-17T08:48:23Z)
• Robotic Control via Embodied Chain-of-Thought Reasoning [86.6680905262442]
  Key limitation of learned robot control policies is their inability to generalize outside their training data.<n>Recent works on vision-language-action models (VLAs) have shown that the use of large, internet pre-trained vision-language models can substantially improve their robustness and generalization ability.<n>We introduce Embodied Chain-of-Thought Reasoning (ECoT) for VLAs, in which we train VLAs to perform multiple steps of reasoning about plans, sub-tasks, motions, and visually grounded features before predicting the robot action.
  arXiv  Detail & Related papers  (2024-07-11T17:31:01Z)
• Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning [61.294110816231886]
  We introduce a sparse, reusable, and flexible policy, Sparse Diffusion Policy (SDP) SDP selectively activates experts and skills, enabling efficient and task-specific learning without retraining the entire model. Demos and codes can be found in https://forrest-110.io/sparse_diffusion_policy/.
  arXiv  Detail & Related papers  (2024-07-01T17:59:56Z)
• RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents [105.13169239919272]
  We propose RH20T-P, a primitive-level robotic manipulation dataset.<n>It contains about 38k video clips covering 67 diverse manipulation tasks in real-world scenarios.<n>We standardize a plan-execute CGA paradigm and implement an exemplar baseline called RA-P on our RH20T-P.
  arXiv  Detail & Related papers  (2024-03-28T17:42:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.