Keypoint-based Diffusion for Robotic Motion Planning on the NICOL Robot

• URL: http://arxiv.org/abs/2509.04076v2
• Date: Tue, 16 Sep 2025 08:06:29 GMT
• Title: Keypoint-based Diffusion for Robotic Motion Planning on the NICOL Robot
• Authors: Lennart Clasmeier, Jan-Gerrit Habekost, Connor Gäde, Philipp Allgeuer, Stefan Wermter,
• Abstract summary: We propose a novel diffusion-based action model for robotic motion planning.<n>By leveraging the power of deep learning, we are able to achieve good results in a much smaller runtime.
• Score: 7.239128729983817
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a novel diffusion-based action model for robotic motion planning. Commonly, established numerical planning approaches are used to solve general motion planning problems, but have significant runtime requirements. By leveraging the power of deep learning, we are able to achieve good results in a much smaller runtime by learning from a dataset generated by these planners. While our initial model uses point cloud embeddings in the input to predict keypoint-based joint sequences in its output, we observed in our ablation study that it remained challenging to condition the network on the point cloud embeddings. We identified some biases in our dataset and refined it, which improved the model's performance. Our model, even without the use of the point cloud encodings, outperforms numerical models by an order of magnitude regarding the runtime, while reaching a success rate of up to 90% of collision free solutions on the test set.

Related papers

• Closing the Train-Test Gap in World Models for Gradient-Based Planning [64.36544881136405]
  We propose improved methods for training world models that enable efficient gradient-based planning.<n>At test time, our approach outperforms or matches the classical gradient-free cross-entropy method.
  arXiv  Detail & Related papers  (2025-12-10T18:59:45Z)
• Planning-Query-Guided Model Generation for Model-Based Deformable Object Manipulation [24.086752654743957]
  This paper introduces a method that automatically generates task-specific, spatially adaptive dynamics models.<n>On a tree-manipulation task, our method doubles planning speed with only a small decrease in task performance over using a full-resolution model.
  arXiv  Detail & Related papers  (2025-08-26T17:03:39Z)
• Real-Time Model Checking for Closed-Loop Robot Reactive Planning [0.0]
  We present a new application of model checking which achieves real-time multi-step planning and obstacle avoidance on a real autonomous robot.<n>We have developed a small, purpose-built model checking algorithm which generates plans in situ based on "core" knowledge and attention as found in biological agents.<n>Our approach is based on chaining temporary control systems which are spawned to counteract disturbances in the local environment.
  arXiv  Detail & Related papers  (2025-08-26T16:49:30Z)
• Action Flow Matching for Continual Robot Learning [57.698553219660376]
  Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks.<n>We introduce a generative framework leveraging flow matching for online robot dynamics model alignment.<n>We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently.
  arXiv  Detail & Related papers  (2025-04-25T16:26:15Z)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models [61.145371212636505]
  Reinforcement learning (RL) learns policies through trial and error, and optimal control, which plans actions using a learned or known dynamics model.<n>We systematically analyze the performance of different RL and control-based methods under datasets of varying quality.<n>Our results show that model-free RL excels when abundant, high-quality data is available, while model-based planning excels in generalization to novel environment layouts, trajectory stitching, and data-efficiency.
  arXiv  Detail & Related papers  (2025-02-20T18:39:41Z)
• Adaptive Planning with Generative Models under Uncertainty [20.922248169620783]
  Planning with generative models has emerged as an effective decision-making paradigm across a wide range of domains. While continuous replanning at each timestep might seem intuitive because it allows decisions to be made based on the most recent environmental observations, it results in substantial computational challenges. Our work addresses this challenge by introducing a simple adaptive planning policy that leverages the generative model's ability to predict long-horizon state trajectories.
  arXiv  Detail & Related papers  (2024-08-02T18:07:53Z)
• Learning-based adaption of robotic friction models [50.72489248401199]
  We introduce a novel approach to adapt an existing friction model to new dynamics using as little data as possible.<n>Our method does not rely on data with external load during training, eliminating the need for external torque sensors.
  arXiv  Detail & Related papers  (2023-10-25T14:50:15Z)
• Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models [14.171207239507789]
  Learning trajectory generative models as priors for a new planning problem is highly desirable. We propose learning diffusion models as priors to bootstrapping the motion planning problem. Our experiments show that diffusion models are strong priors to encode high-dimensional trajectory distributions of robot motions.
  arXiv  Detail & Related papers  (2023-08-03T06:36:21Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• Temporal Predictive Coding For Model-Based Planning In Latent Space [80.99554006174093]
  We present an information-theoretic approach that employs temporal predictive coding to encode elements in the environment that can be predicted across time. We evaluate our model on a challenging modification of standard DMControl tasks where the background is replaced with natural videos that contain complex but irrelevant information to the planning task.
  arXiv  Detail & Related papers  (2021-06-14T04:31:15Z)
• Goal-Aware Prediction: Learning to Model What Matters [105.43098326577434]
  One of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model and that of the downstream planner or policy. We propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space. We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning.
  arXiv  Detail & Related papers  (2020-07-14T16:42:59Z)
• Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Network [5.000272778136268]
  This study shows that the predictive coding (PC) and active inference (AIF) frameworks can develop better generalization by learning a prior distribution in a low dimensional latent state space. In our proposed model, learning is carried out by inferring optimal latent variables as well as synaptic weights for maximizing the evidence lower bound. Our proposed model was evaluated with both simple and complex robotic tasks in simulation, which demonstrated sufficient generalization in learning with limited training data.
  arXiv  Detail & Related papers  (2020-05-27T06:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.