Movement Primitive Diffusion: Learning Gentle Robotic Manipulation of Deformable Objects

• URL: http://arxiv.org/abs/2312.10008v2
• Date: Mon, 10 Jun 2024 08:11:00 GMT
• Title: Movement Primitive Diffusion: Learning Gentle Robotic Manipulation of Deformable Objects
• Authors: Paul Maria Scheikl, Nicolas Schreiber, Christoph Haas, Niklas Freymuth, Gerhard Neumann, Rudolf Lioutikov, Franziska Mathis-Ullrich,
• Abstract summary: Movement Primitive Diffusion (MPD) is a novel method for imitation learning (IL) in robot-assisted surgery. MPD combines the versatility of diffusion-based imitation learning (DIL) with the high-quality motion generation capabilities of Probabilistic Dynamic Movement Primitives (ProDMPs) We evaluate MPD across various simulated and real world robotic tasks on both state and image observations.
• Score: 14.446751610174868
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Policy learning in robot-assisted surgery (RAS) lacks data efficient and versatile methods that exhibit the desired motion quality for delicate surgical interventions. To this end, we introduce Movement Primitive Diffusion (MPD), a novel method for imitation learning (IL) in RAS that focuses on gentle manipulation of deformable objects. The approach combines the versatility of diffusion-based imitation learning (DIL) with the high-quality motion generation capabilities of Probabilistic Dynamic Movement Primitives (ProDMPs). This combination enables MPD to achieve gentle manipulation of deformable objects, while maintaining data efficiency critical for RAS applications where demonstration data is scarce. We evaluate MPD across various simulated and real world robotic tasks on both state and image observations. MPD outperforms state-of-the-art DIL methods in success rate, motion quality, and data efficiency. Project page: https://scheiklp.github.io/movement-primitive-diffusion/

Related papers

• Prediction with Action: Visual Policy Learning via Joint Denoising Process [14.588908033404474]
  PAD is a visual policy learning framework that unifies image Prediction and robot Action. DiT seamlessly integrates images and robot states, enabling the simultaneous prediction of future images and robot actions. Pad outperforms previous methods, achieving a significant 26.3% relative improvement on the full Metaworld benchmark.
  arXiv  Detail & Related papers  (2024-11-27T09:54:58Z)
• VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulation [79.00294932026266]
  VidMan is a novel framework that employs a two-stage training mechanism to enhance stability and improve data utilization efficiency. Our framework outperforms state-of-the-art baseline model GR-1 on the CALVIN benchmark, achieving a 11.7% relative improvement, and demonstrates over 9% precision gains on the OXE small-scale dataset.
  arXiv  Detail & Related papers  (2024-11-14T03:13:26Z)
• ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Model [9.525806425270428]
  We present emphReinDiffuse that combines reinforcement learning with motion diffusion model to generate physically credible human motions. Our method adapts Motion Diffusion Model to output a parameterized distribution of actions, making them compatible with reinforcement learning paradigms. Our approach outperforms existing state-of-the-art models on two major datasets, HumanML3D and KIT-ML.
  arXiv  Detail & Related papers  (2024-10-09T16:24:11Z)
• DiffGen: Robot Demonstration Generation via Differentiable Physics Simulation, Differentiable Rendering, and Vision-Language Model [72.66465487508556]
  DiffGen is a novel framework that integrates differentiable physics simulation, differentiable rendering, and a vision-language model. It can generate realistic robot demonstrations by minimizing the distance between the embedding of the language instruction and the embedding of the simulated observation. Experiments demonstrate that with DiffGen, we could efficiently and effectively generate robot data with minimal human effort or training time.
  arXiv  Detail & Related papers  (2024-05-12T15:38:17Z)
• Unsupervised Learning of Effective Actions in Robotics [0.9374652839580183]
  Current state-of-the-art action representations in robotics lack proper effect-driven learning of the robot's actions. We propose an unsupervised algorithm to discretize a continuous motion space and generate "action prototypes" We evaluate our method on a simulated stair-climbing reinforcement learning task.
  arXiv  Detail & Related papers  (2024-04-03T13:28:52Z)
• Any-point Trajectory Modeling for Policy Learning [64.23861308947852]
  We introduce Any-point Trajectory Modeling (ATM) to predict future trajectories of arbitrary points within a video frame. ATM outperforms strong video pre-training baselines by 80% on average. We show effective transfer learning of manipulation skills from human videos and videos from a different robot morphology.
  arXiv  Detail & Related papers  (2023-12-28T23:34:43Z)
• Dynamic-Resolution Model Learning for Object Pile Manipulation [33.05246884209322]
  We investigate how to learn dynamic and adaptive representations at different levels of abstraction to achieve the optimal trade-off between efficiency and effectiveness. Specifically, we construct dynamic-resolution particle representations of the environment and learn a unified dynamics model using graph neural networks (GNNs) We show that our method achieves significantly better performance than state-of-the-art fixed-resolution baselines at the gathering, sorting, and redistribution of granular object piles.
  arXiv  Detail & Related papers  (2023-06-29T05:51:44Z)
• MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction [34.978017200500005]
  We propose Multimodal Interactive Latent Dynamics (MILD) to address the problem of two-party physical Human-Robot Interactions (HRIs) We learn the interaction dynamics from demonstrations, using Hidden Semi-Markov Models (HSMMs) to model the joint distribution of the interacting agents in the latent space of a Variational Autoencoder (VAE) MILD generates more accurate trajectories for the controlled agent (robot) when conditioned on the observed agent's (human) trajectory.
  arXiv  Detail & Related papers  (2022-10-22T11:25:11Z)
• One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video [71.43912903508765]
  MDAL is a dynamic online adaptive learning scheme for instrument segmentation in robot-assisted surgery. It learns the general knowledge of instruments and the fast adaptation ability through the video-specific meta-learning paradigm. It outperforms other state-of-the-art methods on two datasets.
  arXiv  Detail & Related papers  (2021-03-24T05:02:18Z)
• Learning to Shift Attention for Motion Generation [55.61994201686024]
  One challenge of motion generation using robot learning from demonstration techniques is that human demonstrations follow a distribution with multiple modes for one task query. Previous approaches fail to capture all modes or tend to average modes of the demonstrations and thus generate invalid trajectories. We propose a motion generation model with extrapolation ability to overcome this problem.
  arXiv  Detail & Related papers  (2021-02-24T09:07:52Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.