Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning

• URL: http://arxiv.org/abs/2412.00086v1
• Date: Wed, 27 Nov 2024 03:33:42 GMT
• Title: Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning
• Authors: Neel Jawale, Byron Boots, Balakumar Sundaralingam, Mohak Bhardwaj,
• Abstract summary: We propose an approach that combines batch reinforcement learning (RL) with model-predictive control (MPC) We validate the proposed approach through extensive simulated and real-world experiments on a Franka Panda robot performing the robot waiter task.
• Score: 24.079032278280447
• License:
• Abstract: We investigate the problem of teaching a robot manipulator to perform dynamic non-prehensile object transport, also known as the `robot waiter' task, from a limited set of real-world demonstrations. We propose an approach that combines batch reinforcement learning (RL) with model-predictive control (MPC) by pretraining an ensemble of value functions from demonstration data, and utilizing them online within an uncertainty-aware MPC scheme to ensure robustness to limited data coverage. Our approach is straightforward to integrate with off-the-shelf MPC frameworks and enables learning solely from task space demonstrations with sparsely labeled transitions, while leveraging MPC to ensure smooth joint space motions and constraint satisfaction. We validate the proposed approach through extensive simulated and real-world experiments on a Franka Panda robot performing the robot waiter task and demonstrate robust deployment of value functions learned from 50-100 demonstrations. Furthermore, our approach enables generalization to novel objects not seen during training and can improve upon suboptimal demonstrations. We believe that such a framework can reduce the burden of providing extensive demonstrations and facilitate rapid training of robot manipulators to perform non-prehensile manipulation tasks. Project videos and supplementary material can be found at: https://sites.google.com/view/cvmpc.

Related papers

• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  We introduce a novel framework for learning world models. By providing a scalable and robust framework, we pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Screw Geometry Meets Bandits: Incremental Acquisition of Demonstrations to Generate Manipulation Plans [9.600625243282618]
  We study the problem of methodically obtaining a sufficient set of kinesthetic demonstrations, one at a time. We present a novel approach to address these open problems using (i) a screw geometric representation to generate manipulation plans from demonstrations. We present experimental results on two example manipulation tasks, namely, pouring and scooping, to illustrate our approach.
  arXiv  Detail & Related papers  (2024-10-23T20:57:56Z)
• RAMPA: Robotic Augmented Reality for Machine Programming by DemonstrAtion [4.963604518596734]
  This paper introduces Robotic Augmented Reality for Machine Programming by Demonstration (RAMPA) RAMPA is the first ML-integrated, XR-driven end-to-end robotic system, allowing training and deployment of ML models such as ProMPs on the fly. Our approach enables in-situ data recording, visualization, and fine-tuning of skill demonstrations directly within the user's physical environment.
  arXiv  Detail & Related papers  (2024-10-17T10:21:28Z)
• LLaRA: Supercharging Robot Learning Data for Vision-Language Policy [56.505551117094534]
  We introduce LLaRA: Large Language and Robotics Assistant, a framework that formulates robot action policy as visuo-textual conversations. First, we present an automated pipeline to generate conversation-style instruction tuning data for robots from existing behavior cloning datasets. We show that a VLM finetuned with a limited amount of such datasets can produce meaningful action decisions for robotic control.
  arXiv  Detail & Related papers  (2024-06-28T17:59:12Z)
• Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks [54.60571399091711]
  Large Language Models (LLMs) have achieved impressive results in creating robotic agents for performing open vocabulary tasks. We present an interactive planning technique for partially observable tasks using LLMs.
  arXiv  Detail & Related papers  (2023-12-11T22:54:44Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation [36.43143326197769]
  Track-Any-Point (TAP) models isolate the relevant motion in a demonstration, and parameterize a low-level controller to reproduce this motion across changes in the scene configuration. We show this results in robust robot policies that can solve complex object-arrangement tasks such as shape-matching, stacking, and even full path-following tasks such as applying glue and sticking objects together.
  arXiv  Detail & Related papers  (2023-08-30T11:57:04Z)
• GAN-MPC: Training Model Predictive Controllers with Parameterized Cost Functions using Demonstrations from Non-identical Experts [14.291720751625585]
  We propose a generative adversarial network (GAN) to minimize the Jensen-Shannon divergence between the state-trajectory distributions of the demonstrator and the imitator. We evaluate our approach on a variety of simulated robotics tasks of DeepMind Control suite.
  arXiv  Detail & Related papers  (2023-05-30T15:15:30Z)
• Human-in-the-Loop Imitation Learning using Remote Teleoperation [72.2847988686463]
  We build a data collection system tailored to 6-DoF manipulation settings. We develop an algorithm to train the policy iteratively on new data collected by the system. We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators.
  arXiv  Detail & Related papers  (2020-12-12T05:30:35Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.