Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulation

• URL: http://arxiv.org/abs/2401.08957v2
• Date: Mon, 17 Feb 2025 06:41:03 GMT
• Title: Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulation
• Authors: Kun Wu, Ning Liu, Zhen Zhao, Di Qiu, Jinming Li, Zhengping Che, Zhiyuan Xu, Qinru Qiu, Jian Tang,
• Abstract summary: Current imitation learning (IL) typically discards imperfect data, focusing solely on successful expert data. We introduce a Self-Supervised Data Filtering framework (SSDF) that combines expert and imperfect data to compute quality scores for failed trajectory segments. SSDF can accurately expand the training dataset with high-quality imperfect data and improve the success rates for all robotic manipulation tasks.
• Score: 31.592761504827187
• License:
• Abstract: Improving data utilization, especially for imperfect data from task failures, is crucial for robotic manipulation due to the challenging, time-consuming, and expensive data collection process in the real world. Current imitation learning (IL) typically discards imperfect data, focusing solely on successful expert data. While reinforcement learning (RL) can learn from explorations and failures, the sim2real gap and its reliance on dense reward and online exploration make it difficult to apply effectively in real-world scenarios. In this work, we aim to conquer the challenge of leveraging imperfect data without the need for reward information to improve the model performance for robotic manipulation in an offline manner. Specifically, we introduce a Self-Supervised Data Filtering framework (SSDF) that combines expert and imperfect data to compute quality scores for failed trajectory segments. High-quality segments from the failed data are used to expand the training dataset. Then, the enhanced dataset can be used with any downstream policy learning method for robotic manipulation tasks. Extensive experiments on the ManiSkill2 benchmark built on the high-fidelity Sapien simulator and real-world robotic manipulation tasks using the Franka robot arm demonstrated that the SSDF can accurately expand the training dataset with high-quality imperfect data and improve the success rates for all robotic manipulation tasks.

Related papers

• Simulation as Reality? The Effectiveness of LLM-Generated Data in Open-ended Question Assessment [7.695222586877482]
  This study investigates the potential and gap of simulative data to address the limitation of AI-based assessment tools. Our findings reveal that while simulative data demonstrates promising results in training automated assessment models, its effectiveness has notable limitations. The absence of real-world noise and biases, which are also present in over-processed real-world data, contributes to this limitation.
  arXiv  Detail & Related papers  (2025-02-10T11:40:11Z)
• So You Think You Can Scale Up Autonomous Robot Data Collection? [22.7035324720716]
  Reinforcement learning (RL) comes with the promise of enabling autonomous data collection. It remains challenging to scale in the real-world partly due to the significant effort required for environment design and instrumentation. Imitation learning (IL) methods require little to no environment design effort, but instead require significant human supervision.
  arXiv  Detail & Related papers  (2024-11-04T05:31:35Z)
• Guided Data Augmentation for Offline Reinforcement Learning and Imitation Learning [3.586527534935176]
  In offline reinforcement learning (RL), an RL agent learns to solve a task using only a fixed dataset of previously collected data. We propose Guided Data Augmentation (GuDA), a human-guided DA framework that generates expert-quality augmented data. GuDA enables learning given a small initial dataset of potentially suboptimal experience.
  arXiv  Detail & Related papers  (2023-10-27T16:34:00Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models [56.27786433792638]
  STAR is a data generation method that leverages Large Language Models (LLMs) to synthesize data instances. We design fine-grained step-by-step instructions to obtain the initial data instances. Our experiments show that the data generated by STAR significantly improve the performance of low-resource event extraction and relation extraction tasks.
  arXiv  Detail & Related papers  (2023-05-24T12:15:19Z)
• Offline Robot Reinforcement Learning with Uncertainty-Guided Human Expert Sampling [11.751910133386254]
  Recent advances in batch (offline) reinforcement learning have shown promising results in learning from available offline data. We propose a novel approach that uses uncertainty estimation to trigger the injection of human demonstration data. Our experiments show that this approach is more sample efficient when compared to a naive way of combining expert data with data collected from a sub-optimal agent.
  arXiv  Detail & Related papers  (2022-12-16T01:41:59Z)
• Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning [70.70104870417784]
  Reinforcement learning (RL) algorithms hold the promise of enabling autonomous skill acquisition for robotic systems. In practice, real-world robotic RL typically requires time consuming data collection and frequent human intervention to reset the environment. In this work, we study how these challenges can be tackled by effective utilization of diverse offline datasets collected from previously seen tasks.
  arXiv  Detail & Related papers  (2022-07-11T08:31:22Z)
• Deep Reinforcement Learning Assisted Federated Learning Algorithm for Data Management of IIoT [82.33080550378068]
  The continuous expanded scale of the industrial Internet of Things (IIoT) leads to IIoT equipments generating massive amounts of user data every moment. How to manage these time series data in an efficient and safe way in the field of IIoT is still an open issue. This paper studies the FL technology applications to manage IIoT equipment data in wireless network environments.
  arXiv  Detail & Related papers  (2022-02-03T07:12:36Z)
• Efficient Realistic Data Generation Framework leveraging Deep Learning-based Human Digitization [0.0]
  The proposed method takes as input real background images and populates them with human figures in various poses. A benchmarking and evaluation in the corresponding tasks shows that synthetic data can be effectively used as a supplement to real data.
  arXiv  Detail & Related papers  (2021-06-28T08:07:31Z)
• A Framework for Efficient Robotic Manipulation [79.10407063260473]
  We show that a single robotic arm can learn sparse-reward manipulation policies from pixels. We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels.
  arXiv  Detail & Related papers  (2020-12-14T22:18:39Z)
• AWAC: Accelerating Online Reinforcement Learning with Offline Datasets [84.94748183816547]
  We show that our method, advantage weighted actor critic (AWAC), enables rapid learning of skills with a combination of prior demonstration data and online experience. Our results show that incorporating prior data can reduce the time required to learn a range of robotic skills to practical time-scales.
  arXiv  Detail & Related papers  (2020-06-16T17:54:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.