SWBT: Similarity Weighted Behavior Transformer with the Imperfect Demonstration for Robotic Manipulation

• URL: http://arxiv.org/abs/2401.08957v1
• Date: Wed, 17 Jan 2024 04:15:56 GMT
• Title: SWBT: Similarity Weighted Behavior Transformer with the Imperfect Demonstration for Robotic Manipulation
• Authors: Kun Wu, Ning Liu, Zhen Zhao, Di Qiu, Jinming Li, Zhengping Che, Zhiyuan Xu, Qinru Qiu, Jian Tang
• Abstract summary: We propose a novel framework named Similarity Weighted Behavior Transformer (SWBT) SWBT effectively learn from both expert and imperfect demonstrations without interaction with environments. We are the first to attempt to integrate imperfect demonstrations into the offline imitation learning setting for robot manipulation tasks.
• Score: 32.78083518963342
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imitation learning (IL), aiming to learn optimal control policies from expert demonstrations, has been an effective method for robot manipulation tasks. However, previous IL methods either only use expensive expert demonstrations and omit imperfect demonstrations or rely on interacting with the environment and learning from online experiences. In the context of robotic manipulation, we aim to conquer the above two challenges and propose a novel framework named Similarity Weighted Behavior Transformer (SWBT). SWBT effectively learn from both expert and imperfect demonstrations without interaction with environments. We reveal that the easy-to-get imperfect demonstrations, such as forward and inverse dynamics, significantly enhance the network by learning fruitful information. To the best of our knowledge, we are the first to attempt to integrate imperfect demonstrations into the offline imitation learning setting for robot manipulation tasks. Extensive experiments on the ManiSkill2 benchmark built on the high-fidelity Sapien simulator and real-world robotic manipulation tasks demonstrated that the proposed method can extract better features and improve the success rates for all tasks. Our code will be released upon acceptance of the paper.

Related papers

• Simulation as Reality? The Effectiveness of LLM-Generated Data in Open-ended Question Assessment [7.695222586877482]
  This study investigates the potential and gap of simulative data to address the limitation of AI-based assessment tools. Our findings reveal that while simulative data demonstrates promising results in training automated assessment models, its effectiveness has notable limitations. The absence of real-world noise and biases, which are also present in over-processed real-world data, contributes to this limitation.
  arXiv  Detail & Related papers  (2025-02-10T11:40:11Z)
• So You Think You Can Scale Up Autonomous Robot Data Collection? [22.7035324720716]
  Reinforcement learning (RL) comes with the promise of enabling autonomous data collection. It remains challenging to scale in the real-world partly due to the significant effort required for environment design and instrumentation. Imitation learning (IL) methods require little to no environment design effort, but instead require significant human supervision.
  arXiv  Detail & Related papers  (2024-11-04T05:31:35Z)
• Guided Data Augmentation for Offline Reinforcement Learning and Imitation Learning [3.586527534935176]
  In offline reinforcement learning (RL), an RL agent learns to solve a task using only a fixed dataset of previously collected data. We propose Guided Data Augmentation (GuDA), a human-guided DA framework that generates expert-quality augmented data. GuDA enables learning given a small initial dataset of potentially suboptimal experience.
  arXiv  Detail & Related papers  (2023-10-27T16:34:00Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models [56.27786433792638]
  STAR is a data generation method that leverages Large Language Models (LLMs) to synthesize data instances. We design fine-grained step-by-step instructions to obtain the initial data instances. Our experiments show that the data generated by STAR significantly improve the performance of low-resource event extraction and relation extraction tasks.
  arXiv  Detail & Related papers  (2023-05-24T12:15:19Z)
• Offline Robot Reinforcement Learning with Uncertainty-Guided Human Expert Sampling [11.751910133386254]
  Recent advances in batch (offline) reinforcement learning have shown promising results in learning from available offline data. We propose a novel approach that uses uncertainty estimation to trigger the injection of human demonstration data. Our experiments show that this approach is more sample efficient when compared to a naive way of combining expert data with data collected from a sub-optimal agent.
  arXiv  Detail & Related papers  (2022-12-16T01:41:59Z)
• Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning [70.70104870417784]
  Reinforcement learning (RL) algorithms hold the promise of enabling autonomous skill acquisition for robotic systems. In practice, real-world robotic RL typically requires time consuming data collection and frequent human intervention to reset the environment. In this work, we study how these challenges can be tackled by effective utilization of diverse offline datasets collected from previously seen tasks.
  arXiv  Detail & Related papers  (2022-07-11T08:31:22Z)
• Deep Reinforcement Learning Assisted Federated Learning Algorithm for Data Management of IIoT [82.33080550378068]
  The continuous expanded scale of the industrial Internet of Things (IIoT) leads to IIoT equipments generating massive amounts of user data every moment. How to manage these time series data in an efficient and safe way in the field of IIoT is still an open issue. This paper studies the FL technology applications to manage IIoT equipment data in wireless network environments.
  arXiv  Detail & Related papers  (2022-02-03T07:12:36Z)
• Efficient Realistic Data Generation Framework leveraging Deep Learning-based Human Digitization [0.0]
  The proposed method takes as input real background images and populates them with human figures in various poses. A benchmarking and evaluation in the corresponding tasks shows that synthetic data can be effectively used as a supplement to real data.
  arXiv  Detail & Related papers  (2021-06-28T08:07:31Z)
• A Framework for Efficient Robotic Manipulation [79.10407063260473]
  We show that a single robotic arm can learn sparse-reward manipulation policies from pixels. We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels.
  arXiv  Detail & Related papers  (2020-12-14T22:18:39Z)
• AWAC: Accelerating Online Reinforcement Learning with Offline Datasets [84.94748183816547]
  We show that our method, advantage weighted actor critic (AWAC), enables rapid learning of skills with a combination of prior demonstration data and online experience. Our results show that incorporating prior data can reduce the time required to learn a range of robotic skills to practical time-scales.
  arXiv  Detail & Related papers  (2020-06-16T17:54:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.