Adapting Skills to Novel Grasps: A Self-Supervised Approach

• URL: http://arxiv.org/abs/2408.00178v1
• Date: Wed, 31 Jul 2024 22:18:09 GMT
• Title: Adapting Skills to Novel Grasps: A Self-Supervised Approach
• Authors: Georgios Papagiannis, Kamil Dreczkowski, Vitalis Vosylius, Edward Johns,
• Abstract summary: We study the problem of adapting manipulation trajectories involving grasped objects (e.g. tools) defined for a single grasp pose to novel grasp poses. A common approach to address this is to define a new trajectory for each possible grasp explicitly, but this is highly inefficient. We propose a method to adapt such trajectories directly while only requiring a period of self-supervised data collection.
• Score: 11.030216531050044
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we study the problem of adapting manipulation trajectories involving grasped objects (e.g. tools) defined for a single grasp pose to novel grasp poses. A common approach to address this is to define a new trajectory for each possible grasp explicitly, but this is highly inefficient. Instead, we propose a method to adapt such trajectories directly while only requiring a period of self-supervised data collection, during which a camera observes the robot's end-effector moving with the object rigidly grasped. Importantly, our method requires no prior knowledge of the grasped object (such as a 3D CAD model), it can work with RGB images, depth images, or both, and it requires no camera calibration. Through a series of real-world experiments involving 1360 evaluations, we find that self-supervised RGB data consistently outperforms alternatives that rely on depth images including several state-of-the-art pose estimation methods. Compared to the best-performing baseline, our method results in an average of 28.5% higher success rate when adapting manipulation trajectories to novel grasps on several everyday tasks. Videos of the experiments are available on our webpage at https://www.robot-learning.uk/adapting-skills

Related papers

• Good Grasps Only: A data engine for self-supervised fine-tuning of pose estimation using grasp poses for verification [0.0]
  We present a novel method for self-supervised fine-tuning of pose estimation for bin-picking. Our approach enables the robot to automatically obtain training data without manual labeling. Our pipeline allows the system to fine-tune while the process is running, removing the need for a learning phase.
  arXiv  Detail & Related papers  (2024-09-17T19:26:21Z)
• RDPN6D: Residual-based Dense Point-wise Network for 6Dof Object Pose Estimation Based on RGB-D Images [13.051302134031808]
  We introduce a novel method for calculating the 6DoF pose of an object using a single RGB-D image. Unlike existing methods that either directly predict objects' poses or rely on sparse keypoints for pose recovery, our approach addresses this challenging task using dense correspondence.
  arXiv  Detail & Related papers  (2024-05-14T10:10:45Z)
• The Change You Want to See (Now in 3D) [65.61789642291636]
  The goal of this paper is to detect what has changed, if anything, between two "in the wild" images of the same 3D scene. We contribute a change detection model that is trained entirely on synthetic data and is class-agnostic. We release a new evaluation dataset consisting of real-world image pairs with human-annotated differences.
  arXiv  Detail & Related papers  (2023-08-21T01:59:45Z)
• Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image [85.91935485902708]
  We show that the key to a zero-shot single-view metric depth model lies in the combination of large-scale data training and resolving the metric ambiguity from various camera models. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problems and can be effortlessly plugged into existing monocular models. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images.
  arXiv  Detail & Related papers  (2023-07-20T16:14:23Z)
• Unseen Object 6D Pose Estimation: A Benchmark and Baselines [62.8809734237213]
  We propose a new task that enables and facilitates algorithms to estimate the 6D pose estimation of novel objects during testing. We collect a dataset with both real and synthetic images and up to 48 unseen objects in the test set. By training an end-to-end 3D correspondences network, our method finds corresponding points between an unseen object and a partial view RGBD image accurately and efficiently.
  arXiv  Detail & Related papers  (2022-06-23T16:29:53Z)
• Perspective Flow Aggregation for Data-Limited 6D Object Pose Estimation [121.02948087956955]
  For some applications, such as those in space or deep under water, acquiring real images, even unannotated, is virtually impossible. We propose a method that can be trained solely on synthetic images, or optionally using a few additional real images. It performs on par with methods that require annotated real images for training when not using any, and outperforms them considerably when using as few as twenty real images.
  arXiv  Detail & Related papers  (2022-03-18T10:20:21Z)
• GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation [71.83992173720311]
  6D pose estimation from a single RGB image is a fundamental task in computer vision. We propose a simple yet effective Geometry-guided Direct Regression Network (GDR-Net) to learn the 6D pose in an end-to-end manner. Our approach remarkably outperforms state-of-the-art methods on LM, LM-O and YCB-V datasets.
  arXiv  Detail & Related papers  (2021-02-24T09:11:31Z)
• CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning [74.53664270194643]
  Modern monocular 6D pose estimation methods can only cope with a handful of object instances. We propose a novel method for class-level monocular 6D pose estimation, coupled with metric shape retrieval. We experimentally demonstrate that we can retrieve precise 6D poses and metric shapes from a single RGB image.
  arXiv  Detail & Related papers  (2020-03-12T15:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.