AGILE: Approach-based Grasp Inference Learned from Element Decomposition
- URL: http://arxiv.org/abs/2402.01303v2
- Date: Tue, 6 Feb 2024 10:01:10 GMT
- Title: AGILE: Approach-based Grasp Inference Learned from Element Decomposition
- Authors: MohammadHossein Koosheshi, Hamed Hosseini, Mehdi Tale Masouleh, Ahmad
Kalhor, Mohammad Reza Hairi Yazdi
- Abstract summary: Humans can grasp objects by taking into account hand-object positioning information.
This work proposes a method to enable a robot manipulator to learn the same, grasping objects in the most optimal way.
- Score: 2.812395851874055
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Humans, this species expert in grasp detection, can grasp objects by taking
into account hand-object positioning information. This work proposes a method
to enable a robot manipulator to learn the same, grasping objects in the most
optimal way according to how the gripper has approached the object. Built on
deep learning, the proposed method consists of two main stages. In order to
generalize the network on unseen objects, the proposed Approach-based Grasping
Inference involves an element decomposition stage to split an object into its
main parts, each with one or more annotated grasps for a particular approach of
the gripper. Subsequently, a grasp detection network utilizes the decomposed
elements by Mask R-CNN and the information on the approach of the gripper in
order to detect the element the gripper has approached and the most optimal
grasp. In order to train the networks, the study introduces a robotic grasping
dataset collected in the Coppeliasim simulation environment. The dataset
involves 10 different objects with annotated element decomposition masks and
grasp rectangles. The proposed method acquires a 90% grasp success rate on seen
objects and 78% on unseen objects in the Coppeliasim simulation environment.
Lastly, simulation-to-reality domain adaptation is performed by applying
transformations on the training set collected in simulation and augmenting the
dataset, which results in a 70% physical grasp success performance using a
Delta parallel robot and a 2 -fingered gripper.
Related papers
- Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking [59.87033229815062]
Articulated object manipulation requires precise object interaction, where the object's axis must be carefully considered.
Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics.
We present a closed-loop pipeline integrating interactive perception with online axis estimation from segmented 3D point clouds.
arXiv Detail & Related papers (2024-09-24T17:59:56Z) - LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion [79.22197702626542]
This paper introduces a framework that explores amodal segmentation for robotic grasping in cluttered scenes.
We propose a Linear-fusion Attention-guided Convolutional Network (LAC-Net)
The results on different datasets show that our method achieves state-of-the-art performance.
arXiv Detail & Related papers (2024-08-06T14:50:48Z) - ICGNet: A Unified Approach for Instance-Centric Grasping [42.92991092305974]
We introduce an end-to-end architecture for object-centric grasping.
We show the effectiveness of the proposed method by extensively evaluating it against state-of-the-art methods on synthetic datasets.
arXiv Detail & Related papers (2024-01-18T12:41:41Z) - Sim2Real Bilevel Adaptation for Object Surface Classification using Vision-Based Tactile Sensors [14.835051543002164]
We train a Diffusion Model to bridge the Sim2Real gap in the field of vision-based tactile sensors for classifying object surfaces.
We employ a simulator to generate images by uniformly sampling the surface of objects from the YCB Model Set.
These simulated images are then translated into the real domain using the Diffusion Model and automatically labeled to train a classifier.
arXiv Detail & Related papers (2023-11-02T16:37:27Z) - Self-Supervised Interactive Object Segmentation Through a
Singulation-and-Grasping Approach [9.029861710944704]
We propose a robot learning approach to interact with novel objects and collect each object's training label.
The Singulation-and-Grasping (SaG) policy is trained through end-to-end reinforcement learning.
Our system achieves 70% singulation success rate in simulated cluttered scenes.
arXiv Detail & Related papers (2022-07-19T15:01:36Z) - Masked Transformer for Neighhourhood-aware Click-Through Rate Prediction [74.52904110197004]
We propose Neighbor-Interaction based CTR prediction, which put this task into a Heterogeneous Information Network (HIN) setting.
In order to enhance the representation of the local neighbourhood, we consider four types of topological interaction among the nodes.
We conduct comprehensive experiments on two real world datasets and the experimental results show that our proposed method outperforms state-of-the-art CTR models significantly.
arXiv Detail & Related papers (2022-01-25T12:44:23Z) - Depth-aware Object Segmentation and Grasp Detection for Robotic Picking
Tasks [13.337131101813934]
We present a novel deep neural network architecture for joint class-agnostic object segmentation and grasp detection for robotic picking tasks.
We introduce depth-aware Coordinate Convolution (CoordConv), a method to increase accuracy for point proposal based object instance segmentation.
We evaluate the accuracy of grasp detection and instance segmentation on challenging robotic picking datasets, namely Sil'eane and OCID_grasp.
arXiv Detail & Related papers (2021-11-22T11:06:33Z) - Learning Co-segmentation by Segment Swapping for Retrieval and Discovery [67.6609943904996]
The goal of this work is to efficiently identify visually similar patterns from a pair of images.
We generate synthetic training pairs by selecting object segments in an image and copy-pasting them into another image.
We show our approach provides clear improvements for artwork details retrieval on the Brueghel dataset.
arXiv Detail & Related papers (2021-10-29T16:51:16Z) - INVIGORATE: Interactive Visual Grounding and Grasping in Clutter [56.00554240240515]
INVIGORATE is a robot system that interacts with human through natural language and grasps a specified object in clutter.
We train separate neural networks for object detection, for visual grounding, for question generation, and for OBR detection and grasping.
We build a partially observable Markov decision process (POMDP) that integrates the learned neural network modules.
arXiv Detail & Related papers (2021-08-25T07:35:21Z) - Where2Act: From Pixels to Actions for Articulated 3D Objects [54.19638599501286]
We extract highly localized actionable information related to elementary actions such as pushing or pulling for articulated objects with movable parts.
We propose a learning-from-interaction framework with an online data sampling strategy that allows us to train the network in simulation.
Our learned models even transfer to real-world data.
arXiv Detail & Related papers (2021-01-07T18:56:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.