OptiGrasp: Optimized Grasp Pose Detection Using RGB Images for Warehouse Picking Robots

• URL: http://arxiv.org/abs/2409.19494v1
• Date: Sun, 29 Sep 2024 00:20:52 GMT
• Title: OptiGrasp: Optimized Grasp Pose Detection Using RGB Images for Warehouse Picking Robots
• Authors: Soofiyan Atar, Yi Li, Markus Grotz, Michael Wolf, Dieter Fox, Joshua Smith,
• Abstract summary: In warehouse environments, robots require robust picking capabilities to manage a wide variety of objects. We propose an innovative approach that leverages foundation models to enhance suction grasping using only RGB images. Our network achieves an 82.3% success rate in real-world applications.
• Score: 27.586777997464644
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In warehouse environments, robots require robust picking capabilities to manage a wide variety of objects. Effective deployment demands minimal hardware, strong generalization to new products, and resilience in diverse settings. Current methods often rely on depth sensors for structural information, which suffer from high costs, complex setups, and technical limitations. Inspired by recent advancements in computer vision, we propose an innovative approach that leverages foundation models to enhance suction grasping using only RGB images. Trained solely on a synthetic dataset, our method generalizes its grasp prediction capabilities to real-world robots and a diverse range of novel objects not included in the training set. Our network achieves an 82.3\% success rate in real-world applications. The project website with code and data will be available at http://optigrasp.github.io.

Related papers

• FetchBot: Object Fetching in Cluttered Shelves via Zero-Shot Sim2Real [22.899593664306717]
  FetchBot is a framework designed to enable zero-shot generalizable and safety-aware object fetching from cluttered shelves in real-world settings. To address data scarcity, we propose an efficient voxel-based method for generating diverse simulated cluttered shelf scenes. To tackle the challenge of limited views, we design a novel architecture for learning multi-view representations.
  arXiv  Detail & Related papers  (2025-02-25T06:32:42Z)
• SparseGrasp: Robotic Grasping via 3D Semantic Gaussian Splatting from Sparse Multi-View RGB Images [125.66499135980344]
  We propose SparseGrasp, a novel open-vocabulary robotic grasping system. SparseGrasp operates efficiently with sparse-view RGB images and handles scene updates fastly. We show that SparseGrasp significantly outperforms state-of-the-art methods in terms of both speed and adaptability.
  arXiv  Detail & Related papers  (2024-12-03T03:56:01Z)
• Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments [26.66666135624716]
  We present Robot Utility Models (RUMs), a framework for training and deploying zero-shot robot policies. RUMs can generalize to new environments without any finetuning. We train five utility models for opening cabinet doors, opening drawers, picking up napkins, picking up paper bags, and reorienting fallen objects.
  arXiv  Detail & Related papers  (2024-09-09T17:59:50Z)
• Close the Sim2real Gap via Physically-based Structured Light Synthetic Data Simulation [16.69742672616517]
  We introduce an innovative structured light simulation system, generating both RGB and physically realistic depth images. We create an RGBD dataset tailored for robotic industrial grasping scenarios. By reducing the sim2real gap and enhancing deep learning training, we facilitate the application of deep learning models in industrial settings.
  arXiv  Detail & Related papers  (2024-07-17T09:57:14Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models. Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning. Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis [72.85526892440251]
  We introduce MetaGraspNet, a large-scale photo-realistic bin picking dataset constructed via physics-based metaverse synthesis. The proposed dataset contains 217k RGBD images across 82 different article types, with full annotations for object detection, amodal perception, keypoint detection, manipulation order and ambidextrous grasp labels for a parallel-jaw and vacuum gripper. We also provide a real dataset consisting of over 2.3k fully annotated high-quality RGBD images, divided into 5 levels of difficulties and an unseen object set to evaluate different object and layout properties.
  arXiv  Detail & Related papers  (2022-08-08T08:15:34Z)
• Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU [58.720142291102135]
  In this paper we propose a pose estimation software exploiting neural network architectures. We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
  arXiv  Detail & Related papers  (2022-04-07T08:53:18Z)
• IFOR: Iterative Flow Minimization for Robotic Object Rearrangement [92.97142696891727]
  IFOR, Iterative Flow Minimization for Robotic Object Rearrangement, is an end-to-end method for the problem of object rearrangement for unknown objects. We show that our method applies to cluttered scenes, and in the real world, while training only on synthetic data.
  arXiv  Detail & Related papers  (2022-02-01T20:03:56Z)
• MetaGraspNet: A Large-Scale Benchmark Dataset for Vision-driven Robotic Grasping via Physics-based Metaverse Synthesis [78.26022688167133]
  We present a large-scale benchmark dataset for vision-driven robotic grasping via physics-based metaverse synthesis. The proposed dataset contains 100,000 images and 25 different object types. We also propose a new layout-weighted performance metric alongside the dataset for evaluating object detection and segmentation performance.
  arXiv  Detail & Related papers  (2021-12-29T17:23:24Z)
• Visual Domain Adaptation for Monocular Depth Estimation on Resource-Constrained Hardware [3.7399856406582086]
  We address the problem of training deep neural networks on resource-constrained hardware in the context of visual domain adaptation. We present an adversarial learning approach that is adapted for training on the device with limited resources. Our experiments show that visual domain adaptation is relevant only for efficient network architectures and training sets.
  arXiv  Detail & Related papers  (2021-08-05T15:10:00Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)
• Virtual to Real adaptation of Pedestrian Detectors [9.432150710329607]
  ViPeD is a new synthetically generated set of images collected with the graphical engine of the video game GTA V - Grand Theft Auto V. We propose two different Domain Adaptation techniques suitable for the pedestrian detection task, but possibly applicable to general object detection. Experiments show that the network trained with ViPeD can generalize over unseen real-world scenarios better than the detector trained over real-world data.
  arXiv  Detail & Related papers  (2020-01-09T14:50:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.