Control of the Final-Phase of Closed-Loop Visual Grasping using Image-Based Visual Servoing

• URL: http://arxiv.org/abs/2001.05650v2
• Date: Fri, 28 Feb 2020 03:14:44 GMT
• Title: Control of the Final-Phase of Closed-Loop Visual Grasping using Image-Based Visual Servoing
• Authors: Jesse Haviland, Feras Dayoub, Peter Corke
• Abstract summary: Many current robotic grasping controllers are not closed-loop and therefore fail for moving objects. We propose the use of image-based visual servoing to guide the robot to the object-relative grasp pose using camera RGB information.
• Score: 12.368559816913585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper considers the final approach phase of visual-closed-loop grasping where the RGB-D camera is no longer able to provide valid depth information. Many current robotic grasping controllers are not closed-loop and therefore fail for moving objects. Closed-loop grasp controllers based on RGB-D imagery can track a moving object, but fail when the sensor's minimum object distance is violated just before grasping. To overcome this we propose the use of image-based visual servoing (IBVS) to guide the robot to the object-relative grasp pose using camera RGB information. IBVS robustly moves the camera to a goal pose defined implicitly in terms of an image-plane feature configuration. In this work, the goal image feature coordinates are predicted from RGB-D data to enable RGB-only tracking once depth data becomes unavailable -- this enables more reliable grasping of previously unseen moving objects. Experimental results are provided.

Related papers

• Depth-based Privileged Information for Boosting 3D Human Pose Estimation on RGB [48.31210455404533]
  Heatmap-based 3D pose estimator is able to hallucinate depth information from the RGB frames given at inference time. depth information is used exclusively during training by enforcing our RGB-based hallucination network to learn similar features to a backbone pre-trained only on depth data.
  arXiv  Detail & Related papers  (2024-09-17T11:59:34Z)
• ASGrasp: Generalizable Transparent Object Reconstruction and Grasping from RGB-D Active Stereo Camera [9.212504138203222]
  We propose ASGrasp, a 6-DoF grasp detection network that uses an RGB-D active stereo camera. Our system distinguishes itself by its ability to directly utilize raw IR and RGB images for transparent object geometry reconstruction. Our experiments demonstrate that ASGrasp can achieve over 90% success rate for generalizable transparent object grasping.
  arXiv  Detail & Related papers  (2024-05-09T09:44:51Z)
• Reconstructing Hand-Held Objects in 3D [53.277402172488735]
  We present a paradigm for handheld object reconstruction that builds on recent breakthroughs in large language/vision models and 3D object datasets. We use GPT-4(V) to retrieve a 3D object model that matches the object in the image and rigidly align the model to the network-inferred geometry. Experiments demonstrate that MCC-HO achieves state-of-the-art performance on lab and Internet datasets.
  arXiv  Detail & Related papers  (2024-04-09T17:55:41Z)
• ImageManip: Image-based Robotic Manipulation with Affordance-guided Next View Selection [10.162882793554191]
  3D articulated object manipulation is essential for enabling robots to interact with their environment. Many existing studies make use of 3D point clouds as the primary input for manipulation policies. RGB images offer high-resolution observations using cost effective devices but lack spatial 3D geometric information. This framework is designed to capture multiple perspectives of the target object and infer depth information to complement its geometry.
  arXiv  Detail & Related papers  (2023-10-13T12:42:54Z)
• RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery [72.13154206106259]
  We propose a novel pipeline that decouples the 6D pose and size estimation to mitigate the influence of imperfect scales on rigid transformations. Specifically, we leverage a pre-trained monocular estimator to extract local geometric information. A separate branch is designed to directly recover the metric scale of the object based on category-level statistics.
  arXiv  Detail & Related papers  (2023-09-19T02:20:26Z)
• Grasping the Inconspicuous [15.274311118568715]
  We study deep learning 6D pose estimation from RGB images only for transparent object grasping. Experiments demonstrate the effectiveness of RGB image space for grasping transparent objects.
  arXiv  Detail & Related papers  (2022-11-15T14:45:50Z)
• RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control [71.51781695764872]
  We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of an RGB camera that is either handheld or mounted on the robot end effector. A NeRF-like process is used to reconstruct the 3D geometry of the scene, from which the Euclidean full signed distance function (ESDF) is computed. A model predictive control algorithm is then used to control the manipulator to reach a desired pose while avoiding obstacles in the ESDF.
  arXiv  Detail & Related papers  (2022-10-21T01:45:08Z)
• Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images [89.81919625224103]
  Training deep models for RGB-D salient object detection (SOD) often requires a large number of labeled RGB-D images. We present a Dual-Semi RGB-D Salient Object Detection Network (DS-Net) to leverage unlabeled RGB images for boosting RGB-D saliency detection.
  arXiv  Detail & Related papers  (2022-01-01T03:02:27Z)
• Real-time RGBD-based Extended Body Pose Estimation [57.61868412206493]
  We present a system for real-time RGBD-based estimation of 3D human pose. We use parametric 3D deformable human mesh model (SMPL-X) as a representation. We train estimators of body pose and facial expression parameters.
  arXiv  Detail & Related papers  (2021-03-05T13:37:50Z)
• RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Images [42.68340286459079]
  General object grasping is an important yet unsolved problem in the field of robotics. We propose RGBD-Grasp, a pipeline that solves this problem by decoupling 7-DoF grasp detection into two sub-tasks. We achieve state-of-the-art results on GraspNet-1Billion dataset.
  arXiv  Detail & Related papers  (2021-03-03T05:12:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.