Monocular One-Shot Metric-Depth Alignment for RGB-Based Robot Grasping

• URL: http://arxiv.org/abs/2506.17110v1
• Date: Fri, 20 Jun 2025 16:11:20 GMT
• Title: Monocular One-Shot Metric-Depth Alignment for RGB-Based Robot Grasping
• Authors: Teng Guo, Baichuan Huang, Jingjin Yu,
• Abstract summary: We propose a novel framework, Monocular One-shot Metric-depth Alignment (MOMA), to recover metric depth from a single RGB image.<n>MOMA performs scale-rotation-shift alignments during camera calibration, guided by sparse ground-truth depth points.<n>Real-world experiments on tabletop 2-finger grasping and suction-based bin-picking applications show MOMA achieves high success rates in diverse tasks.
• Score: 26.7709114619056
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate 6D object pose estimation is a prerequisite for successfully completing robotic prehensile and non-prehensile manipulation tasks. At present, 6D pose estimation for robotic manipulation generally relies on depth sensors based on, e.g., structured light, time-of-flight, and stereo-vision, which can be expensive, produce noisy output (as compared with RGB cameras), and fail to handle transparent objects. On the other hand, state-of-the-art monocular depth estimation models (MDEMs) provide only affine-invariant depths up to an unknown scale and shift. Metric MDEMs achieve some successful zero-shot results on public datasets, but fail to generalize. We propose a novel framework, Monocular One-shot Metric-depth Alignment (MOMA), to recover metric depth from a single RGB image, through a one-shot adaptation building on MDEM techniques. MOMA performs scale-rotation-shift alignments during camera calibration, guided by sparse ground-truth depth points, enabling accurate depth estimation without additional data collection or model retraining on the testing setup. MOMA supports fine-tuning the MDEM on transparent objects, demonstrating strong generalization capabilities. Real-world experiments on tabletop 2-finger grasping and suction-based bin-picking applications show MOMA achieves high success rates in diverse tasks, confirming its effectiveness.

Related papers

• Reasoning and Learning a Perceptual Metric for Self-Training of Reflective Objects in Bin-Picking with a Low-cost Camera [10.976379239028455]
  Bin-picking of metal objects using low-cost RGB-D cameras often suffers from sparse depth information and reflective surface textures.<n>We propose a two-stage framework consisting of a metric learning stage and a self-training stage.<n>Our approach outperforms several state-of-the-art methods on both the ROBI dataset and our newly introduced Self-ROBI dataset.
  arXiv  Detail & Related papers  (2025-03-26T04:03:51Z)
• UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler [62.06785782635153]
  We propose a new model, UniDepthV2, capable of reconstructing metric 3D scenes from solely single images across domains.<n>UniDepthV2 directly predicts metric 3D points from the input image at inference time without any additional information.<n>Our model exploits a pseudo-spherical output representation, which disentangles the camera and depth representations.
  arXiv  Detail & Related papers  (2025-02-27T14:03:15Z)
• RoMeO: Robust Metric Visual Odometry [11.381243799745729]
  Visual odometry (VO) aims to estimate camera poses from visual inputs -- a fundamental building block for many applications such as VR/AR and robotics.<n>Existing approaches lack robustness under this challenging scenario and fail to generalize to unseen data (especially outdoors)<n>We propose Robust Metric Visual Odometry (RoMeO), a novel method that resolves these issues leveraging priors from pre-trained depth models.
  arXiv  Detail & Related papers  (2024-12-16T08:08:35Z)
• Category-level Object Detection, Pose Estimation and Reconstruction from Stereo Images [15.921719523588996]
  Existing monocular and RGB-D methods suffer from scale ambiguity due to missing or depth measurements. We present CODERS, a one-stage approach for Category-level Object Detection, pose Estimation and Reconstruction from Stereo images. Our dataset, code, and demos will be available on our project page.
  arXiv  Detail & Related papers  (2024-07-09T15:59:03Z)
• VFMM3D: Releasing the Potential of Image by Vision Foundation Model for Monocular 3D Object Detection [80.62052650370416]
  monocular 3D object detection holds significant importance across various applications, including autonomous driving and robotics. In this paper, we present VFMM3D, an innovative framework that leverages the capabilities of Vision Foundation Models (VFMs) to accurately transform single-view images into LiDAR point cloud representations.
  arXiv  Detail & Related papers  (2024-04-15T03:12:12Z)
• Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation [74.28509379811084]
  Metric3D v2 is a geometric foundation model for zero-shot metric depth and surface normal estimation from a single image.<n>We propose solutions for both metric depth estimation and surface normal estimation.<n>Our method enables the accurate recovery of metric 3D structures on randomly collected internet images.
  arXiv  Detail & Related papers  (2024-03-22T02:30:46Z)
• RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery [72.13154206106259]
  We propose a novel pipeline that decouples the 6D pose and size estimation to mitigate the influence of imperfect scales on rigid transformations. Specifically, we leverage a pre-trained monocular estimator to extract local geometric information. A separate branch is designed to directly recover the metric scale of the object based on category-level statistics.
  arXiv  Detail & Related papers  (2023-09-19T02:20:26Z)
• Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image [85.91935485902708]
  We show that the key to a zero-shot single-view metric depth model lies in the combination of large-scale data training and resolving the metric ambiguity from various camera models. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problems and can be effortlessly plugged into existing monocular models. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images.
  arXiv  Detail & Related papers  (2023-07-20T16:14:23Z)
• TransPose: A Transformer-based 6D Object Pose Estimation Network with Depth Refinement [5.482532589225552]
  We propose TransPose, an improved Transformer-based 6D pose estimation with a depth refinement module. The architecture takes in only an RGB image as input with no additional supplementing modalities such as depth or thermal images. A novel depth refinement module is then used alongside the predicted centers, 6D poses and depth patches to refine the accuracy of the estimated 6D pose.
  arXiv  Detail & Related papers  (2023-07-09T17:33:13Z)
• DeepRM: Deep Recurrent Matching for 6D Pose Refinement [77.34726150561087]
  DeepRM is a novel recurrent network architecture for 6D pose refinement. The architecture incorporates LSTM units to propagate information through each refinement step. DeepRM achieves state-of-the-art performance on two widely accepted challenging datasets.
  arXiv  Detail & Related papers  (2022-05-28T16:18:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.