Mind the Shape Gap: A Benchmark and Baseline for Deformation-Aware 6D Pose Estimation of Agricultural Produce

• URL: http://arxiv.org/abs/2603.27429v1
• Date: Sat, 28 Mar 2026 22:06:51 GMT
• Title: Mind the Shape Gap: A Benchmark and Baseline for Deformation-Aware 6D Pose Estimation of Agricultural Produce
• Abstract summary: We introduce PEAR, the first benchmark providing joint 6D pose and per-instance 3D deformation ground truth across 8 produce categories.<n>We show that state-of-the-art methods suffer up to 6x performance degradation when faced with the inherent geometric deviations of real-world produce.<n>We propose SEED, a unified RGB-only framework that jointly predicts 6D pose and explicit lattice deformations from a single image.
• Score: 32.03751544524548
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate 6D pose estimation for robotic harvesting is fundamentally hindered by the biological deformability and high intra-class shape variability of agricultural produce. Instance-level methods fail in this setting, as obtaining exact 3D models for every unique piece of produce is practically infeasible, while category-level approaches that rely on a fixed template suffer significant accuracy degradation when the prior deviates from the true instance geometry. To bridge such lack of robustness to deformation, we introduce PEAR (Pose and dEformation of Agricultural pRoduce), the first benchmark providing joint 6D pose and per-instance 3D deformation ground truth across 8 produce categories, acquired via a robotic manipulator for high annotation accuracy. Using PEAR, we show that state-of-the-art methods suffer up to 6x performance degradation when faced with the inherent geometric deviations of real-world produce. Motivated by this finding, we propose SEED (Simultaneous Estimation of posE and Deformation), a unified RGB-only framework that jointly predicts 6D pose and explicit lattice deformations from a single image across multiple produce categories. Trained entirely on synthetic data with generative texture augmentation applied at the UV level, SEED outperforms MegaPose on 6 out of 8 categories under identical RGB-only conditions, demonstrating that explicit shape modeling is a critical step toward reliable pose estimation in agricultural robotics.

Related papers

• PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defects [36.94429692322632]
  PIXIE is a zero-shot framework that estimates the 6D pose of an object from an RGB image using only an untextured 3D model.<n>We evaluate on widely-used public benchmarks, reporting results on texture-less objects without object-specific training, and introduce a novel dataset to demonstrate real-world applicability.
  arXiv  Detail & Related papers  (2026-07-17T14:48:43Z)
• Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single View [69.6117755984012]
  Estimating an object's 6D pose, size, and shape from visual input is a fundamental problem in computer vision.<n>We propose a unified category-agnostic framework that simultaneously predicts 6D pose, size, and dense shape from a single RGB-D image.
  arXiv  Detail & Related papers  (2025-10-13T17:49:15Z)
• UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimation [19.76147681894604]
  UnPose is a framework for zero-shot, model-free 6D object pose estimation and reconstruction.<n>It exploits 3D priors and uncertainty estimates from a pre-trained diffusion model.<n>It significantly outperforms existing approaches in both 6D pose estimation accuracy and 3D reconstruction quality.
  arXiv  Detail & Related papers  (2025-08-21T21:31:04Z)
• Category-Level 6D Object Pose Estimation in Agricultural Settings Using a Lattice-Deformation Framework and Diffusion-Augmented Synthetic Data [22.68237431620023]
  We develop a novel framework for category 6D estimation that relies purely on RGB input.<n>We demonstrate the effectiveness of our framework on a challenging benchmark evaluation of various shapes, sizes, and ripeness status.
  arXiv  Detail & Related papers  (2025-05-30T14:25:52Z)
• Any6D: Model-free 6D Pose Estimation of Novel Objects [76.30057578269668]
  We introduce Any6D, a model-free framework for 6D object pose estimation.<n>It requires only a single RGB-D anchor image to estimate both the 6D pose and size of unknown objects in novel scenes.<n>We evaluate our method on five challenging datasets.
  arXiv  Detail & Related papers  (2025-03-24T13:46:21Z)
• Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation [66.3814684757376]
  This work presents Zero123-6D, the first work to demonstrate the utility of Diffusion Model-based novel-view-synthesizers in enhancing RGB 6D pose estimation at category-level. The outlined method shows reduction in data requirements, removal of the necessity of depth information in zero-shot category-level 6D pose estimation task, and increased performance, quantitatively demonstrated through experiments on the CO3D dataset.
  arXiv  Detail & Related papers  (2024-03-21T10:38:18Z)
• FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects [37.175069234979645]
  This work addresses the challenging task of 3D object recognition without the reliance on real-world 3D labeled data. Our goal is to predict the 3D shape, size, and 6D pose of objects within a single RGB-D image, operating at the category level and eliminating the need for CAD models during inference.
  arXiv  Detail & Related papers  (2023-10-19T17:59:09Z)
• Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
  This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos. We model hands as articulated objects inducing non-rigid face deformations during an active interaction. Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
  arXiv  Detail & Related papers  (2023-09-28T17:59:51Z)
• Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images [60.0898989456276]
  We present a new framework named Cas6D for few-shot 6DoF pose estimation that is generalizable and uses only RGB images. To address the false positives of target object detection in the extreme few-shot setting, our framework utilizes a self-supervised pre-trained ViT to learn robust feature representations. Experimental results on the LINEMOD and GenMOP datasets demonstrate that Cas6D outperforms state-of-the-art methods by 9.2% and 3.8% accuracy (Proj-5) under the 32-shot setting.
  arXiv  Detail & Related papers  (2023-06-13T07:45:42Z)
• SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation [98.83762558394345]
  SO-Pose is a framework for regressing all 6 degrees-of-freedom (6DoF) for the object pose in a cluttered environment from a single RGB image. We introduce a novel reasoning about self-occlusion, in order to establish a two-layer representation for 3D objects. Cross-layer consistencies that align correspondences, self-occlusion and 6D pose, we can further improve accuracy and robustness.
  arXiv  Detail & Related papers  (2021-08-18T19:49:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.