Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single View

• URL: http://arxiv.org/abs/2510.11687v1
• Date: Mon, 13 Oct 2025 17:49:15 GMT
• Title: Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single View
• Authors: Jinyu Zhang, Haitao Lin, Jiashu Hou, Xiangyang Xue, Yanwei Fu,
• Abstract summary: Estimating an object's 6D pose, size, and shape from visual input is a fundamental problem in computer vision.<n>We propose a unified category-agnostic framework that simultaneously predicts 6D pose, size, and dense shape from a single RGB-D image.
• Score: 69.6117755984012
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating an object's 6D pose, size, and shape from visual input is a fundamental problem in computer vision, with critical applications in robotic grasping and manipulation. Existing methods either rely on object-specific priors such as CAD models or templates, or suffer from limited generalization across categories due to pose-shape entanglement and multi-stage pipelines. In this work, we propose a unified, category-agnostic framework that simultaneously predicts 6D pose, size, and dense shape from a single RGB-D image, without requiring templates, CAD models, or category labels at test time. Our model fuses dense 2D features from vision foundation models with partial 3D point clouds using a Transformer encoder enhanced by a Mixture-of-Experts, and employs parallel decoders for pose-size estimation and shape reconstruction, achieving real-time inference at 28 FPS. Trained solely on synthetic data from 149 categories in the SOPE dataset, our framework is evaluated on four diverse benchmarks SOPE, ROPE, ObjaversePose, and HANDAL, spanning over 300 categories. It achieves state-of-the-art accuracy on seen categories while demonstrating remarkably strong zero-shot generalization to unseen real-world objects, establishing a new standard for open-set 6D understanding in robotics and embodied AI.

Related papers

• OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding [2.1987601456703474]
  We introduce a unified, end-to-end framework that seamlessly integrates object detection and pose estimation.<n>Our system first employs the CNOS detector to localize target objects.<n>For each detection, our novel pose estimation module, OPFormer, infers the precise 6D pose.
  arXiv  Detail & Related papers  (2025-11-16T14:19:52Z)
• One2Any: One-Reference 6D Pose Estimation for Any Object [98.50085481362808]
  6D object pose estimation remains challenging for many applications due to dependencies on complete 3D models, multi-view images, or training limited to specific object categories.<n>We propose a novel method One2Any that estimates the relative 6-degrees of freedom (DOF) object pose using only a single reference-single query RGB-D image.<n> Experiments on multiple benchmark datasets demonstrate that our model generalizes well to novel objects, achieving state-of-the-art accuracy and even rivaling methods that require multi-view or CAD inputs, at a fraction of compute.
  arXiv  Detail & Related papers  (2025-05-07T03:54:59Z)
• Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videos [15.532504015622159]
  Category-level 3D pose estimation is a fundamentally important problem in computer vision and robotics. We tackle the problem of learning to estimate the category-level 3D pose only from casually taken object-centric videos.
  arXiv  Detail & Related papers  (2024-07-05T09:43:05Z)
• GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence [64.77224422330737]
  GigaPose is a fast, robust, and accurate method for CAD-based novel object pose estimation in RGB images. Our approach samples templates in only a two-degrees-of-freedom space instead of the usual three. It achieves state-of-the-art accuracy and can be seamlessly integrated with existing refinement methods.
  arXiv  Detail & Related papers  (2023-11-23T18:55:03Z)
• MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare [84.80956484848505]
  MegaPose is a method to estimate the 6D pose of novel objects, that is, objects unseen during training. We present a 6D pose refiner based on a render&compare strategy which can be applied to novel objects. Second, we introduce a novel approach for coarse pose estimation which leverages a network trained to classify whether the pose error between a synthetic rendering and an observed image of the same object can be corrected by the refiner.
  arXiv  Detail & Related papers  (2022-12-13T19:30:03Z)
• Single-stage Keypoint-based Category-level Object Pose Estimation from an RGB Image [27.234658117816103]
  We propose a single-stage, keypoint-based approach for category-level object pose estimation. The proposed network performs 2D object detection, detects 2D keypoints, estimates 6-DoF pose, and regresses relative bounding cuboid dimensions. We conduct extensive experiments on the challenging Objectron benchmark, outperforming state-of-the-art methods on the 3D IoU metric.
  arXiv  Detail & Related papers  (2021-09-13T17:55:00Z)
• DONet: Learning Category-Level 6D Object Pose and Size Estimation from Depth Observation [53.55300278592281]
  We propose a method of Category-level 6D Object Pose and Size Estimation (COPSE) from a single depth image. Our framework makes inferences based on the rich geometric information of the object in the depth channel alone. Our framework competes with state-of-the-art approaches that require labeled real-world images.
  arXiv  Detail & Related papers  (2021-06-27T10:41:50Z)
• Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation [62.618227434286]
  We present a novel learning approach to recover the 6D poses and sizes of unseen object instances from an RGB-D image. We propose a deep network to reconstruct the 3D object model by explicitly modeling the deformation from a pre-learned categorical shape prior.
  arXiv  Detail & Related papers  (2020-07-16T16:45:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.