IST-Net: Prior-free Category-level Pose Estimation with Implicit Space Transformation

• URL: http://arxiv.org/abs/2303.13479v2
• Date: Wed, 19 Jul 2023 16:11:13 GMT
• Title: IST-Net: Prior-free Category-level Pose Estimation with Implicit Space Transformation
• Abstract summary: Category-level 6D pose estimation aims to predict the poses and sizes of unseen objects from a specific category. Thanks to prior deformation, which explicitly adapts a category-specific 3D prior to a given object instance, prior-based methods attained great success. However, obtaining category-specific priors requires collecting a large amount of 3D models, which is labor-consuming and often not accessible in practice. We introduce a simple prior-free implicit space transformation network, namely IST-Net, to transform camera-space features to world-space counterparts.
• Score: 39.6433705759858
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Category-level 6D pose estimation aims to predict the poses and sizes of unseen objects from a specific category. Thanks to prior deformation, which explicitly adapts a category-specific 3D prior (i.e., a 3D template) to a given object instance, prior-based methods attained great success and have become a major research stream. However, obtaining category-specific priors requires collecting a large amount of 3D models, which is labor-consuming and often not accessible in practice. This motivates us to investigate whether priors are necessary to make prior-based methods effective. Our empirical study shows that the 3D prior itself is not the credit to the high performance. The keypoint actually is the explicit deformation process, which aligns camera and world coordinates supervised by world-space 3D models (also called canonical space). Inspired by these observations, we introduce a simple prior-free implicit space transformation network, namely IST-Net, to transform camera-space features to world-space counterparts and build correspondence between them in an implicit manner without relying on 3D priors. Besides, we design camera- and world-space enhancers to enrich the features with pose-sensitive information and geometrical constraints, respectively. Albeit simple, IST-Net achieves state-of-the-art performance based-on prior-free design, with top inference speed on the REAL275 benchmark. Our code and models are available at https://github.com/CVMI-Lab/IST-Net.

Related papers

• UniPose9D: Universal Category-Agnostic Object Pose Estimation [10.964744555206243]
  UniPose9D is a category-agnostic foundation model for 9D object pose estimation.<n>It estimates rotation, translation, and metric size without category labels, CAD models, mean-shape priors, or reference views.<n>To improve accuracy, we introduce a point-pair-based RANSAC N-hop Kabsch--Umeyama algorithm with an adaptive threshold.
  arXiv  Detail & Related papers  (2026-07-10T21:24:23Z)
• Category-Level 3D Correspondence in Camera Space via Morphable Object Priors [25.509243380578692]
  Understanding 3D objects from images is fundamental to robotics and AR/VR applications.<n>In this work, we study category-level 3D correspondence in camera space.<n>We show that it can emerge without explicit correspondence supervision by learning a shared morphable object prior.
  arXiv  Detail & Related papers  (2026-05-27T10:08:32Z)
• Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single View [69.6117755984012]
  Estimating an object's 6D pose, size, and shape from visual input is a fundamental problem in computer vision.<n>We propose a unified category-agnostic framework that simultaneously predicts 6D pose, size, and dense shape from a single RGB-D image.
  arXiv  Detail & Related papers  (2025-10-13T17:49:15Z)
• TrackAny3D: Transferring Pretrained 3D Models for Category-unified 3D Point Cloud Tracking [25.788917457593673]
  TrackAny3D is the first framework to transfer large-scale pretrained 3D models for category-agnostic 3D SOT.<n>MoGE architecture adaptively activates specialized 3works based on distinct geometric characteristics.<n>Experiments show that TrackAny3D establishes new state-of-the-art performance on category-agnostic 3D SOT.
  arXiv  Detail & Related papers  (2025-07-26T10:41:55Z)
• E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models [78.1674905950243]
  We present the first comprehensive benchmark for 3D geometric foundation models (GFMs)<n>GFMs directly predict dense 3D representations in a single feed-forward pass, eliminating the need for slow or unavailable precomputed camera parameters.<n>We evaluate 16 state-of-the-art GFMs, revealing their strengths and limitations across tasks and domains.<n>All code, evaluation scripts, and processed data will be publicly released to accelerate research in 3D spatial intelligence.
  arXiv  Detail & Related papers  (2025-06-02T17:53:09Z)
• General Geometry-aware Weakly Supervised 3D Object Detection [62.26729317523975]
  A unified framework is developed for learning 3D object detectors from RGB images and associated 2D boxes. Experiments on KITTI and SUN-RGBD datasets demonstrate that our method yields surprisingly high-quality 3D bounding boxes with only 2D annotation.
  arXiv  Detail & Related papers  (2024-07-18T17:52:08Z)
• Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videos [15.532504015622159]
  Category-level 3D pose estimation is a fundamentally important problem in computer vision and robotics. We tackle the problem of learning to estimate the category-level 3D pose only from casually taken object-centric videos.
  arXiv  Detail & Related papers  (2024-07-05T09:43:05Z)
• SAI3D: Segment Any Instance in 3D Scenes [68.57002591841034]
  We introduce SAI3D, a novel zero-shot 3D instance segmentation approach. Our method partitions a 3D scene into geometric primitives, which are then progressively merged into 3D instance segmentations. Empirical evaluations on ScanNet, Matterport3D and the more challenging ScanNet++ datasets demonstrate the superiority of our approach.
  arXiv  Detail & Related papers  (2023-12-17T09:05:47Z)
• Open-Pose 3D Zero-Shot Learning: Benchmark and Challenges [23.663199578392447]
  We propose a more realistic and challenging scenario named open-pose 3D zero-shot classification. First, we revisit the current research on 3D zero-shot classification. We propose two benchmark datasets specifically designed for the open-pose setting.
  arXiv  Detail & Related papers  (2023-12-12T07:52:33Z)
• ShapeShift: Superquadric-based Object Pose Estimation for Robotic Grasping [85.38689479346276]
  Current techniques heavily rely on a reference 3D object, limiting their generalizability and making it expensive to expand to new object categories. This paper proposes ShapeShift, a superquadric-based framework for object pose estimation that predicts the object's pose relative to a primitive shape which is fitted to the object.
  arXiv  Detail & Related papers  (2023-04-10T20:55:41Z)
• NOPE: Novel Object Pose Estimation from a Single Image [67.11073133072527]
  We propose an approach that takes a single image of a new object as input and predicts the relative pose of this object in new images without prior knowledge of the object's 3D model. We achieve this by training a model to directly predict discriminative embeddings for viewpoints surrounding the object. This prediction is done using a simple U-Net architecture with attention and conditioned on the desired pose, which yields extremely fast inference.
  arXiv  Detail & Related papers  (2023-03-23T18:55:43Z)
• Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild [47.80637472803838]
  We introduce a self-supervised learning approach trained directly on large-scale real-world object videos for category-level 6D pose estimation in the wild. Our framework reconstructs the canonical 3D shape of an object category and learns dense correspondences between input images and the canonical shape via surface embedding. Surprisingly, our method, without any human annotations or simulators, can achieve on-par or even better performance than previous supervised or semi-supervised methods on in-the-wild images.
  arXiv  Detail & Related papers  (2022-10-13T17:19:22Z)
• Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints [24.232254155643574]
  We consider a problem where one is given 2D or 3D sensor data picturing an object of a given category (e.g., a car) and has to reconstruct the 3D pose and shape of the object. Our first contribution is to develop PACE3D* and PACE2D*, the first certifiably optimal solvers for pose and shape estimation. Our second contribution is to developrobust versions of both solvers, named PACE3D# and PACE2D#.
  arXiv  Detail & Related papers  (2022-06-24T21:58:00Z)
• BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models [1.14219428942199]
  This work proposes BundleTrack, a general framework for 6D pose tracking of objects. An efficient implementation of the framework provides a real-time performance of 10Hz for the entire framework.
  arXiv  Detail & Related papers  (2021-08-01T18:14:46Z)
• Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation [62.618227434286]
  We present a novel learning approach to recover the 6D poses and sizes of unseen object instances from an RGB-D image. We propose a deep network to reconstruct the 3D object model by explicitly modeling the deformation from a pre-learned categorical shape prior.
  arXiv  Detail & Related papers  (2020-07-16T16:45:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.