Harnessing Generative Image Models for Training-Free Primitive Shape Abstraction

• URL: http://arxiv.org/abs/2607.05568v1
• Date: Mon, 06 Jul 2026 19:10:43 GMT
• Title: Harnessing Generative Image Models for Training-Free Primitive Shape Abstraction
• Abstract summary: Generative image models trained at scale can identify and segment object parts directly in the image domain.<n>Our pipeline renders multi-view images of a 3D object, uses a vision-language model to analyze its semantic parts, and fits a superquadric primitive to each part.<n>Our method achieves the lowest Chamfer distance among all evaluated methods, using 5--9 primitives per object on average.
• Score: 9.774685205718065
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Representing 3D shapes as compact sets of geometric primitives is fundamental to robotics, simulation, and scene understanding. Generative image models trained at scale have recently emerged as generalist visual learners that can identify and segment object parts directly in the image domain, across arbitrary categories and without task-specific training. Adapting such models to downstream tasks typically requires fine-tuning; we ask whether their pretrained capability can instead be harnessed directly, without any training, and answer affirmatively with a training-free harness. Our pipeline renders multi-view images of a 3D object, uses a vision-language model to analyze its semantic parts, prompts a generative image model to paint a color-coded part segmentation mask, reprojects it onto the geometry, and fits a superquadric primitive to each part via parameter optimization. The approach contains no learned parameters: it is category-agnostic and orientation-invariant, properties that previous learning-based models struggled with. Its accuracy ceiling rises with future generative-model improvements, which we confirm with a ground-truth segmentation study showing that part segmentation, not primitive fitting, is the current accuracy bottleneck. On HumanPrim and Toys4K, our method achieves the lowest Chamfer distance among all evaluated methods, using 5--9 primitives per object on average.

Related papers

• Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space [58.623106094568776]
  3D morphable models (3DMMs) are a powerful tool to represent the possible shapes and appearances of an object category.<n>We introduce a new method, Common3D, that learns 3DMMs of common objects in a fully self-supervised manner from a collection of object-centric videos.<n>Common3D is the first completely self-supervised method that can solve various vision tasks in a zero-shot manner.
  arXiv  Detail & Related papers  (2025-04-30T15:42:23Z)
• GS-Pose: Category-Level Object Pose Estimation via Geometric and Semantic Correspondence [5.500735640045456]
  Category-level pose estimation is a challenging task with many potential applications in computer vision and robotics. We propose to utilize both geometric and semantic features obtained from a pre-trained foundation model. This requires significantly less data to train than prior methods since the semantic features are robust to object texture and appearance.
  arXiv  Detail & Related papers  (2023-11-23T02:35:38Z)
• MagicPony: Learning Articulated 3D Animals in the Wild [81.63322697335228]
  We present a new method, dubbed MagicPony, that learns this predictor purely from in-the-wild single-view images of the object category. At its core is an implicit-explicit representation of articulated shape and appearance, combining the strengths of neural fields and meshes.
  arXiv  Detail & Related papers  (2022-11-22T18:59:31Z)
• Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images [82.32776379815712]
  We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. We adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable function for test-time optimization.
  arXiv  Detail & Related papers  (2022-04-21T03:42:31Z)
• Semantic keypoint-based pose estimation from single RGB frames [64.80395521735463]
  We present an approach to estimating the continuous 6-DoF pose of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. We show that our approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios.
  arXiv  Detail & Related papers  (2022-04-12T15:03:51Z)
• Multi-Category Mesh Reconstruction From Image Collections [90.24365811344987]
  We present an alternative approach that infers the textured mesh of objects combining a series of deformable 3D models and a set of instance-specific deformation, pose, and texture. Our method is trained with images of multiple object categories using only foreground masks and rough camera poses as supervision. Experiments show that the proposed framework can distinguish between different object categories and learn category-specific shape priors in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-21T16:32:31Z)
• Learning Generative Models of Textured 3D Meshes from Real-World Images [26.353307246909417]
  We propose a GAN framework for generating textured triangle meshes without relying on such annotations. We show that the performance of our approach is on par with prior work that relies on ground-truth keypoints.
  arXiv  Detail & Related papers  (2021-03-29T14:07:37Z)
• Weakly Supervised Learning of Multi-Object 3D Scene Decompositions Using Deep Shape Priors [69.02332607843569]
  PriSMONet is a novel approach for learning Multi-Object 3D scene decomposition and representations from single images. A recurrent encoder regresses a latent representation of 3D shape, pose and texture of each object from an input RGB image. We evaluate the accuracy of our model in inferring 3D scene layout, demonstrate its generative capabilities, assess its generalization to real images, and point out benefits of the learned representation.
  arXiv  Detail & Related papers  (2020-10-08T14:49:23Z)
• Monocular Human Pose and Shape Reconstruction using Part Differentiable Rendering [53.16864661460889]
  Recent works succeed in regression-based methods which estimate parametric models directly through a deep neural network supervised by 3D ground truth. In this paper, we introduce body segmentation as critical supervision. To improve the reconstruction with part segmentation, we propose a part-level differentiable part that enables part-based models to be supervised by part segmentation.
  arXiv  Detail & Related papers  (2020-03-24T14:25:46Z)
• PolyGen: An Autoregressive Generative Model of 3D Meshes [22.860421649320287]
  We present an approach which models the mesh directly using a Transformer-based architecture. Our model can condition on a range of inputs, including object classes, voxels, and images. We show that the model is capable of producing high-quality, usable meshes, and establish log-likelihood benchmarks for the mesh-modelling task.
  arXiv  Detail & Related papers  (2020-02-23T17:16:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.