Compositionally Generalizable 3D Structure Prediction

• URL: http://arxiv.org/abs/2012.02493v3
• Date: Thu, 22 Apr 2021 02:15:38 GMT
• Title: Compositionally Generalizable 3D Structure Prediction
• Authors: Songfang Han, Jiayuan Gu, Kaichun Mo, Li Yi, Siyu Hu, Xuejin Chen, Hao Su
• Abstract summary: Single-image 3D shape reconstruction is an important and long-standing problem in computer vision. We propose a novel framework that could better generalize to unseen object categories. Experiments on PartNet show that we achieve superior performance than state-of-the-art.
• Score: 41.641683644620464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Single-image 3D shape reconstruction is an important and long-standing problem in computer vision. A plethora of existing works is constantly pushing the state-of-the-art performance in the deep learning era. However, there remains a much more difficult and under-explored issue on how to generalize the learned skills over unseen object categories that have very different shape geometry distributions. In this paper, we bring in the concept of compositional generalizability and propose a novel framework that could better generalize to these unseen categories. We factorize the 3D shape reconstruction problem into proper sub-problems, each of which is tackled by a carefully designed neural sub-module with generalizability concerns. The intuition behind our formulation is that object parts (slates and cylindrical parts), their relationships (adjacency and translation symmetry), and shape substructures (T-junctions and a symmetric group of parts) are mostly shared across object categories, even though object geometries may look very different (e.g. chairs and cabinets). Experiments on PartNet show that we achieve superior performance than state-of-the-art. This validates our problem factorization and network designs.

Related papers

• StructRe: Rewriting for Structured Shape Modeling [63.792684115318906]
  We present StructRe, a structure rewriting system, as a novel approach to structured shape modeling. Given a 3D object represented by points and components, StructRe can rewrite it upward into more concise structures, or downward into more detailed structures.
  arXiv  Detail & Related papers  (2023-11-29T10:35:00Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape Generation [98.96086261213578]
  We introduce DSG-Net, a deep neural network that learns a disentangled structured and geometric mesh representation for 3D shapes. This supports a range of novel shape generation applications with disentangled control, such as of structure (geometry) while keeping geometry (structure) unchanged. Our method not only supports controllable generation applications but also produces high-quality synthesized shapes, outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-12T17:06:51Z)
• A Study of Compositional Generalization in Neural Models [22.66002315559978]
  We introduce ConceptWorld, which enables the generation of images from compositional and relational concepts. We perform experiments to test the ability of standard neural networks to generalize on relations with compositional arguments. For simple problems, all models generalize well to close concepts but struggle with longer compositional chains.
  arXiv  Detail & Related papers  (2020-06-16T18:29:58Z)
• Fine-Grained 3D Shape Classification with Hierarchical Part-View Attentions [70.0171362989609]
  We propose a novel fine-grained 3D shape classification method named FG3D-Net to capture the fine-grained local details of 3D shapes from multiple rendered views. Our results under the fine-grained 3D shape dataset show that our method outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-05-26T06:53:19Z)
• Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image [102.44347847154867]
  We propose a novel formulation that allows to jointly recover the geometry of a 3D object as a set of primitives. Our model recovers the higher level structural decomposition of various objects in the form of a binary tree of primitives. Our experiments on the ShapeNet and D-FAUST datasets demonstrate that considering the organization of parts indeed facilitates reasoning about 3D geometry.
  arXiv  Detail & Related papers  (2020-04-02T17:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.