ArcPro: Architectural Programs for Structured 3D Abstraction of Sparse Points

• URL: http://arxiv.org/abs/2503.02745v2
• Date: Wed, 05 Mar 2025 04:49:18 GMT
• Title: ArcPro: Architectural Programs for Structured 3D Abstraction of Sparse Points
• Authors: Qirui Huang, Runze Zhang, Kangjun Liu, Minglun Gong, Hao Zhang, Hui Huang,
• Abstract summary: ArcPro is a learning framework built on architectural programs to recover structured 3D abstractions from point clouds.<n>We train an encoder-decoder on the points-program pairs to establish a mapping from unstructured point clouds to architectural programs.<n>Inference by our method is highly efficient and produces plausible and faithful 3D abstractions.
• Score: 21.052495894521872
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce ArcPro, a novel learning framework built on architectural programs to recover structured 3D abstractions from highly sparse and low-quality point clouds. Specifically, we design a domain-specific language (DSL) to hierarchically represent building structures as a program, which can be efficiently converted into a mesh. We bridge feedforward and inverse procedural modeling by using a feedforward process for training data synthesis, allowing the network to make reverse predictions. We train an encoder-decoder on the points-program pairs to establish a mapping from unstructured point clouds to architectural programs, where a 3D convolutional encoder extracts point cloud features and a transformer decoder autoregressively predicts the programs in a tokenized form. Inference by our method is highly efficient and produces plausible and faithful 3D abstractions. Comprehensive experiments demonstrate that ArcPro outperforms both traditional architectural proxy reconstruction and learning-based abstraction methods. We further explore its potential to work with multi-view image and natural language inputs.

Related papers

• FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views [93.6881532277553]
  We present FLARE, a feed-forward model designed to infer high-quality camera poses and 3D geometry from uncalibrated sparse-view images.<n>Our solution features a cascaded learning paradigm with camera pose serving as the critical bridge, recognizing its essential role in mapping 3D structures onto 2D image planes.
  arXiv  Detail & Related papers  (2025-02-17T18:54:05Z)
• Synthesizing 3D Abstractions by Inverting Procedural Buildings with Transformers [2.199128905898291]
  We generate abstractions of buildings by learning to invert procedural models.<n>Our approach achieves good reconstruction accuracy in terms of geometry and structure, as well as structurally consistent inpainting.
  arXiv  Detail & Related papers  (2025-01-28T16:09:34Z)
• Large Spatial Model: End-to-end Unposed Images to Semantic 3D [79.94479633598102]
  Large Spatial Model (LSM) processes unposed RGB images directly into semantic radiance fields. LSM simultaneously estimates geometry, appearance, and semantics in a single feed-forward operation. It can generate versatile label maps by interacting with language at novel viewpoints.
  arXiv  Detail & Related papers  (2024-10-24T17:54:42Z)
• PyTorchGeoNodes: Enabling Differentiable Shape Programs for 3D Shape Reconstruction [27.98261111833689]
  We propose PyTorchGeoNodes, a differentiable module for reconstructing 3D objects from images using interpretable shape programs. In our experiments, we apply our algorithm to reconstruct 3D objects in the ScanNet dataset and evaluate our results against CAD model retrieval-based reconstructions.
  arXiv  Detail & Related papers  (2024-04-16T14:43:33Z)
• Human as Points: Explicit Point-based 3D Human Reconstruction from Single-view RGB Images [71.91424164693422]
  We introduce an explicit point-based human reconstruction framework called HaP. Our approach is featured by fully-explicit point cloud estimation, manipulation, generation, and refinement in the 3D geometric space. Our results may indicate a paradigm rollback to the fully-explicit and geometry-centric algorithm design.
  arXiv  Detail & Related papers  (2023-11-06T05:52:29Z)
• ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapes [32.267066838654834]
  We present a novel implicit representation to efficiently and accurately encode large datasets of complex 3D shapes. Our implicit Recursive Octree Auto-Decoder (ROAD) learns a hierarchically structured latent space enabling state-of-the-art reconstruction results at a compression ratio above 99%.
  arXiv  Detail & Related papers  (2022-12-12T19:09:47Z)
• Learning Local Displacements for Point Cloud Completion [93.54286830844134]
  We propose a novel approach aimed at object and semantic scene completion from a partial scan represented as a 3D point cloud. Our architecture relies on three novel layers that are used successively within an encoder-decoder structure. We evaluate both architectures on object and indoor scene completion tasks, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-30T18:31:37Z)
• Translational Symmetry-Aware Facade Parsing for 3D Building Reconstruction [11.263458202880038]
  In this paper, we present a novel translational symmetry-based approach to improving the deep neural networks. We propose a novel scheme to fuse anchor-free detection in a single stage network, which enables the efficient training and better convergence. We employ an off-the-shelf rendering engine like Blender to reconstruct the realistic high-quality 3D models using procedural modeling.
  arXiv  Detail & Related papers  (2021-06-02T03:10:51Z)
• Convolutional Occupancy Networks [88.48287716452002]
  We propose Convolutional Occupancy Networks, a more flexible implicit representation for detailed reconstruction of objects and 3D scenes. By combining convolutional encoders with implicit occupancy decoders, our model incorporates inductive biases, enabling structured reasoning in 3D space. We empirically find that our method enables the fine-grained implicit 3D reconstruction of single objects, scales to large indoor scenes, and generalizes well from synthetic to real data.
  arXiv  Detail & Related papers  (2020-03-10T10:17:07Z)
• Unsupervised Learning of Intrinsic Structural Representation Points [50.92621061405056]
  Learning structures of 3D shapes is a fundamental problem in the field of computer graphics and geometry processing. We present a simple yet interpretable unsupervised method for learning a new structural representation in the form of 3D structure points.
  arXiv  Detail & Related papers  (2020-03-03T17:40:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.