Automatic Reverse Engineering: Creating computer-aided design (CAD) models from multi-view images

• URL: http://arxiv.org/abs/2309.13281v1
• Date: Sat, 23 Sep 2023 06:42:09 GMT
• Title: Automatic Reverse Engineering: Creating computer-aided design (CAD) models from multi-view images
• Authors: Henrik Jobczyk and Hanno Homann
• Abstract summary: We present a novel network for an automated reverse engineering task. A proof-of-concept is demonstrated by successfully reconstructing a number of valid CAD models. It is shown that some of the capabilities of our network can be transferred to this domain.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generation of computer-aided design (CAD) models from multi-view images may be useful in many practical applications. To date, this problem is usually solved with an intermediate point-cloud reconstruction and involves manual work to create the final CAD models. In this contribution, we present a novel network for an automated reverse engineering task. Our network architecture combines three distinct stages: A convolutional neural network as the encoder stage, a multi-view pooling stage and a transformer-based CAD sequence generator. The model is trained and evaluated on a large number of simulated input images and extensive optimization of model architectures and hyper-parameters is performed. A proof-of-concept is demonstrated by successfully reconstructing a number of valid CAD models from simulated test image data. Various accuracy metrics are calculated and compared to a state-of-the-art point-based network. Finally, a real world test is conducted supplying the network with actual photographs of two three-dimensional test objects. It is shown that some of the capabilities of our network can be transferred to this domain, even though the training exclusively incorporates purely synthetic training data. However to date, the feasible model complexity is still limited to basic shapes.

Related papers

• Large Spatial Model: End-to-end Unposed Images to Semantic 3D [79.94479633598102]
  Large Spatial Model (LSM) processes unposed RGB images directly into semantic radiance fields. LSM simultaneously estimates geometry, appearance, and semantics in a single feed-forward operation. It can generate versatile label maps by interacting with language at novel viewpoints.
  arXiv  Detail & Related papers  (2024-10-24T17:54:42Z)
• Img2CAD: Reverse Engineering 3D CAD Models from Images through VLM-Assisted Conditional Factorization [12.12975824816803]
  Reverse engineering 3D computer-aided design (CAD) models from images is an important task for many downstream applications. In this work, we introduce a novel approach that conditionally factorizes the task into two sub-problems. We propose TrAssembler that conditioned on the discrete structure with semantics predicts the continuous attribute values.
  arXiv  Detail & Related papers  (2024-07-19T06:53:30Z)
• LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image [64.94932577552458]
  Large Reconstruction Models have made significant strides in the realm of automated 3D content generation from single or multiple input images. Despite their success, these models often produce 3D meshes with geometric inaccuracies, stemming from the inherent challenges of deducing 3D shapes solely from image data. We introduce a novel framework, the Large Image and Point Cloud Alignment Model (LAM3D), which utilizes 3D point cloud data to enhance the fidelity of generated 3D meshes.
  arXiv  Detail & Related papers  (2024-05-24T15:09:12Z)
• Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks [53.67497327319569]
  We introduce a novel neural rendering technique to solve image-to-3D from a single view. Our approach employs the signed distance function as the surface representation and incorporates generalizable priors through geometry-encoding volumes and HyperNetworks. Our experiments show the advantages of our proposed approach with consistent results and rapid generation.
  arXiv  Detail & Related papers  (2023-12-24T08:42:37Z)
• DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image [34.47379913018661]
  We propose DiffCAD, the first weakly-supervised probabilistic approach to CAD retrieval and alignment from an RGB image. We formulate this as a conditional generative task, leveraging diffusion to learn implicit probabilistic models capturing the shape, pose, and scale of CAD objects in an image. Our approach is trained only on synthetic data, leveraging monocular depth and mask estimates to enable robust zero-shot adaptation to various real target domains.
  arXiv  Detail & Related papers  (2023-11-30T15:10:21Z)
• Sparse Multi-Object Render-and-Compare [33.97243145891282]
  Reconstructing 3D shape and pose of static objects from a single image is an essential task for various industries. Directly predicting 3D shapes produces unrealistic, overly smoothed or tessellated shapes. Retrieving CAD models ensures realistic shapes but requires robust and accurate alignment.
  arXiv  Detail & Related papers  (2023-10-17T12:01:32Z)
• Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images [82.32776379815712]
  We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. We adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable function for test-time optimization.
  arXiv  Detail & Related papers  (2022-04-21T03:42:31Z)
• Meta Internal Learning [88.68276505511922]
  Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image. We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
  arXiv  Detail & Related papers  (2021-10-06T16:27:38Z)
• CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assembly [17.82598676258891]
  We introduce CAPRI-Net, a neural network for learning compact and interpretable implicit representations of 3D computer-aided design (CAD) models. Our network takes an input 3D shape that can be provided as a point cloud or voxel grids, and reconstructs it by a compact assembly of quadric surface primitives. We evaluate our learning framework on both ShapeNet and ABC, the largest and most diverse CAD dataset to date, in terms of reconstruction quality, shape edges, compactness, and interpretability.
  arXiv  Detail & Related papers  (2021-04-12T17:21:19Z)
• CAD2Real: Deep learning with domain randomization of CAD data for 3D pose estimation of electronic control unit housings [0.20999222360659608]
  We train state-of-the-art artificial neural networks (ANNs) on purely synthetic training data. By randomizing parameters during rendering of training images, we enable inference on RGB images of a real sample part.
  arXiv  Detail & Related papers  (2020-09-25T16:08:16Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.