OpenMaterial: A Large-scale Dataset of Complex Materials for 3D Reconstruction

• URL: http://arxiv.org/abs/2406.08894v2
• Date: Sun, 02 Nov 2025 14:01:13 GMT
• Title: OpenMaterial: A Large-scale Dataset of Complex Materials for 3D Reconstruction
• Authors: Zheng Dang, Jialu Huang, Fei Wang, Mathieu Salzmann,
• Abstract summary: We introduce OpenMaterial, a large-scale semi-synthetic dataset for material-aware 3D reconstruction.<n>It comprises 1,001 objects spanning 295 distinct materials, including conductors, dielectrics, plastics, and their roughened variants, captured under 714 diverse lighting conditions.<n>It provides multi-view images, 3D shape models, camera poses, depth maps, and object masks, establishing the first extensive benchmark for evaluating 3D reconstruction on challenging materials.
• Score: 55.052637670485716
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in deep learning, such as neural radiance fields and implicit neural representations, have significantly advanced 3D reconstruction. However, accurately reconstructing objects with complex optical properties, such as metals, glass, and plastics, remains challenging due to the breakdown of multi-view color consistency in the presence of specular reflections, refractions, and transparency. This limitation is further exacerbated by the lack of benchmark datasets that explicitly model material-dependent light transport. To address this, we introduce OpenMaterial, a large-scale semi-synthetic dataset for benchmarking material-aware 3D reconstruction. It comprises 1,001 objects spanning 295 distinct materials, including conductors, dielectrics, plastics, and their roughened variants, captured under 714 diverse lighting conditions. By integrating lab-measured Index of Refraction (IOR) spectra, OpenMaterial enables the generation of high-fidelity multi-view images that accurately simulate complex light-matter interactions. It provides multi-view images, 3D shape models, camera poses, depth maps, and object masks, establishing the first extensive benchmark for evaluating 3D reconstruction on challenging materials. We evaluate 11 state-of-the-art methods for 3D reconstruction and novel view synthesis, conducting ablation studies to assess the impact of material type, shape complexity, and illumination on reconstruction performance. Our results indicate that OpenMaterial provides a strong and fair basis for developing more robust, physically-informed 3D reconstruction techniques to better handle real-world optical complexities.

Related papers

• LARM: A Large Articulated-Object Reconstruction Model [29.66486888001511]
  LARM is a unified feedforward framework that reconstructs 3D articulated objects from sparse-view images.<n>LARM generates auxiliary outputs such as depth maps and part masks to facilitate explicit 3D mesh extraction and joint estimation.<n>Our pipeline eliminates the need for dense supervision and supports high-fidelity reconstruction across diverse object categories.
  arXiv  Detail & Related papers  (2025-11-14T18:55:27Z)
• Large Material Gaussian Model for Relightable 3D Generation [54.10879517395551]
  We introduce a novel framework designed to generate high-quality 3D content with Physically Based Rendering (PBR) materials.<n>Our method not only exhibit greater visual appeal compared to baseline methods but also enhance material modeling, thereby enabling practical downstream rendering applications.
  arXiv  Detail & Related papers  (2025-09-26T09:35:12Z)
• Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey [171.72616707259306]
  3D reconstruction and view synthesis are foundational problems in computer vision, graphics, and immersive technologies such as augmented reality (AR), virtual reality (VR), and digital twins.<n>Recent advances in feed-forward approaches, driven by deep learning, have revolutionized this field by enabling fast and generalizable 3D reconstruction and view synthesis.
  arXiv  Detail & Related papers  (2025-07-19T06:13:25Z)
• MatDecompSDF: High-Fidelity 3D Shape and PBR Material Decomposition from Multi-View Images [20.219010684946888]
  MatDecompSDF is a framework for recovering high-fidelity 3D shapes and decomposing their physically-based material properties from multi-view images.<n>Our method produces editable and relightable assets that can be seamlessly integrated into standard graphics pipelines.
  arXiv  Detail & Related papers  (2025-07-07T08:22:32Z)
• Object-X: Learning to Reconstruct Multi-Modal 3D Object Representations [112.29763628638112]
  Object-X is a versatile multi-modal 3D representation framework.<n>It can encoding rich object embeddings and decoding them back into geometric and visual reconstructions.<n>It supports a range of downstream tasks, including scene alignment, single-image 3D object reconstruction, and localization.
  arXiv  Detail & Related papers  (2025-06-05T09:14:42Z)
• A Generative Approach to High Fidelity 3D Reconstruction from Text Data [0.0]
  This research proposes a fully automated pipeline that seamlessly integrates text-to-image generation, various image processing techniques, and deep learning methods for reflection removal and 3D reconstruction.<n>By leveraging state-of-the-art generative models like Stable Diffusion, the methodology translates natural language inputs into detailed 3D models through a multi-stage workflow.<n>This approach addresses key challenges in generative reconstruction, such as maintaining semantic coherence, managing geometric complexity, and preserving detailed visual information.
  arXiv  Detail & Related papers  (2025-03-05T16:54:15Z)
• ArtGS: Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting [66.29782808719301]
  Building articulated objects is a key challenge in computer vision. Existing methods often fail to effectively integrate information across different object states. We introduce ArtGS, a novel approach that leverages 3D Gaussians as a flexible and efficient representation.
  arXiv  Detail & Related papers  (2025-02-26T10:25:32Z)
• IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations [64.07859467542664]
  Capturing geometric and material information from images remains a fundamental challenge in computer vision and graphics. Traditional optimization-based methods often require hours of computational time to reconstruct geometry, material properties, and environmental lighting from dense multi-view inputs. We introduce IDArb, a diffusion-based model designed to perform intrinsic decomposition on an arbitrary number of images under varying illuminations.
  arXiv  Detail & Related papers  (2024-12-16T18:52:56Z)
• GlossyGS: Inverse Rendering of Glossy Objects with 3D Gaussian Splatting [21.23724172779984]
  GlossyGS aims to precisely reconstruct the geometry and materials of glossy objects by integrating material priors. We demonstrate through quantitative analysis and qualitative visualization that the proposed method is effective to reconstruct high-fidelity geometries and materials of glossy objects.
  arXiv  Detail & Related papers  (2024-10-17T09:00:29Z)
• AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction [55.69271635843385]
  We present AniSDF, a novel approach that learns fused-granularity neural surfaces with physics-based encoding for high-fidelity 3D reconstruction. Our method boosts the quality of SDF-based methods by a great scale in both geometry reconstruction and novel-view synthesis.
  arXiv  Detail & Related papers  (2024-10-02T03:10:38Z)
• Neural-PBIR Reconstruction of Shape, Material, and Illumination [26.628189591572074]
  We introduce an accurate and highly efficient object reconstruction pipeline combining neural based object reconstruction and physics-based inverse rendering (PBIR) Our pipeline firstly leverages a neural SDF based shape reconstruction to produce high-quality but potentially imperfect object shape. In the last stage, by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction of object shape, material, and illumination.
  arXiv  Detail & Related papers  (2023-04-26T11:02:04Z)
• Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes [62.769186261245416]
  We present a novel inverse rendering framework for large urban scenes capable of jointly reconstructing the scene geometry, spatially-varying materials, and HDR lighting from a set of posed RGB images with optional depth. Specifically, we use a neural field to account for the primary rays, and use an explicit mesh (reconstructed from the underlying neural field) for modeling secondary rays that produce higher-order lighting effects such as cast shadows.
  arXiv  Detail & Related papers  (2023-04-06T17:51:54Z)
• Unsupervised Learning of 3D Object Categories from Videos in the Wild [75.09720013151247]
  We focus on learning a model from multiple views of a large collection of object instances. We propose a new neural network design, called warp-conditioned ray embedding (WCR), which significantly improves reconstruction. Our evaluation demonstrates performance improvements over several deep monocular reconstruction baselines on existing benchmarks.
  arXiv  Detail & Related papers  (2021-03-30T17:57:01Z)
• Shape From Tracing: Towards Reconstructing 3D Object Geometry and SVBRDF Material from Images via Differentiable Path Tracing [16.975014467319443]
  Differentiable path tracing is an appealing framework as it can reproduce complex appearance effects. We show how to use differentiable ray tracing to refine an initial coarse mesh and per-mesh-facet material representation. We also show how to refine initial reconstructions of real-world objects in unconstrained environments.
  arXiv  Detail & Related papers  (2020-12-06T18:55:35Z)
• 3DMaterialGAN: Learning 3D Shape Representation from Latent Space for Materials Science Applications [7.449993399792031]
  3DMaterialGAN is capable of recognizing and synthesizing individual grains whose morphology conforms to a given 3D polycrystalline material microstructure. We show that this method performs comparably or better than state-of-the-art on benchmark annotated 3D datasets. This framework lays the foundation for the recognition and synthesis of polycrystalline material microstructures.
  arXiv  Detail & Related papers  (2020-07-27T21:55:16Z)
• Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes [75.63464905190061]
  Complex light paths induced by refraction and reflection have prevented both traditional and deep multiview stereo from solving this problem. We propose a physically-based network to recover 3D shape of transparent objects using a few images acquired with a mobile phone camera. Our experiments show successful recovery of high-quality 3D geometry for complex transparent shapes using as few as 5-12 natural images.
  arXiv  Detail & Related papers  (2020-04-22T23:51:30Z)
• Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance [46.488713939892136]
  We introduce a neural network that simultaneously learns the unknown geometry, camera parameters, and a neural architecture that approximates the light reflected from the surface towards the camera. We trained our network on real world 2D images of objects with different material properties, lighting conditions, and noisy camera materials from the DTU MVS dataset.
  arXiv  Detail & Related papers  (2020-03-22T10:20:13Z)
• Seeing the World in a Bag of Chips [73.561388215585]
  We address the dual problems of novel view synthesis and environment reconstruction from hand-held RGBD sensors. Our contributions include 1) modeling highly specular objects, 2) modeling inter-reflections and Fresnel effects, and 3) enabling surface light field reconstruction with the same input needed to reconstruct shape alone.
  arXiv  Detail & Related papers  (2020-01-14T06:44:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.