A study of the effect of the illumination model on the generation of synthetic training datasets

• URL: http://arxiv.org/abs/2006.08819v1
• Date: Mon, 15 Jun 2020 23:22:24 GMT
• Title: A study of the effect of the illumination model on the generation of synthetic training datasets
• Abstract summary: We study how the illumination model used by the rendering software affects the quality of the generated images. We created eight training sets, each one with a different illumination model, and tested them on three different network architectures.
• Score: 4.238220229373858
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The use of computer generated images to train Deep Neural Networks is a viable alternative to real images when the latter are scarce or expensive. In this paper, we study how the illumination model used by the rendering software affects the quality of the generated images. We created eight training sets, each one with a different illumination model, and tested them on three different network architectures, ResNet, U-Net and a combined architecture developed by us. The test set consisted of photos of 3D printed objects produced from the same CAD models used to generate the training set. The effect of the other parameters of the rendering process, such as textures and camera position, was randomized. Our results show that the effect of the illumination model is important, comparable in significance to the network architecture. We also show that both light probes capturing natural environmental light, and modelled lighting environments, can give good results. In the case of light probes, we identified as two significant factors affecting performance the similarity between the light probe and the test environment, as well as the light probe's resolution. Regarding modelled lighting environment, similarity with the test environment was again identified as a significant factor.

Related papers

• Do Image Editing Models Understand Lighting? [52.657799986035464]
  We introduce the 3D-anchored Light Probe (3DLP) benchmark, for which we have captured a new high-fidelity HDR dataset of real-world lighting changes.<n>The dataset consists of 1K image pairs of diverse indoor scenery in which light probes are physically turned on and off.<n>We evaluate a range of state-of-the-art image editing models by measuring how well their light probe edits align with reality.
  arXiv  Detail & Related papers  (2026-06-25T08:20:34Z)
• Robust Single-shot Structured Light 3D Imaging via Neural Feature Decoding [33.013553875034795]
  We consider the problem of active 3D imaging using single-shot structured light systems.<n>Traditional structured light methods typically decode depth correspondences through pixel-domain matching algorithms.<n>Inspired by recent advances in neural feature matching, we propose a learning-based structured light decoding framework.
  arXiv  Detail & Related papers  (2025-12-16T02:47:38Z)
• 3DPR: Single Image 3D Portrait Relight using Generative Priors [101.74130664920868]
  3DPR is an image-based relighting model that leverages generative priors learnt from multi-view One-Light-at-A-Time (OLAT) images.<n>We leverage the latent space of a pre-trained generative head model that provides a rich prior over face geometry learnt from in-the-wild image datasets.<n>Our reflectance network operates in the latent space of the generative head model, crucially enabling a relatively small number of lightstage images to train the reflectance model.
  arXiv  Detail & Related papers  (2025-10-17T17:37:42Z)
• ROGR: Relightable 3D Objects using Generative Relighting [71.35020300131261]
  We introduce ROGR, a novel approach that reconstructs a relightable 3D model of an object captured from multiple views.<n>We train a lighting-conditioned Neural Radiance Field (NeRF) that outputs the object's appearance under any input environmental lighting.<n>We evaluate our approach on the established TensoIR and Stanford-ORB datasets, and showcase our approach on real-world object captures.
  arXiv  Detail & Related papers  (2025-10-03T16:35:22Z)
• Neural Appearance Modeling From Single Images [3.3090362820994526]
  We propose a material appearance modeling neural network for visualizing plausible, spatially-varying materials under diverse view and lighting conditions. Our network is composed of two network stages: a network that infers learned per-pixel neural parameters of a material from a single input photograph, and a network that renders the material.
  arXiv  Detail & Related papers  (2024-06-08T18:56:03Z)
• WinSyn: A High Resolution Testbed for Synthetic Data [41.11481327112564]
  We present WinSyn, a unique dataset and testbed for creating high-quality synthetic data with procedural modeling techniques. The dataset contains high-resolution photographs of windows, selected from locations around the world, with 89,318 individual window crops showcasing diverse geometric and material characteristics. We evaluate a procedural model by training semantic segmentation networks on both synthetic and real images and then comparing their performances on a shared test set of real images.
  arXiv  Detail & Related papers  (2023-10-09T20:18:10Z)
• Relightify: Relightable 3D Faces from a Single Image via Diffusion Models [86.3927548091627]
  We present the first approach to use diffusion models as a prior for highly accurate 3D facial BRDF reconstruction from a single image. In contrast to existing methods, we directly acquire the observed texture from the input image, thus, resulting in more faithful and consistent estimation.
  arXiv  Detail & Related papers  (2023-05-10T11:57:49Z)
• Extracting Triangular 3D Models, Materials, and Lighting From Images [59.33666140713829]
  We present an efficient method for joint optimization of materials and lighting from multi-view image observations. We leverage meshes with spatially-varying materials and environment that can be deployed in any traditional graphics engine.
  arXiv  Detail & Related papers  (2021-11-24T13:58:20Z)
• DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer [78.91753256634453]
  We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiables. In this work, we propose DIBR++, a hybrid differentiable which supports these effects by combining specularization and ray-tracing. Compared to more advanced physics-based differentiables, DIBR++ is highly performant due to its compact and expressive model.
  arXiv  Detail & Related papers  (2021-10-30T01:59:39Z)
• Sim2Air - Synthetic aerial dataset for UAV monitoring [2.1638817206926855]
  We propose to accentuate shape-based object representation by applying texture randomization. A diverse dataset with photorealism in all parameters is created in a 3D modelling software Blender.
  arXiv  Detail & Related papers  (2021-10-11T10:36:33Z)
• PX-NET: Simple and Efficient Pixel-Wise Training of Photometric Stereo Networks [26.958763133729846]
  Retrieving accurate 3D reconstructions of objects from the way they reflect light is a very challenging task in computer vision. We propose a novel pixel-wise training procedure for normal prediction by replacing the training data (observation maps) of globally rendered images with independent per-pixel generated data. Our network, PX-NET, achieves the state-of-the-art performance compared to other pixelwise methods on synthetic datasets.
  arXiv  Detail & Related papers  (2020-08-11T18:03:13Z)
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition [67.9464567157846]
  We propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties. Our experiments confirm that a joint treatment of rendering and decomposition is indeed beneficial and that our approach outperforms state-of-the-art image-to-image translation baselines both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-06-29T12:53:58Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)
• Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials [65.95928593628128]
  We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique. Our algorithm is suitable for perspective cameras and nearby point light sources.
  arXiv  Detail & Related papers  (2020-01-18T12:26:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.